JP2005031987A - Content layout management system and content layout management program for content delivery system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a content layout management system for a content delivery system capable of reducing the management cost required for the layout management of contents by autonomously performing the layout management of the contents within the content delivery system. <P>SOLUTION: This device comprises an access monitoring device 2011 for managing a content access index that is a request from a client to which a content is delivered or the result of the delivery on a content delivery system basis. Based on the content access index that is an index showing the access state to the contents managed on the content delivery system basis, the layout of contents is autonomously determined and executed on the content delivery system basis. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a content arrangement management system and a content arrangement management method for autonomously arranging contents in a content distribution system connected to a wide area in a computer network.
[0002]
[Prior art]
In a computer network, a content distribution server that distributes content to clients is connected to a plurality of clients that receive the content via the network. For this reason, the content distribution server described above may receive connection requests from a plurality of clients at the same time, or may distribute contents to a plurality of clients at the same time. However, when the content distribution server does not have sufficient processing capability as a computer that responds to a request from the client or content distribution capability, the quality of the content received by the client deteriorates.
[0003]
Specifically, response time degradation, video disturbance, and the like can be mentioned. Furthermore, when the load such as the connection request processing or content distribution described above is further increased, content distribution may be stopped. Note that the contents here refer to all objects accessed from the user through the network. Specific examples include, but are not limited to, Web documents and images, video and audio streaming data, general data files, programs executed in response to requests from clients, and the like. is not.
[0004]
In order to cope with these situations, as shown in FIG. 26, a plurality of content distribution servers 103 to 107 are prepared, and a method of distributing loads from the clients 109 to 111 is often employed. A server group composed of a plurality of servers is called a cluster 102. In addition, a content distribution server that requires a system administrator to explicitly place a copy of a content on multiple servers is called a mirror server. As a result of relaying the content requested by the client, the copy content is passively stored. This server is called a cache server. The content copied by the system administrator or the content copied as a result is called a replica. 26, in the content distribution system 112, a plurality of content distribution servers 103 to 107 constitute a cluster 202, and a plurality of clients 109 to 111 connected to the network 113 are connected to the content via the load balancer 101. Connected to distribution servers 103-107.
[0005]
A device that allocates requests from the clients 109 to 111 to each mirror server or cache server is called a load balancer 101 (or load balancer). Such a group of content distribution apparatuses including the cluster 102, the load distribution apparatus 101, and the like is referred to as a content distribution system 112.
[0006]
Each content distribution server is connected to a network such as NAS or SAN as shown in FIG. 27 in addition to the configuration as shown in FIG. 26 provided with an external storage device for storing content. There are also configurations using shared storage. 27, in this content distribution system 212, content distribution servers 203 to 207 and shared storages 213 and 214 are connected by a network 215 to form a cluster 202. A plurality of clients 209 to 211 connected to the network 208 are connected to the content distribution servers 203 to 207 via the load balancer 201.
[0007]
The load distribution within the content distribution system 112 described above is load distribution within a geographically close region. Therefore, when a client geographically far from the content distribution system receives distribution, the response time is delayed. In order to cope with such geographical distribution of distribution requests, a plurality of content distribution systems are geographically distributed. The plurality of content distribution systems are connected via a network to form a wide area content distribution system.
[0008]
FIG. 28 shows a load distribution configuration in the wide area content distribution system. Here, content replication is performed between the plurality of content distribution systems 307 to 310. In this wide area content distribution system, a content request from a client is guided to one of the content distribution systems. At that time, the content request can be first guided to the content distribution system closest to the client. However, when the content does not exist in the nearest content distribution system, it is guided to another content distribution system. In FIG. 28, in the wide area content distribution system 320, a plurality of content distribution systems 301 to 305 are configured by clusters 301B to 305B and load distribution apparatuses 301A to 305A, respectively, and a plurality of clients 314 to 316 are connected to the network 312. A plurality of clients 317 to 319 are connected to the wide area content distribution system via the network 313.
[0009]
In the content distribution system and the wide area content distribution system described above, the performance of the system varies greatly depending on the content replica arrangement method. However, the optimum content distribution method and content arrangement method for the load distribution system in the content distribution system and the wide area load distribution system are different. In the wide area load distribution system as shown in FIG. 28, in consideration of the limited network bandwidth in the network including the content distribution systems 301 to 305 as the bases and the clients 314 to 319 and the complexity of the network topology, It is determined whether it is optimal to place content in the content distribution system.
[0010]
On the other hand, in a content distribution system having a sufficient network bandwidth and a known topology, content arrangement optimization for the purpose of load distribution of the server and disk device is performed. For this reason, it is necessary to perform the arrangement in consideration of the distribution performance of the content distribution server, the finite storage capacity such as the hard disks or RAIDs included in the clusters 301B to 305B, the limit throughput, and the like together with the request guidance method for load distribution.
[0011]
In many content distribution systems, a plurality of different contents are arranged, and the access frequency from the client differs depending on the contents. That is, the content distribution system includes popular content and content that is not. If these individual contents are placed in the content distribution system without discrimination, the server's distribution performance will reach the limit for popular contents, and the client may experience quality degradation. Occur. Or a lot of disk space may be occupied by unpopular content. Therefore, it is important for the content arrangement method in the content distribution system to consider the load distribution and the popularity of the content.
[0012]
As a content arrangement management method in a wide area load distribution system considering content popularity, for example, there is a method described in Japanese Patent Laid-Open No. 2001-67377 (Patent Document 1). This reserves resources based on the prediction of the usage status of computer resources, and manages the arrangement of contents within the content distribution system. Also, the document “A dynamic object replication and migration protocol for an Internet hosting service,” (Distributed Computing Systems, 1999. Proceedings. The content arrangement is determined according to the popularity of the contents, but it relates to a content arrangement management method in the wide area load distribution as in the above publication.
[0013]
Next, a content arrangement management method in the load distribution system in the content distribution system considering content popularity will be described. This method differs depending on whether the server constituting the cluster is a mirror server or a cache server. As a method of performing arrangement according to the popularity of content using a mirror server, there is a method in which a content distribution system administrator explicitly performs it. In other words, this is a method of arranging content through human judgment, but this increases system management costs. Furthermore, when the content to be placed on the mirror server increases and sufficient capacity is not secured for the finite disk capacity that the mirror server has, the system administrator explicitly needs little like the content placement. The replica needs to be deleted. This imposes a heavy burden on the administrator, which increases the management cost.
[0014]
On the other hand, in a system using both a cache server and a load balancer, there is no need for an administrator to explicitly place content. However, since the request from the client is automatically and fairly distributed to each cache server by the load balancer, almost the same content exists in all the cache servers. This leads to inefficiency of the external storage device of the cache server. This is because the capacity of the external storage device is saved if the content handled by each server is different while the load balancer allocates a request from a client to the server.
[0015]
Furthermore, the fact that content is automatically arranged means that the arrangement reflecting the administrator's policy becomes impossible. Furthermore, when the disk capacity of the cache server approaches the capacity limit due to an increase in content, the content is deleted by a deletion algorithm such as LRU (Least-Recently-Used) or LFU (Least-Frequency-Used). This is performed only for each server, and content replica deletion is not performed after monitoring the content arrangement status of the entire cluster.
[0016]
In a load distribution system in a content distribution system using a mirror server, the popularity of content (for example, the number of requests and the number of distribution results) and the load that the content gives to the system accompanying the distribution (for example, the average number of simultaneous distributions, the server distributes the content) In order to perform content arrangement according to the average number of bits output for the purpose, the system administrator needs to explicitly determine the arrangement of the content.
[0017]
As a content arrangement management system that does not need to be explicitly performed, there is a method described in Japanese Patent Laid-Open No. 2002-259354 (Patent Document 2), which manages the arrangement of contents to be distributed on a server basis. .
[0018]
FIG. 29 shows an outline of this system. The load such as the number of accesses to each of the mirror servers 403 to 407 is monitored by the control server 416, and the load balancer 401 allocates a request from the client to the mirror servers 403 to 407 in response to the reference result of the control server 416. The unit of the monitored and managed content is a content group in which a plurality of contents are grouped. In addition, the unit in which the content group is arranged is a server unit. Therefore, when the load on the content group increases, the load is distributed by increasing or decreasing the number of servers. FIG. 30 shows a correspondence table between the content group 501 and the server 502 in charge of distribution thereof. Although each content type can be arranged in a plurality of servers, it is difficult to allocate in units of contents as shown in FIG. 30 and to arrange a plurality of contents on the same server.
[0019]
This can be said to be load distribution in units of servers in charge of distribution of content groups. In addition, it is necessary to prepare servers necessary for dealing with the increase / decrease of the load from the normal time. This means that an inactive spare server is prepared, which leads to a reduction in the operation rate. Further, this method does not have a function of automatically reducing the number of content replicas when the capacity of the external storage device reaches the limit. Further, when there is no spare server, there is no function to reassign a server of a content group with a low load to a server of a content group with a high load. Furthermore, it is impossible to introduce a different management policy for each content.
[0020]
In the method described in Japanese Patent Laid-Open No. 2001-27965 (Patent Document 3), contents are managed in units of directories, and further, the method described in Japanese Patent Laid-Open No. 2002-63064 (Patent Document 4). Then, content management in directory units is autonomously performed by policy control. Here, control such as improving the hit rate is performed by automatically switching the replacement policy applied to the directory and automatically adjusting the allocation capacity between the parameters. However, policy management has not been attempted with respect to an arrangement policy for each content such as the addition of replicas for each content or behavior at the time of competition, and it cannot be said that this is a method for realizing content arrangement according to popularity.
[0021]
[Patent Document 1]
JP 2001-67377 A
[Patent Document 2]
JP 2002-259354 A
[Patent Document 3]
JP 2001-27965 A
[Patent Document 4]
JP 2002-63064 A
[Non-Patent Document 1]
“A dynamic object replication and migration protocol for an Internet hosting service”, Distributed Computing Systems, 1999. Proceedings. 19th IEEE International Conference on, pp. 101-113, (1999).
[0022]
[Problems to be solved by the invention]
The method described in Patent Document 1 is a content placement management method in a wide area load distribution system, and does not touch the placement management method in the content distribution system. For this reason, another content arrangement management method is required in the content distribution system.
[0023]
The method described in Patent Document 2 is a content arrangement management method for load distribution in a content distribution system, but is a content group in which a unit for disk management is set for each content provider. Furthermore, the allocation destination is a server unit. For this reason, there is no mechanism for assigning a plurality of content groups to the same server. Further, when there is no spare server, there is no function to reassign a server of a content group with a low load to a server of a content group with a high load. These will lead to a reduction in availability. There is no means for deleting a replica when the disk capacity of the mirror server reaches its limit. Therefore, a large disk capacity is required.
[0024]
Furthermore, in the methods described in Patent Document 3 and Patent Document 4, content arrangement management is performed in units of directories. This is also not arranged according to popularity in content units. Since the user shows interest in content units, it is desirable to realize an arrangement method in content units, not server units or directory units. In addition, when the number of accesses from clients increases and replicas are generated on the cache server or mirror server, the server's disk capacity limit is reached, so conventionally, replica deletion management that has been determined on a server basis is performed. Therefore, replica deletion management that was determined in cluster units was not performed.
[0025]
The present invention performs content arrangement management based on the popularity of content in units of content. There, the system administrator sets the policy so that the content is autonomously placed on the optimal server. In addition, when the disk capacity of the server reaches the limit, eviction between contents is performed, so the system administrator does not need to consider the disk capacity limit. Conventionally, there has been no method for simultaneously realizing the functions such as content allocation management according to the above-mentioned popularity, allocation management in units of content, and eviction in consideration of server disk capacity.
[0026]
The problems in the prior art described above are summarized as follows.
[0027]
The first problem is that in the content arrangement management method in the content distribution system using a mirror server, the system administrator needs to explicitly determine the content arrangement.
[0028]
This is because the expense and labor cost paid to the system administrator who performs content arrangement management are large, leading to an increase in system cost. In order to solve this, it is necessary to introduce a mechanism for autonomously managing the arrangement.
[0029]
The second problem is that in the content distribution management method in the content distribution system using the cache server, it is impossible to manage the arrangement according to the popularity of the content, and the use efficiency of the external storage device such as a disk is lowered. It is in. In addition, content may be deleted when the amount of content stored in the cache server increases and the disk capacity limit is exceeded. However, content is deleted from all cache servers almost at the same time. There is also a problem that the number of contents cannot be arranged.
[0030]
The reason is that when the load balancer distributes a request from a client to a cache server, the content is distributed for the purpose of making the server load uniform, such as a round robin algorithm, so that a replica of the content is generated on all cache servers. It is. It is necessary to manage the arrangement according to the popularity of content not in cache servers but in content distribution systems. By managing the arrangement in units of content distribution systems, it is possible to delete unpopular content first when the popularity among a plurality of content differs.
[0031]
The third problem is that, conventionally, content placement management is performed in units of content groups or directories, and placement management in units of content cannot be performed by introducing a policy in units of content.
[0032]
The reason for this is that using a management unit that includes multiple contents such as content groups and directory groups means that content with various attributes and contents with different popularity are placed together, and discs are used for content that is not popular. Causes a decrease in disk utilization efficiency resulting from the fact that the disk is occupied. In general, it is the content unit that the user is interested in. Therefore, concentration of access should be performed on a content basis. In addition, it is important to provide a policy for each content unit because different attributes exist for each content unit. For example, since news and movies have different attributes, it is necessary to set a policy in consideration of priority, viewable time, etc. for each content.
[0033]
The fourth problem is that in the conventional method, the operating rate of the server is low or it is necessary to prepare a spare standby server. This problem is due to the fact that the unit for managing the arrangement on the disk is a group of contents grouped by a content provider or the like, and the allocation destination is a server unit. Since there is no mechanism for assigning a plurality of content groups or content to the same server, the granularity of load distribution becomes coarse. Therefore, it is necessary to prepare a relatively large number of servers. Further, when there is no spare server, there is no function to reassign a server of a content group with a low load to a server of a content group with a high load. These things cause a reduction in the availability rate of the server. In order to solve this problem, the unit for managing the arrangement on the disc should be a finer unit such as content, a plurality of content groups, or a mechanism for allocating content to the same server, content with low load It is necessary to introduce a mechanism for reassigning a group of servers to a server of a content group with a high load.
[0034]
A fifth problem is that a conventional content distribution system does not have a function that can cope with suddenly popular content.
[0035]
The reason for this is that, in the content arrangement control method using a policy, there is a certain time until the optimum arrangement state is reached based on the state when the number of accesses or the like is monitored. If access concentrates on the content in the meantime, content will be delivered from other content delivery systems. In this case, the response will be slow, causing network congestion with other content delivery systems. There is a risk of causing problems such as. However, the methods using the conventional cache server cannot solve the first to fourth problems. There is a need to construct a system that incorporates both a mechanism for solving the first to fourth problems and a mechanism that can cope with suddenly popular content.
[0036]
OBJECT OF THE INVENTION
A first object of the present invention is to perform content arrangement management in a content distribution system autonomously, thereby reducing a management cost required for content arrangement management, and a content arrangement management system and content arrangement management in a content distribution system To propose a program.
[0037]
The second object of the present invention is to arrange or delete content according to conditions set for each content such as popularity of the content, so that a limited number of discs of the content distribution system can be used. The object is to propose a content arrangement management system and a content arrangement management program in a content distribution system that improves the use efficiency.
[0038]
A third object of the present invention is to propose a content arrangement management system and a content arrangement management program in a content distribution system that introduces a policy in units of contents and realizes autonomous arrangement in units of contents.
[0039]
A fourth object of the present invention is to propose a content arrangement management system and a content arrangement management program in a content distribution system that increases the operating rate of a server and eliminates the need for a standby server.
[0040]
A fifth object of the present invention is to provide a content arrangement management system in a content distribution system that achieves the first to fourth objects and at the same time realizes high-response and high-throughput distribution for suddenly popular content, and It is to propose a content arrangement management program.
[0041]
[Means for Solving the Problems]
The present invention that achieves the above object is a content placement management system that manages the placement of content in a content delivery system, and requests from the client for the content, the delivery results of the content accompanying the request, and conversion from them A content access index, which is an index indicating the load of the content to be applied to the system, for each content distribution system, and a cluster in the content distribution system based on the content access index of the access monitoring apparatus It is characterized by autonomously allocating the content in charge of distribution to the server and creating or deleting replicas on the disk.
[0042]
The content arrangement management system of the present invention according to claim 2 is characterized in that the number of replicas of the content in the content distribution system is calculated from the content access index.
[0043]
The content arrangement management system of the present invention according to claim 3 is characterized in that a threshold used for calculating the number of replicas of the content in the content distribution system from a content access index is set for each content.
[0044]
The content arrangement management system of the present invention according to claim 4 is a value that varies depending on the number of replicas in the content distribution system as a threshold used when calculating the number of replicas of the content in the content distribution system from a content access index. Is set for each content.
[0045]
The content arrangement management system of the present invention according to claim 5 is characterized in that the number of servers in charge of distributing the content to the client in the content distribution system is calculated from the content access index.
[0046]
The content arrangement management system according to claim 6 of the present invention provides, for each content, a threshold value used when calculating the number of servers in charge of distributing the content to the client in the content distribution system from the content access index. It is characterized by setting.
[0047]
The content distribution management system of the present invention according to claim 7 uses the content distribution system as a threshold used when calculating the number of servers in charge of distributing the content to the client in the content distribution system from the content access index. A different value is set for each content depending on the number of servers in charge of distribution to the client.
[0048]
The content arrangement management system of the present invention according to claim 8 is configured to create a replica of the content on a predetermined disk in the content distribution system, and when it is determined that the allowable load value of the disk is exceeded, The content replica to be deleted from the disk in order to reduce the load on the disk is determined based on a content access index that is an index representing an access status to the content managed in units of the content distribution system. .
[0049]
In the content arrangement management system of the present invention according to claim 9, when the replica of the content is created on a predetermined disk in the content distribution system, it is determined that the load of the disk exceeds an allowable value, When a content replica is selected and deleted from the predetermined disk in order to reduce the load, the access status to the content managed in the content distribution system unit is checked for the content replica on the predetermined disk. The content disk index is calculated by dividing the content access index, which is the index to represent, by the number of replicas in the content distribution system of the content, and the content for which the replica is to be created is managed on a content distribution system basis For content A predicted content disk score obtained by dividing the content access index, which is an index representing the access status of the content, by the value of the number of replicas in the content distribution system plus the number of copies to be made from now, When a replica of content whose content disc score is lower than the content disc score is extracted as a deletion candidate, and all deletion candidates are deleted, even if a replica is expected, the load on the disc is not expected to exceed an allowable value, In order of increasing content disk core, the replica of the content is created after deleting the replica until it is predicted that the disk load will not exceed the allowable value even if the replica is created.
[0050]
According to the content arrangement management system of the present invention of claim 10, when a replica of content is created on a predetermined disk in the content distribution system, the load of the disk is the lowest or the allowable value of the remaining disk load is The largest disk is selected as a replica creation destination.
[0051]
The content arrangement management system of the present invention according to claim 11, when allocating a predetermined server in the content distribution system as a person in charge of distribution of the content, when it is determined that the load of the server exceeds an allowable value, The content replica to be removed from the server distribution person in order to reduce the server load is determined based on a content access index that is an index representing an access status to the content managed in units of the content distribution system. .
[0052]
The content arrangement management system of the present invention according to claim 12, when allocating a predetermined server in the content distribution system as a person in charge of distribution of the content, when it is determined that the load of the server exceeds an allowable value, When determining the content replica to be removed from the server's distribution staff in order to reduce the server load based on the content access index, which is an index indicating the access status to the content managed in the content distribution system unit, For a content replica that has already been assigned to the server, calculate the value of the content server score by dividing the content access index by the number of servers to which content distribution is assigned, , The content access index A value that is a predicted content server score is calculated by dividing the number of servers to which the distribution of tents is allocated by the value of the number of servers to be allocated, and the content server score is calculated from the predicted content server score. If replicas of low content are extracted as deletion candidates and deletion candidates are deleted, if it is expected that the server load will not exceed the allowable value even if replicas are created, replicas are created in ascending order of the content server scores. Even if the replica is deleted until it is predicted that the load on the server does not exceed the allowable value, the content replica is created.
[0053]
In the content arrangement management system according to the thirteenth aspect of the present invention, when a predetermined server in the content distribution system is allocated as a person in charge of content distribution, the load on the server is the lowest or the allowable value of the remaining server load is the highest. A large server is selected as a distribution assignment destination.
[0054]
The content arrangement management system of the present invention according to claim 14 is characterized in that a disk is connected to each of a plurality of servers in the content distribution system.
[0055]
The content arrangement management system of the present invention according to claim 15 is characterized in that the disks in the content distribution system are connected as shared disks that can be referred to from a plurality of servers.
[0056]
The content placement management system of the present invention according to claim 16 is characterized in that the content autonomous placement management content is set as a policy which is a combination of conditions and processing details at that time.
[0057]
The content arrangement management system of the present invention according to claim 17 is characterized in that the policy is set for each content.
[0058]
The content placement management system of the present invention according to claim 18 is a policy management device that sets the policy for each content, and creates a replica of the content on a server in the cluster while referring to the content access index and the policy, Alternatively, a content placement control device that controls the placement of content by executing deletion of a replica from a server in the cluster is provided.
[0059]
A content arrangement management system according to a nineteenth aspect of the present invention includes a content arrangement status management apparatus that manages content location information, and the content arrangement control apparatus refers to the content location information of the content arrangement status management apparatus. The content arrangement is controlled.
[0060]
The content arrangement management system of the present invention according to claim 20 sets the autonomous arrangement management contents of contents as a policy that is a combination of conditions and processing contents at that time, and (1) a fixed policy (condition: set) When selected as a deletion candidate during the period / action: not to be deleted from the disk or the target of reduction of the number of servers in charge of distribution), (2) new replica creation policy (condition: content during the set period The access index exceeds the set threshold value and the content replica is not in the distribution management system / action: execute processing for creating a replica), (3) replica addition policy (condition: during the set period) When the value obtained by dividing the content access index by the set threshold exceeds the number of replicas / Action: Another one in the distribution management system (4) Replica deletion policy (Condition: When the content access index falls below the set threshold during the set period / Action: The replica is set from the disk (5) replica eviction policy (condition: when selected as a deletion candidate during a set period / action: deleted from disk), (6) new distribution server allocation policy (condition) : During the set period, the content access index exceeds the set threshold value, and the content distribution server is not in the distribution management system / action: execute the process of assigning the distribution server, (7) distribution server Expansion policy (Condition: During the set period, the value divided by the threshold value that set the content access index is the distribution responsibility. When the number of servers is exceeded / Action: Process for increasing the number of servers in charge of distributing content in the distribution management system is executed), (8) Distribution manager reduction policy (Condition: Content within a set period) When the access index falls below a set threshold value / action: the number of servers in charge of distribution is reduced by the set number), it is selected and set.
[0061]
A content arrangement management system according to a twenty-first aspect of the present invention is characterized in that a proxy cache server is provided in the content distribution system, and a part of content is distributed via the proxy cache server.
[0062]
The content placement management system according to the present invention of claim 22 causes a server in the content distribution system to function as a proxy cache server, and indirectly executes a content replica creation or deletion instruction by guidance of a load balancer. It is characterized by.
[0063]
The content placement management system according to the present invention of claim 23 monitors the disk load, and if there is a disk whose observed disk load exceeds the allowable value of the disk load, the replica of the content having the largest content access index is found. It is created in the content distribution system.
[0064]
The content arrangement management system of the present invention according to claim 24 monitors the server load, and if there is a server whose observed server load exceeds the allowable value of the server load, distribution of a replica of the content having the largest content access index It is characterized by increasing the number of servers in charge.
[0065]
The content arrangement management system of the present invention according to claim 25 is characterized in that an initial value of a content access index set when a request for content or a delivery result is generated for the first time is a dynamically changing reference value.
[0066]
The content arrangement management system of the present invention according to claim 26 is characterized in that the initial value of the observation server load of the server newly restarted or newly added is set as a dynamically changing reference value.
[0067]
The content allocation management system according to a twenty-seventh aspect of the present invention is characterized in that an initial value of an observed disk load of a disk that has been restarted or newly added is set as a dynamically changing reference value.
[0068]
The present invention of claim 28 is a content arrangement management program that is executed on a computer and manages the arrangement of contents in a content distribution system, wherein a request from the client to the contents, and the distribution results of the contents accompanying the request The content access index, which is an index indicating the load that the content converted from them has on the system, is managed for each content distribution system, and based on the content access index, to the clustered servers in the content distribution system The function of autonomously allocating content in charge and creating and deleting replicas on disk is executed.
[0069]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
[0070]
(First embodiment)
Next, a first embodiment of the present invention will be described in detail with reference to the drawings.
[0071]
Referring to FIG. 1, a system according to the first embodiment of the present invention includes a plurality of clients (three clients in FIG. 1) 2001, 2002, 2003 viewing content, and an access network 2004 in which users connect via a network. A content distribution system 2010 connected to the access network 2004 by a network, and a network 2005 for connecting the content distribution system 2010 to an external network or other content distribution systems.
[0072]
Here, the content distribution system 2010 includes a cluster 2020 composed of a group of servers to which disks are connected, a load balancer 2012 that guides user requests while distributing the load to these servers and external content distribution systems, In order to manage a content arrangement status management apparatus 2014 that manages location information, an access monitoring apparatus 2011 that monitors a user's request status for content, a content arrangement control apparatus 2030 that controls content arrangement, and a content arrangement policy (policy) The policy management apparatus 2015 includes a management terminal 2006 for inputting the policy and setting the content distribution management system.
[0073]
The load distribution apparatus 2012 and the access monitoring apparatus 2011 are connected so that a request for content from a user (clients 2001 to 2003) using the content distribution system 2010 passes.
[0074]
The load balancer 2012 and the cluster 2020 are connected by a local network 2013. The load balancer 2012 is also connected to an external content distribution system via a network 2005. The content arrangement status management apparatus 2014, content arrangement control apparatus 2030, policy management apparatus 2015, management terminal 2006, cluster 2020, load distribution apparatus 2012, and access monitoring apparatus 2011 are connected to each other by a local network 2013 in the system.
[0075]
The cluster 2020 includes a server group to which disks are connected (a server 2021 (server (1)) to which a disk 2024 is connected, a server 2022 (server (2)) to which a disk 2025 is connected, and a server 2023 to which a disk 2026 is connected. In addition to the three servers (server (3)), the server status monitoring device 2027 that monitors and manages the usage status and load of each server 2021 to 2023 in the cluster, and the usage status of each disk 2024 to 2026 And a disk status monitoring device 2028 for monitoring and managing the load status.
[0076]
Here, the server status and the disk status are described as a configuration in which the server status monitoring device 2027 and the disk status monitoring device 2028 are centrally managed in one place in the cluster 2020. The apparatus 2028 may be configured to perform distributed management for each server or disk. As shown in FIG. 1, the advantage of centralized management is that both the server status and disk status inquiries performed by the content arrangement control apparatus 2030 can be performed at once.
[0077]
The server status monitoring device 2027 manages the observation server load (an index indicating the processing amount of the server) of each server 2021 to 2023 and the server limit load value that is a limit value of the load. The server load index may be a so-called server load, the server CPU usage rate, the cumulative number of accesses observed for each server, the number of accesses for a certain period, the current number of simultaneous connections, the current server throughput (from server to client It is possible to define various indicators such as a value obtained by dividing the total amount of data transmitted to the unit time).
[0078]
Moreover, you may hold | maintain the set of the observation server load mentioned above and a server load limit value regarding a some parameter | index. Furthermore, you may define the new parameter | index obtained by some calculation formula from some indexes as an observation server load.
[0079]
FIG. 2 shows an example in which the server load index is the CPU usage rate of the server. The server limit load value is generally set when the system is started, but can also be set from the management terminal 2006 as needed. Regarding the observation server load, it is possible to select a method of measuring and updating the load at a set period, a method of updating in an event-driven manner every time the content arrangement status and the user access status change, and the like. Furthermore, it is also possible to use both a method of measuring and updating a load at a set cycle and a method of updating in an event-driven manner every time the content arrangement status and the user access status change.
[0080]
The disk status monitoring device 2028 manages the observed disk load of each of the disks 2024 to 2026 and a disk limit load value that is a limit value of the load. As an index representing the disk load, the capacity of the used disk, the operation rate for each disk, the number of writes, the number of reads, the current read / write throughput, and the like can be considered.
[0081]
FIG. 3 shows an example in which the used disk capacity (unit B) is used as an index of the disk load. The disk limit load value is generally set when the system is started, but can also be set from the management terminal 2006 as needed. Regarding the observation disk load, it is desirable to update in an event-driven manner every time the content arrangement status changes, but in consideration of reducing the processing load on the system, the load is measured and updated at a set cycle. It is good as a method. Further, a method of measuring and updating a load at a set cycle and a method of performing event-driven updating each time a content arrangement status or a user access status changes may be used in combination.
[0082]
As described above, the load balancer 2012 is connected to the access network 2004 of the clients 2001 to 2003, is connected to the cluster 2020 and the local network 2013, and is connected to an external network and other content distribution systems via the network 2005. Yes.
[0083]
When the request from the clients 2001 to 2003 is received, the requested content is stored in a disk connected to a server in the cluster, and if distribution from the server is possible, the load balancer 2012 sends the request to the disk. Guide the content from the server to the client.
[0084]
For example, when the content A requested by the client is stored in the server (1) 2021 as shown in FIG. 4, the client request is guided to the server (1) 2021, and distribution from the server (1) 2021 to the client is performed. Establish.
[0085]
Further, when the content B requested by the client is placed in both the server (1) 2021 and the server (2) 2022, the load balancer 2012 selects a round robin or a server with a low server load, etc. In accordance with various control methods, the server requests are distributed to one of the servers in a manner that distributes the server load, and distribution from the server is established.
[0086]
Further, when the content C requested by the user is not in the server in the cluster 2020 but in the server of the external content distribution system 2310, the load balancer load distribution device 2012 sends the client request to the external content distribution system 2310. And the distribution from the server in the content distribution system 2310 is established.
[0087]
The load balancer 2012 needs to manage candidates for destination of client requests for each content. For example, in the above-described example, as illustrated in FIG. 5, it is necessary to manage the server 1 for the content A and the server 1 and the server 2 for the content B as guidance destination candidates.
[0088]
With respect to the content C, in FIG. 5, the content distribution system 2310 is managed as a guide destination candidate (that is, the load distribution device 2012 is left to which server in the content distribution system 2310 to be guided). )
[0089]
There are many other methods of guiding to other content distribution systems when they are not in the server in the own content distribution system. Specifically, there is a method in which a server holding the content of the content distribution system 2310 is managed as a candidate and is selected and guided. In addition, there is another method in which a system for determining a guide destination exists and the system is registered as a guide destination. Here, a method of guiding to another content distribution system when there is no server in the content distribution system is not specified. Since the candidate for guiding the client request is determined depending on the arrangement state of the content, the management information needs to be updated according to the arrangement of the content.
[0090]
Further, since the request should not be guided to a server that cannot be distributed, it is also necessary to manage whether each of the induction candidate destination servers can be distributed (Active) or cannot be distributed (Down). For example, as shown in FIG. 4, it is assumed that the content D is placed on both the server 2 and the server 3, but the server 2 is active but the server 3 is down. At this time, when a request for the content D is received from the client, the load balancer 2012 checks the server distributable status, guides the client request to the server 2, and establishes the distribution from the server.
[0091]
Note that the server distribution status does not necessarily have to be managed independently by the load balancer 2012, and there is no problem with the method of referring to the information managed by the previous server status monitoring device 2027. In the following description, it is assumed that information managed by the server status monitoring apparatus 2027 is referred to.
[0092]
The content arrangement status management apparatus 2014 manages the location information of each content in a cluster (which content replica (copy) is located on which server disk). Furthermore, the attribute information of each content is managed simultaneously.
[0093]
Specifically, the server load (content server load) generated as a result of placing content on the server, the disk load (content disc load) generated as a result of placing content on the disc, and an index (content access) indicating the access status to the content Index), the number of replicas indicating the number of replicas on the server's disk in the cluster, a content disk score indicating the value of the content described later, and the like as attributes.
[0094]
For example, FIG. 6 shows a CPU usage rate required for one distribution as a content server load, a content size as a content disk load, and an average number of simultaneous distributions as a content access index, in addition to a placement server field for indicating content location information. In this example, the content disk index is a value obtained by dividing the content access index by the number of replicas. These pieces of information must be updated and managed in accordance with changes in content placement performed by the content placement control apparatus 2030, requests from users, and distribution results to users.
[0095]
The access monitoring apparatus 2011 manages a content access index that is an index indicating a load on the system by a request for a content from a client, a delivery result accompanying the content, or a content converted from these requests, through the content distribution system 2010.
[0096]
As this content access index, for example, the cumulative number of requests for each content, the number of requests for each content for a certain period, the number of simultaneous distributions for each content, the number of bytes of distribution, the number of cumulative distributions for each content, the certain period It is possible to define the number of distribution results for each content. Further, as the content access index, a plurality of indices among the above-mentioned indices may be held, or one obtained from some index by some calculation formula may be used as one content access index. Further, since the content access index is information managed for each content, it may be managed together with the management information of the content arrangement status management apparatus 2014.
[0097]
FIG. 6 shows an example in which the management information of the content arrangement status management apparatus 2014 is collected using the average simultaneous distribution number as a content access index. Further, it is necessary to update the content disc score simultaneously with the update of the content access index.
[0098]
These updates may be event-driven for each client request or delivery completion, but it is not necessary to update to event-driven, and batch updates such as periodic updates are also possible. It is.
[0099]
The content placement control device 2030 that controls the placement of content selects content that creates a replica (copy) on a server in the cluster or content that removes a replica (copy) from a server in the cluster, based on the access status or the like The arrangement target selection unit 2032 to perform, the arrangement server selection unit 2034 to select a server for creating a replica, and a replica of content to be deleted from the disk when there is a possibility that the disk limit load value of the disk will be exceeded. A deletion target selection unit 2033 to be selected, and a content arrangement determination unit 2031 that performs functions of the above-described constituent elements to perform content mirroring and deletion execution instructions are provided. The function of each of these components will be described in the description of the operation described later.
[0100]
In the content placement control apparatus 2030, the placement target selection unit 2032 receives the content access index from the access monitoring apparatus 2011, and further refers to the management information of the policy management apparatus 2015, and takes action on the content according to the value of the content access index. Specifically, a decision is made to mirror the content, delete it, or do nothing. What value is selected and what selection is made can be set uniquely in the system, or can be set for each content or each content group. The policy management apparatus 2015 receives this setting input from the management terminal 2006 and manages it.
[0101]
Most specifically, a pair of a condition and an action (processing content) called a policy is set for each content. Here, the case of setting for each content will be described as an example. Assuming that the content access index is the average number of simultaneous distributions, for example, in the case of content A shown in FIG. 7, when the average number of simultaneous distributions exceeds 200 and there are no replicas in the cluster (number of replicas = 0) (condition 1) The content is mirrored on any of the servers (action 1). When the average number of simultaneous distributions divided by 100 exceeds the number of replicas in the current cluster (Condition 2), one more replica is added in the cluster (Action 2), etc. it can.
[0102]
Further, for example, as in content B in FIG. 7, when the average number of simultaneous connections falls below 10 (condition 1), all replicas of content B in the cluster are deleted (action 1), etc. be able to.
[0103]
Each of the above-described components is realized by a program-controllable computer processing unit (CPU), and content that will be described in detail below when the content placement control device 2030 executes a content placement management program for performing content placement management. Manage the placement of This content arrangement management program is stored in a magnetic disk, a semiconductor memory, or other recording medium, and is loaded from the recording medium into the memory of the computer processing unit (CPU) of the content arrangement control device 2030, which is each constituent route. Each function necessary for management is realized by controlling the operation of the unit.
[0104]
(Operation of the first embodiment)
Next, the operation when the content distribution system according to the present embodiment updates the content arrangement status will be described in order. This update of the arrangement status is a feature of the present invention that is differentiated from the prior art.
[0105]
The arrangement target selection unit 2032 receives the content access index from the access monitoring apparatus 2011, and further refers to the management information of the policy management apparatus 2015, and determines the action for the content, specifically the content, according to the value of the content access index. Decide whether to mirror, delete, or do nothing.
[0106]
When the execution of the action is determined, the arrangement target selection unit 2032 instructs the content arrangement determination unit 2031 to execute the action. Specifically, the action is execution of mirroring of a certain content or execution of deletion. This will be explained in the following order.
[0107]
First, processing details when the instruction is a mirror of content will be described with reference to the flowchart of FIG.
[0108]
The content placement determination unit 2031 gives an instruction from the placement target selection unit 2032 regarding the identifier of the content to be mirrored and the number of new replicas to be created in the cluster (the number of copies to be newly created on a disk connected to the server). Receive (step 2800).
[0109]
The placement target selection unit 2032 determines the identifier of the content to be mirrored and the number of new replicas to be created in the cluster while referring to the information managed by the access monitoring device 2011 and the information managed by the policy management device 2015. ing. Here, for example, as shown in FIG. 7, if the content access index is “200 or more” and the number of replicas existing in the cluster is “0”, as shown in FIG. Suppose that one server is set to create a replica. Then, as shown in FIG. 24, it is assumed that the content access index of content A is “300”. At this time, the arrangement target selection unit 2032 instructs the content arrangement determination unit 2031 to specify “A” as the identifier of the content to be mirrored and “1” as the number of new replicas to be created in the cluster.
[0110]
The content placement determination unit 2031 initializes the placement destination determination server list (sets it to an empty state), and further initializes the number of placement destination servers to “0” (step 2801). Then, referring to the management information of the server status monitoring device and the management information of the content arrangement status management device, a server that meets the set criteria is registered in the candidate server list (step 2802). As an example of the reference set here, it is possible to use a server that does not contain content to be mirrored and is active.
[0111]
Then, the following processing is executed until the candidate server list becomes empty (step 2803) or until the number of placement destination servers reaches the number of new replicas instructed from the placement target selection unit 2032 (step 2804).
[0112]
First, one server is selected by the placement server selection unit 2034 from the candidate server list (step 2805). The following methods are conceivable as selection methods at this time.
[0113]
Method 1: Select the server with the smallest observation server load value.
[0114]
Method 2: Select the server with the smallest observed disk load value.
[0115]
Method 3: Select from among servers whose residual disk load is larger than the content disk load of the mirrored content by Method 1 or Method 2. If there is no server having enough disk space for placing content, a method such as selecting a server by either method 1 or method 2 can be adopted.
[0116]
A server is selected according to any of the above methods or some other algorithm.
[0117]
The content arrangement determination unit 2031 inquires of the disk status monitoring apparatus 2028 about the disk status of the server selected in step 2805 and the content arrangement status management apparatus 2014 about the content disk load of the mirrored content. Depending on whether content is placed on the server or not expected to exceed the disk limit load value, specifically, the total of the observed disk load and the mirrored content disk load exceeds the disk limit load value. The process branches depending on whether or not it exceeds (step 2807).
[0118]
If it is predicted that the disk limit load value will not be exceeded, the process proceeds to step 2824 described later.
[0119]
Here, a specific example when the disk limit load value is exceeded is shown. The disk capacity is an index of the disk load, and the content size is an index of the content disk load. Assume that the disk load, that is, the already used disk capacity is 1.8 GB, and the total size of the contents to be mirrored is 0.8 GB. If the disk limit load value, that is, the total available disk capacity is 2.0 GB, it is determined that the disk limit load value is exceeded. In this case, it is necessary to first delete any content from the server disk by 0.6 GB or more.
[0120]
Processing when it is predicted that the disk limit load value will be exceeded will be described. The content arrangement determination unit 2031 inquires of the deletion target selection unit 2033 whether there is content that can be deleted (step 2809). If it is determined that there is no content that can be deleted (step 2810), the process proceeds to step 2830 and later.
[0121]
When it is instructed that there is content to be deleted, after these contents are deleted, it is predicted whether or not the server load will exceed the server limit load value after the mirrored replica is created on the server (step 2811). ).
[0122]
In particular,
(Expression 1): Server load− (server load given by content listed as deletion target) + server load expected to be given by mirror target content> Check whether server limit load value is satisfied.
[0123]
If the relationship of (Expression 1) is satisfied, it is determined that the server load exceeds the server limit load value (step 2812), and the process proceeds to step 2830 and later described.
[0124]
Here, a specific example in the case where the server limit load value is exceeded is shown. Here, the CPU usage rate is an index of server load.
[0125]
As shown in FIG. 2, the observation server load of the server (1) 2021 is currently “80%”, and the server limit load value is “90%”. Then, it is assumed that the deletion target is the content B, the mirror target is the content C, and the mirror destination server is the server (1) 2021, and each content server load is “1.0%”, as shown in FIG. It is assumed that “1.5%” is set.
[0126]
The server load given to the content listed as the deletion target is “20%” because it is calculated by the product of the content server load (CPU usage rate / distribution) and the content access index (average number of simultaneous distributions). Similarly, the server load expected to be given by the mirrored content is calculated as 30%. As a result, the left side of (Equation 1) is 85-20 + 30 = 95%, which exceeds the server limit load value “90%”, and thus is determined to exceed the server limit load value.
[0127]
If it is determined that the relationship of (Equation 1) does not hold, that is, the server load does not exceed the server limit load value, the content selected by the deletion target selection unit 2033 as the deletion target is deleted from the server. (Step 2813). Note that the processing of step 2811 and step 2812 is not necessarily performed.
[0128]
Next, mirroring of the content to be mirrored (copying of content to the server) is executed (step 2824).
[0129]
Then, the server load status is updated (step 2825). Specifically, the content server load corresponding to the deleted content server load is subtracted from the observation server load, and the server load expected to be given by the mirror target content may be added. However, this update processing step 2825 does not necessarily have to be performed, and as shown in the description of the configuration, the server status monitoring apparatus 2027 may periodically update using the observation value.
[0130]
Next, the placement server information, which is the management information of the content placement status management apparatus 2014, is updated in accordance with the execution status of deletion and mirror (step 2826). In addition, the number of replicas is also updated. At this time, the content disc score is also updated. The content disc score is an index used for selecting a deletion target, which will be described later, and indicates the value of a content replica unit. The detailed definition will be described later in the operation of selecting a deletion target.
[0131]
Next, the disk status is updated (step 2827). Specifically, a process of subtracting the deleted content disk load from the observed disk load, adding the mirrored content disk load, and updating the remaining disk load is performed. Note that the disk load need not always be updated, and as shown in the description of the configuration, the disk status monitoring apparatus 2028 may periodically update the disk load using observation values.
[0132]
Next, the management information in the load balancer 2012 is also updated to maintain consistency with the content arrangement status (step 2828). Specifically, for mirrored content, the server is added to the server list to be guided. For the deleted content, the server from which the replica has been deleted is deleted from the request destination list of the load balancer 2012 in accordance with the deletion implementation status. If the number of replicas becomes “0” and there are no content replicas on the content distribution system 2010, another content distribution system or another system that determines the guide destination is registered as the guide destination.
[0133]
Then, “1” is added to the number of distribution destination servers (step 2829), and the server proceeds to the processing after step 2803 except for the servers selected from the candidate server list.
[0134]
The above is the processing content when the instruction from the arrangement target selection unit 2032 to the content arrangement determination unit 2031 is a content mirror.
[0135]
It can be assumed that an instruction to specifically place the content on any of the servers is given. In this case, the candidate server list created in step 2802 is made only to the designated server, and then steps 2803 and after are performed. Just do it.
[0136]
Here, the detailed operation of the deletion target selection process in step 2809 of FIG. 8 will be described in order with reference to FIG.
[0137]
The deletion target selection unit 2033 selects content to be deleted using the numerical values of the content access index and the content disc score calculated from the number of content replicas as indexes.
[0138]
As the simplest method, there is a method of using a value obtained by dividing a content access index by the number of replicas as a content disc score. The content disc score is managed by the content arrangement status management device 2014.
[0139]
First, for the content to be mirrored, not only simply divided by the number of replicas, but also the value of the predicted content disk score, which is obtained by dividing the content access index by the value obtained by adding “1” to the number of replicas before mirroring. Calculate (step 2901). As an initialization procedure, an empty deletion target content list is created (step 2902), and the erasable disk load is set to “0” (step 2903).
[0140]
Next, content having a content disc score lower than the predicted content disc score is registered in the deletion candidate content list from the content placed on the server disc (step 2904).
[0141]
Then, the deletion candidate is in the deletion candidate content list (step 2905), and the value obtained by subtracting the disk load from the disk limit load and further this deletable disk load value (remaining disk load) is the content disk load of the content to be mirrored. (Step 2906), the content to be deleted is selected by the procedure from step 2907 to step 2909.
[0142]
First, from the deletion candidate content list, the one with the lowest content disc score is selected as the deletion target content and added to the deletion target content list (step 2907), and the content disc load value is added to the deleteable disc load value. (Step 2908). This content is deleted from the deletion candidates (step 2909), and the process returns to step 2905.
[0143]
If the deletion candidate content list is empty in step 2905, the content arrangement determining unit 2031 is notified that there is no content that can be deleted on the server to be arranged (step 2910).
[0144]
If it is determined in step 2906 that the value obtained by subtracting the disk load from the disk limit load and the disk load value that can be deleted exceeds the content disk load of the content to be mirrored, the content that can be deleted is the server to be placed. The content placement determination unit 2031 is notified of the content existing above and the content to be deleted (step 2911).
[0145]
In the above, the content disk score is defined as the value obtained by dividing the content access index by the number of replicas, and the predicted content disk score is defined as the value obtained by adding “1” to the number of replicas before mirroring and the value obtained by dividing the content access index. However, this may be a value calculated from the content access index, which is an index aggregated in units of content distribution systems, and the number of replicas, and is not limited to the above method.
[0146]
Next, processing contents when the instruction from the arrangement target selection unit 2032 to the content arrangement determination unit 2031 is content deletion will be described.
[0147]
A case where the number of replicas to be deleted is designated in the content deletion instruction will be described in detail with reference to FIG.
[0148]
The content placement determining unit 2031 receives an instruction from the placement target selecting unit 2032 regarding the identifier of the content to be deleted and the number of deleted replicas to be deleted from the cluster (step 3000).
[0149]
The placement target selection unit 2032 determines the identifier of the content to be deleted and the number of replicas to be deleted from the cluster while referring to the information managed by the access monitoring device 2011 and the information managed by the policy management device 2015.
[0150]
For example, as shown in FIG. 7, it is assumed that content B is set to delete all replicas existing in the cluster when the content access index falls below “10”. Then, as shown in FIG. 24, it is assumed that the content access index of the content B is “5”, which is lower than the set threshold “10”.
[0151]
At this time, the arrangement target selection unit 2032 refers to the management information of the content arrangement state management apparatus 2014 and refers to the number of replicas of the content B. Then, the arrangement target selecting unit 2032 instructs the content arrangement determining unit 2031 to specify “b” as the identifier of the content to be deleted and the number of replicas of the content B referred to earlier as the number of new replicas to be created in the cluster. .
[0152]
The content arrangement determining unit 2031 initializes the deletion source determination server list (sets it to an empty state), and further sets the number of deletion source servers to “0” (step 3001). Then, referring to the management information of the content arrangement status management apparatus, the server where the replica of the specified content is placed on the disk is registered in the deletion candidate server list (step 3002).
[0153]
Next, the processing described below is executed until the deletion candidate server list becomes empty (step 3003) or until the number of deletion source servers reaches the number of deletion replicas instructed from the arrangement target selection unit 2032 (step 3004).
[0154]
First, the server selection unit 2034 selects one server from the candidate server list (3005). As a selection method at this time, there are the following methods. Method 1: Select the server with the highest observed server load value.
Method 2: Select the server with the highest observed disk load value.
A server is selected according to any of the above methods or some other algorithm.
[0155]
Next, the content to be deleted is deleted (step 3006).
[0156]
Then, the server load status is updated (step 3007). Specifically, the deleted content server load is subtracted from the observation server load. However, this update processing 3007 does not necessarily have to be performed, and as shown in the description of the configuration, the server status monitoring apparatus 2027 may periodically update using the observation value.
[0157]
Next, information on the arrangement server, which is management information of the content arrangement status management apparatus 2014, is updated in accordance with the execution status of deletion (step 3008). Specifically, the server that has been deleted is deleted from the list of placement servers, the number of replicas is decremented by 1, and the content disk score is updated using the updated number of replicas.
[0158]
Next, the disk status is updated (step 3009). Specifically, a process of subtracting the deleted content disk load from the observed disk load and updating the remaining disk load is performed. Note that the disk load need not always be updated, and as shown in the description of the configuration, the disk status monitoring apparatus 2028 may periodically update the disk load using observation values.
[0159]
Next, the management information in the load balancer 2012 is also updated to maintain consistency with the content arrangement status (step 3010). The server from which the replica has been deleted is deleted from the request destination list of the load balancer 2012 in accordance with the deletion implementation status. If the number of replicas becomes “0” and there are no content replicas on the content distribution system 2010, another content distribution system or another system that determines the guide destination is registered as the guide destination.
[0160]
Then, “1” is added to the number of deletion source servers (step 3011), the servers selected from the deletion candidate server list are excluded (step 3012), and the process proceeds to step 3003 and subsequent steps.
[0161]
In addition, it is assumed that the content deletion instruction is performed by designating a server. In this case, after registering only the server designated in the deletion candidate server list created in step 3002, step 3003 and the subsequent steps are registered. Should be executed. In addition, when instructed to delete the designated content from all servers on the distribution system, the content placement status is confirmed with the content placement status management device 2014, and the server where the content is placed is extracted. After these are registered in the deletion candidate server list, step 3003 and subsequent steps may be executed.
[0162]
It is assumed that content mirroring and deletion instructions can be given from the management terminal 2006 at any time. With such a configuration, it is also possible to realize content arrangement according to the judgment of the administrator. However, if content mirroring is performed using the above framework, a replica may not be created if the content disc score is low. In order to perform mirroring reliably, it is necessary to correct the content disc score to a sufficiently high value.
[0163]
Here, of the above-described processes, a more detailed explanation will be given on how to execute the process of content deletion and content mirroring (processing for creating a replica as a copy on a disk and determining a server in charge of distribution). Do.
[0164]
Regarding the deletion, the content arrangement control device 2030 directly deletes the replica from the disk, and issues an instruction to update the management information to the content arrangement state management device 2014 and an instruction to update the management information to the load distribution device 2012. Just do it. If the server has an interface that accepts deletion processing, the replica can be deleted from the disk using that interface. In the embodiment of the present invention, the deletion method may be any of the above methods, and may be another method as long as it can be used.
[0165]
Regarding the mirror, there is no content replica in the content distribution system, and there are differences between the case where the content is distributed from another content distribution system and the case where the content replica already exists in the content distribution system.
[0166]
First, a case will be described in which there is no content replica in the content distribution system and distribution is performed from another content distribution system. In this case, there are broadly divided into a PULL method shown in FIG. 21 and a PUSH method shown in FIG.
[0167]
First, the PULL method will be described with reference to FIG. As shown in FIG. 21, the mirror destination server 4101 receives an instruction from the content arrangement state management apparatus 4130, and the server 4101 sends a content distribution request to another content distribution system 4160. Then, there is a method in which the server 4201 receives the content distribution from the server 4151 in the other content distribution system 4160 that holds the content replica, and creates a replica on the disk 4202. Here, the delivery mechanism may be different from the delivery method to the client. For example, although the distribution is performed by UDP for the client, the distribution for the mirror may be performed by FTP. This method is a so-called PULL method in which a mirror destination server issues a request to acquire content.
[0168]
Next, the PUSH method will be described with reference to FIG. As shown in FIG. 22, the instruction from the content arrangement state management device 4230 is sent to another content distribution system 4260. Another content distribution system has a mirror distribution reception unit 4280 for receiving a mirror distribution, which receives which content is sent to which server of which content distribution system. Then, the mirror of the target content is performed from the server 4251 in the other content distribution system 4160 that holds the replica of the content to the specified server 4201 of the specified distribution system, and the replica is created on the disk 4202 To do. This method is called a push method because a request is not sent from the mirror destination server but sent from the mirror source suddenly.
[0169]
In the embodiment of the present invention, the distribution method from other content distribution systems of mirror processing may be either PULL type or PUSH type. Further, if it exists, another method may be used.
[0170]
Next, a case where a content replica already exists in the content distribution system will be described. As a method not to waste the network with the outside, as shown in FIG. 23, a method of performing mirroring by distribution in the content distribution system is desirable. However, as shown in FIG. 21 or 42, a method of mirroring from another content distribution system may be used. Furthermore, the delivery method here may be either the PULL type or the PUSH type.
[0171]
This embodiment differs greatly from the conventional example in that the content arrangement is not determined on a server basis, but on a content distribution system basis. Determining the number of content replicas (number of replicas) in the content distribution system based on the content access index, which is an index representing the access status to content managed in units of content distribution systems. A major feature is that replacement of content replicas for efficient use is determined based on a content disk score calculated from a content access index and the number of replicas.
[0172]
With these features, it is possible to appropriately distribute the load distribution and disk usage in the content distribution system according to the popularity of the content and the load on the system.
[0173]
(Second Embodiment)
Next, a second embodiment of the present invention will be described in detail with reference to the drawings.
[0174]
FIG. 11 shows the configuration of a system according to the second embodiment of the present invention. Compared to FIG. 1, the difference from the first embodiment is the arrangement of the disks in the cluster 3120. The disks 2124 and 3215 are not directly connected to each server, but are shared disks connected to all servers in the cluster by a local network (local network 3140).
[0175]
In FIG. 11, a disk 3124 and a disk 3125 are connected to a server 2021, a server 2022, and a server 2023 via a local network 3140. Examples of system connection forms with the servers 2021 to 2023 include NFS, SAN, NAS, and the like, but any method may be used.
[0176]
As shown in FIG. 4, in the first embodiment, content B is created by creating replicas on both disks 2024 and 2025 connected to server (1) 2021 and server (2) 2022. The server distribution capability for distributing the server was realized. On the other hand, in the configuration of the present embodiment, by simply placing a content replica on one of the disks, the server distribution is increased by increasing the number of servers in charge of the distribution (unless the disk throughput becomes a bottleneck). Capability can be enhanced. For this reason, the disk capacity required on the cluster can be further reduced as compared with the first embodiment.
[0177]
Due to the difference in the configuration described above, the functions and operations of each component are also partially different from those in the first embodiment. Hereinafter, the differences from the first embodiment will be described in order.
[0178]
First, the function of the server status monitoring apparatus 2027 is the same as that of the first embodiment.
[0179]
The function of the disk status monitoring device 2028 is the same as that of the first embodiment, but since the disk is not connected to only one server, it is in units of servers as in the example shown in FIG. Instead of managing, it is necessary to manage in units of disks as shown in FIG.
[0180]
The function of the load balancer 2012 is the same as that of the first embodiment. Here, a list of guidance destinations for managing candidate candidates for guiding client requests for each content as shown in FIG. 5 is referred to. The candidate server is a server in charge of distribution. However, the difference from the first embodiment is that the content is not stored on the disk directly attached to the server.
[0181]
The content arrangement status management apparatus 2014 manages the location information of each content in the cluster, as in the first embodiment. The difference from the first embodiment is that, as shown in FIG. 13, as information to be managed, correspondence between a server that executes content distribution and a disk from which the server reads content is managed. is there.
[0182]
As mentioned earlier, unless the throughput of reading from the disk reaches the limit, the content should basically be stored on any one disk, but if the disk throughput becomes a bottleneck It is also possible to create a replica on another disk and perform load distribution. Therefore, one disc does not necessarily correspond to one content. In such a case, the correspondence between the server and the disk to be read out should be managed for each server, and FIG. 13 shows such an example.
[0183]
It also manages a field for the number of servers in charge of distribution representing the number of servers in charge of new distribution. In addition, since the correspondence between the server and the disk is not uniquely determined, the content score is divided into two types: a content server score calculated from the number of servers in charge of distribution and a content access index, and a content disk score calculated from the number of replicas and the content access index. Shall be managed.
[0184]
In the example of FIG. 13, the content server score is a value obtained by dividing the content access index by the number of servers in charge of distribution, and the content disk score is a value obtained by dividing the content access index by the number of replicas. In this example, the content disc score and the content server score are calculated from the average number of simultaneous distributions that are the same content access index, but there is no problem even if they are calculated from different content access indexes.
[0185]
The function of the access monitoring apparatus 2011 is the same as that of the first embodiment. The content arrangement control apparatus 2030 that controls the arrangement of contents is the same as that in the first embodiment.
[0186]
Further, the function of the policy management apparatus 2015 is the same as that of the first embodiment, but in addition to the content of mirroring or deleting the content as the content of the action, a distribution server is added or reduced. You can also do that.
[0187]
For example, when the average number of simultaneous distributions exceeds “200” and there are no replicas in the cluster (number of replicas = 0) as in content A in FIG. 14, the content is sent to any server in the cluster. Mirror (action 1). If the value obtained by dividing the average number of simultaneous distributions by “200” exceeds the number of servers in charge of distribution in the current cluster (condition 2), another server in charge of distribution is increased by one (action 2), etc. Can be set. Further, when the value obtained by dividing the average number of simultaneous distributions by “1000” exceeds the number of servers in charge of distribution in the current cluster (condition 2) as in content B, another distribution server in the cluster is 1 At the same time, a replica can be created on another disk (action 2).
[0188]
(Operation of Second Embodiment)
Next, the operation when the content distribution system updates the content arrangement status in the second embodiment will be described in order.
[0189]
The arrangement target selection unit 2032 receives the content access index from the access monitoring apparatus 2011, and further refers to the management information of the policy management apparatus 2015, and determines the action for the content, specifically the content, according to the value of the content access index. Determine actions such as mirroring or deleting, adding a server in charge of distribution, reducing, or doing nothing.
[0190]
When the execution of the action is determined, the arrangement target selection unit 2032 instructs the content arrangement determination unit 2031 to execute the action. Specifically, the action is execution of allocation of a mirror and a distribution server to a disk of a certain content, addition of a distribution server (for a content that has already been mirrored), reduction of a distribution server, deletion of a content replica, Either. Hereinafter, it demonstrates in order.
[0191]
First, the processing contents in the case where the instruction is to execute the allocation of the content disk to the disk and the distribution server will be described with reference to the flowchart of FIG.
[0192]
The content arrangement determining unit 2031 receives an instruction from the arrangement target selecting unit 2032 regarding the execution of the mirror and the identifier of the target content (step 3500). Then, the content arrangement determination unit 2031 selects a candidate disk to be arranged while referring to the management information of the content arrangement state management apparatus 2014 (step 3503). As the selection method, the simplest method is to select the disk with the smallest observed disk load value from the disks on which the content to be mirrored is not arranged.
[0193]
The content arrangement determination unit 2031 inquires of the disk status monitoring apparatus 2028 about the disk status of the disk selected in step 3503 and the content arrangement status management apparatus 2014 about the content disk load of the mirrored content (step 3504).
[0194]
Whether the total of the observed disk load and the mirrored content disk load exceeds the disk limit load value depends on whether or not it is expected to exceed the disk limit load value by placing the content on the server. The process branches depending on whether or not (step 3505). If it is predicted that the disk limit load value will not be exceeded, the process proceeds to step 3508 described later.
If it is predicted in step 3505 that the disk limit load value is exceeded, the content arrangement determination unit 2031 inquires of the deletion target selection unit 2033 whether there is content that can be deleted (step 3506). If it is determined that there is no content that can be deleted (step 3507), the process ends. Step 3506, which is a deletion target selection process, is the same as the operation of step 2810 in the first embodiment, using the content disc score as an index.
When the deletion target is determined, the distribution server is assigned. The content arrangement determination unit 2031 causes the arrangement server selection unit 2034 to select one server candidate to which distribution is assigned (step 3508). As a selection method at this time, the simplest method is to select a server having the smallest observation server load value from among servers to which distribution of the target content has not yet been assigned.
Of the contents to be deleted selected in step 3504, the contents selected by the server selected in step 3508 are extracted. Then, the total value of the server loads given to the target server by these contents is calculated as the deletion target content server load total value.
[0195]
Then, after deleting these contents, a prediction is made as to whether or not the server load will not exceed the server limit load value when the server is in charge of distributing the contents to be mirrored (step 3509).
[0196]
In particular,
(Expression 2): Server load-total content server load to be deleted + server load expected to be given by mirror target content> server limit load value
Confirm whether or not holds.
If the relationship of the above (Formula 2) is satisfied, it is determined that the server load exceeds the server limit load value (step 3510), and the server manages the server by removing the content for which the server is in charge of distribution. It is confirmed whether the server load can be suppressed below the limit load value (step 3511). A specific method will be described in detail separately. If it is possible to select the content to be excluded from the person in charge of distribution, the process proceeds to step 3513 and subsequent steps. If it cannot be selected, the process ends. Note that the processing in steps 3511 and 3512 may be omitted.
[0197]
Next, the content selected by the deletion target selection unit 2033 as the deletion target is deleted from the server (step 3513). Next, mirroring of content to be mirrored (copying of content to a disc) is executed (step 3513). Then, the server selected in step 3508 is set as the server in charge of distributing the mirrored content (step 3515).
[0198]
Then, the server load status is updated (step 3516). Specifically, the update is performed by subtracting the deleted content server load total value from the observation server load and adding the server load expected to be provided by the mirrored content. However, the update process in step 3526 is not necessarily performed, and as shown in the description of the configuration, the server status monitoring apparatus 2027 may periodically update the observation value.
[0199]
Next, the arrangement server and disk information, which is management information of the content arrangement status management apparatus 2014, is updated in accordance with the execution status of deletion and mirror (step 3517). At this time, the number of replicas and the number of servers in charge of placement are also updated. At the same time, the content server score and the content disc score are updated.
[0200]
Next, the disk status is updated (step 3518). Specifically, a process of subtracting the deleted content disk load from the observed disk load, adding the mirrored content disk load, and updating the remaining disk load is performed. Note that the disk load need not always be updated, and as shown in the description of the configuration, the disk status monitoring apparatus 2028 may periodically update the disk load using observation values.
[0201]
Next, the management information in the load balancer 2012 is also updated to maintain consistency with the content arrangement status (step 3519). Specifically, for mirrored content, the server is added to the server list to be guided. For the deleted content, the server from which the replica has been deleted is deleted from the request destination list of the load balancer 2012 in accordance with the deletion implementation status. If the number of replicas becomes “0” and there are no content replicas on the content distribution system 2010, another content distribution system or another system that determines the guide destination is registered as the guide destination.
[0202]
The above is the processing contents in the case where the instruction from the arrangement target selection unit 2032 to the content arrangement determination unit 2031 is execution of allocation of a mirror to a content disk and a server in charge of distribution.
[0203]
Although it can be assumed that an instruction to place content on any one of the discs is specifically given, in this case, the candidate disc selected in step 3503 may be the designated disc. In addition, although it can be assumed that any server is specified as the content distribution server and an instruction is given, in this case, the candidate server selected in step 3508 may be the designated server. .
[0204]
Here, details of the content selection processing in step 3511 which is excluded from the distribution responsibility of the server selected in step 3508 will be described in order with reference to FIG.
[0205]
The deletion target selection unit 2033 selects content (exclusion target content) to be excluded from distribution by using the numerical values of the content access index and the content server score calculated from the number of content distribution servers as indexes. The simplest definition of the content server score is a value obtained by dividing the content access index by the number of servers in charge of distribution. The content server score is managed by the content arrangement state management device 2014.
[0206]
First, for the content to which the distribution source server is assigned, the content access index is divided by the value obtained by adding 1 to the number of servers in charge of distribution before distribution (current), instead of simply dividing by the number of servers in charge of distribution. Then, a numerical value called predicted content server score is calculated (step 3601).
[0207]
As an initialization procedure, an empty exclusion target content list is created (step 3602), and the server load that can be excluded is set to “0” (step 3603).
[0208]
Next, content having a content server score lower than the predicted content server score is registered in the exclusion candidate content list from the content assigned by the server in charge of distribution (step 3604), and the exclusion candidate content list Is not empty (step 3605), and the value obtained by subtracting the observable server load from the server limit load and further this exemptable disk load value (residual server load) represents the content server load of the content to which the distribution source server is assigned. The content to be excluded is selected by the procedure from step 3607 to step 3609 until it exceeds (step 3606).
[0209]
First, from the exclusion candidate content list, the one with the lowest content server score is selected as the exclusion target content to be removed from the server's distributor, and added to the exclusion target content list (step 3607). This is added to the disk load value that can be excluded (step 3608). Then, this content is deleted from the exclusion candidates (step 3609), and the process returns to step 3605.
[0210]
If the exclusion candidate content list is empty in step 3605, the content arrangement determining unit 2031 is notified that there is no content that can be excluded from the distribution target on the server to be allocated (step 3610).
[0211]
If it is determined in step 3606 that the value obtained by subtracting the server load limit from the server load limit and the server load value that can be excluded exceeds the content server load of the content to be distributed, it is excluded from the distribution target. The content that can be stored exists on the server to be allocated, and the content list to be excluded is notified to the content arrangement determination unit 2031 (step 3611).
[0212]
In the above, the value obtained by dividing the content access index by the value obtained by dividing the content server score by the content access index by the number of servers in charge of distribution, and the predicted content disk score by adding “1” to the number of distribution servers before distribution allocation. However, this may be a value calculated from the content access index, which is an index aggregated for each content distribution system, and the number of servers in charge of distribution, and the above method is only one variation. Absent.
[0213]
Next, the additional processing contents of the server in charge of distributing the content that has already been mirrored will be described.
[0214]
The processing content is almost the same as the processing content in the execution of the allocation of the content mirror to the disk and the distribution server as described in FIG. In FIG. 15, a processing procedure in which steps 3503 to 3507 and steps 3513 and 3514 are omitted may be executed.
[0215]
Next, processing contents in the case of reducing the number of servers in charge of distribution will be described with reference to the flowchart of FIG.
[0216]
The content arrangement determining unit 2031 receives an instruction from the arrangement target selecting unit 2032 regarding the identifier of the content to be reduced of the distribution server and the number of reduction distribution servers to be reduced from within the cluster (step 3700).
[0217]
The content arrangement determining unit 2031 initializes the reduction source determination server list (sets it to an empty state), and further sets the number of reduction source servers to “0” (step 3001). Then, with reference to the management information of the content arrangement status management apparatus 2014, the server to which the designated content is to be distributed is registered in the reduction candidate server list (step 3702).
[0218]
The processing described below is executed until the reduction candidate server list becomes empty (step 3703) or until the number of reduction source servers reaches the number of servers in charge of reduction distribution instructed from the arrangement target selection unit 2032 (step 3704). To do.
[0219]
First, one server is selected from the reduction candidate server list by the placement server selection unit 2034 (step 3705). As a selection method at this time, a method of selecting a server having the highest observation server load value may be considered.
[0220]
Next, the content to be excluded is excluded from the distribution targets of this server (step 3706).
[0221]
Then, the server load status is updated in the server status monitoring apparatus 2027 (step 3707). Specifically, processing for subtracting the deleted content server load from the observation server load is performed. However, this update processing 3707 does not necessarily have to be performed, and as shown in the description of the configuration, the server status monitoring apparatus 2027 may periodically update the observation value.
Next, the arrangement server information, which is the management information of the content arrangement status management apparatus 2014, is updated in accordance with the execution status of the deletion (step 3708). Specifically, deleting the server removed from the placement manager from the placement server list, subtracting one placement manager server, updating the content disk score using the updated number of placement manager servers, The process is performed.
[0222]
Next, the management information in the load balancer 2012 is also updated to maintain consistency with the content arrangement status (step 3710). Furthermore, the server from which the replica has been deleted is deleted from the request guide list of the load balancer 2012 in accordance with the deletion implementation status. If the number of replicas becomes “0” and there are no content replicas on the content distribution system 2010, another content distribution system or another system that determines the guide destination is registered as the guide destination.
[0223]
Then, “1” is added to the number of deletion source servers (step 3711), the servers selected from the deletion candidate server list are excluded (step 3712), and the process proceeds to step 3703 and subsequent steps. In addition, it is assumed that the content deletion instruction is performed by designating a server. In this case, after registering only the server designated in the deletion candidate server list created in step 3702, step 3703 is performed. You can do the following:
[0224]
In addition, when instructed to delete the designated content from all servers on the distribution system, the content placement status is confirmed with the content placement status management device 2014, and the server where the content is placed is extracted. After registering only the servers specified in the deletion candidate server list from these, step 3703 and the subsequent steps may be executed.
[0225]
Next, the case where the content deletion instruction specifies the number of replicas to be deleted will be described in detail with reference to FIG. A case where the number of replicas to be deleted is designated in the content deletion instruction will be described in detail with reference to FIG.
[0226]
The content placement determination unit 2031 is instructed by the placement target selection unit 2032 regarding the identifier of the content to be deleted and the number of deleted replicas to be deleted from the cluster (step 3800). The content arrangement determination unit 2031 refers to the management information of the content arrangement state management apparatus, and registers the disk in which the specified content replica is placed on the disk in the deletion candidate disk list (step 3801).
[0227]
Further, the deletion source determination disk list is initialized (empty), and the number of deletion source disks is set to “0” (step 3802).
[0228]
The processing described below is executed until the deletion candidate disk list becomes empty (step 3803) or until the number of deletion source disks reaches the number of deletion replicas instructed from the arrangement target selection unit 2032 (step 3804).
[0229]
First, the server selection unit 2034 selects one server from the candidate server list (step 3005). As a selection method at this time, a method of selecting a server having the highest observed disk load value can be considered.
[0230]
Next, the content to be deleted is deleted (step 3806).
[0231]
Then, the server load status is updated (step 3807). Specifically, the content server load of the deleted replica is subtracted from the observation server load of the server that was targeted for distribution of the deleted content replica. However, the update process in step 3807 does not necessarily have to be performed, and as shown in the description of the configuration, the server status monitoring apparatus 2027 may periodically update the observation value.
[0232]
Next, the arrangement server information, which is the management information of the content arrangement status management apparatus 2014, is updated in accordance with the execution status of the deletion (step 3808). Specifically, the server that was targeted for distribution of the deleted content replica and the deleted disk record from the list of placement servers, the process of subtracting the number of replicas, and the replica was targeted for distribution A process of subtracting the number of servers from the number of servers in charge of distribution, a process of updating the content disk score using the updated number of replicas, and a process of updating the content server score using the updated number of servers in charge of distribution.
[0233]
Next, the disk status is updated (step 3809). Specifically, a process of subtracting the deleted content disk load from the observed disk load and updating the remaining disk load is performed. Note that the disk load need not always be updated, and as shown in the description of the configuration, the disk status monitoring apparatus 2028 may periodically update the disk load using observation values.
[0234]
Next, the management information in the load balancer is also updated to maintain consistency with the content arrangement status (step 3810). The server from which the replica has been deleted is deleted from the request destination list of the load balancer 2012 in accordance with the deletion implementation status. If the number of replicas becomes “0” and there are no content replicas on the content distribution system 2010, another content distribution system or another content distribution system that determines the destination is registered as the destination. To do.
[0235]
Then, “1” is added to the number of deletion source disks (step 3811), the disks selected from the deletion candidate disk list are excluded (step 3812), and the process proceeds to step 3803 and subsequent steps.
[0236]
In addition, it is assumed that the content deletion instruction is performed by designating a disc. In this case, after registering only the disc designated in the deletion candidate disc list created in step 3802, step 3803 and the subsequent steps are performed. It is sufficient to execute the process.
[0237]
In addition, when instructed to delete the designated content from all the disks on the distribution system, the content placement status is confirmed with the content placement status management device 2014, and all the disks on which the content is placed are checked. After extracting and registering them in the deletion candidate disk list, the processing from step 3803 onward may be executed.
[0238]
Note that, as in the first embodiment, it is assumed that a content mirror or deletion instruction can be given from the management terminal 2006 as needed. With this configuration, it is possible to realize content arrangement based on the judgment of the administrator. However, if content mirroring is performed using the above framework, a replica may not be created if the content disc score is low. Similarly, if the content server score is low, there is a possibility that the server in charge of distribution cannot be allocated. In order to perform mirroring reliably, it is necessary to correct the content disc score and content server score to a sufficiently high value.
[0239]
In the present embodiment, the disk is shared between the servers, but the content distribution system is based on an index that represents an access status to content managed in units of content distribution systems, which is a content access index. Determine the number of replicas of content (number of replicas) and the number of servers in charge of distribution, replace content replicas for efficient use of disks, and replace the content that the server is responsible for distribution This is a major feature. With these features, it is possible to appropriately distribute the load distribution and disk usage in the content distribution system according to the popularity of the content and the load on the system.
[0240]
(Third embodiment)
Next, a third embodiment of the present invention will be described. The configuration of the system according to the third embodiment is the same as that of the first or second embodiment.
[0241]
The third embodiment is different from the first or second embodiment in the type of policy and the management method necessary for performing flexible and autonomous content arrangement management.
[0242]
In the first or second embodiment described above, according to the policy for each content set in the policy management device 2015, the content mirroring or deletion execution processing is executed as an action. When attempting to mirror some content, if the disk load on the disk was expected to exceed the disk limit load, the one with the lower content disk score was deleted from the disk.
[0243]
In the second embodiment, if the server load of the server is predicted to exceed the server limit load, the number of servers in charge of distributing content with a low content server score is reduced.
[0244]
However, depending on the content, it may be possible to suppress the deletion from the disk or the reduction in the number of servers in charge of distribution for a certain period of time. In order to realize such flexible content arrangement according to the characteristics of each content, various policy types are required. In addition, since the type of policy to be applied changes with time, it is desirable that the application time can be set.
[0245]
In a general content distribution system, it is desirable that the following control is performed autonomously.
(1) The designated content can be suppressed from being deleted from the disk and the number of servers in charge of distribution can be reduced during the desired period.
(2) Popular content is autonomously distributed in the content distribution system according to popularity.
(3) For other contents, for efficient operation of the disk, the disk space is released by receiving the deletion from the disk and the reduction of the number of servers in charge of distribution according to the rules set on the system.
[0246]
Here, in the third embodiment, a policy that is a set of conditions and actions as described below is introduced.
[0247]
In addition to the control based on the policy described below, a command for performing control such as creation or deletion of a replica at a specified time is prepared, and the administrator instructs the management terminal 2006 using the command to execute the replica. It is also possible to create and delete.
[0248]
(1) Fixed policy:
Condition: When selected as a deletion candidate for a set period of time
Action: Do not delete from the disk or reduce the number of servers in charge of distribution.
[0249]
(2) New replica creation policy:
Condition: During the set period, the content access index exceeds the set threshold, and there is no content replica in the distribution management system.
[0250]
Action: Perform processing to create a replica.
[0251]
(3) Replica expansion policy:
Condition: When the value divided by the set threshold value exceeds the number of replicas during the set period
Action: Execute processing for creating another replica in the distribution management system.
[0252]
(4) Replica deletion policy:
Condition: When the content access index falls below the set threshold during the set period
Action: Delete a set number of replicas from disk.
[0253]
(5) Replica eviction policy:
Condition: When selected as a deletion candidate during a set period
Action: Removed from disk.
[0254]
(6) New distribution server allocation policy:
Condition: During the set period, the content access index exceeds the set threshold, and the content distribution server is not in the distribution management system
Action: Execute processing to assign the server in charge of distribution.
[0255]
(7) Distribution server expansion policy:
Condition: When the value divided by the set threshold for the content access index exceeds the number of servers in charge of distribution during the set period
Action: Execute processing for increasing the number of content distribution servers in the distribution management system by one.
[0256]
(8) Distribution charge reduction policy:
Condition: When the content access index falls below the set threshold during the set period
Action: Reduce the number of servers in charge of distribution by the set number.
[0257]
In the case of the configuration of the first embodiment, the policies (1) to (5) are applied, and in the case of the configuration of the second embodiment, the policies (1) to (8) are applied. Thus, it is possible to realize control that satisfies the above requirements.
[0258]
For example, a setting example as shown in FIG. 19 will be described. In FIG. 19, content A is prohibited from being deleted from the disk or the number of servers in charge of distribution is reduced from “10:00 on May 5, 2003 to 10:00 on May 19”. Then, between “12:00 on May 12, 2003 and 10:00 on May 19, 2003”, when the value obtained by dividing the content access index by the set threshold value “15” exceeds the number of replicas, 1 replica is set. The process for adding one is executed.
[0259]
For content B, from “12:00 on May 12, 2003 to 10:00 on May 19, 2003”, if the content access index exceeds the set threshold value “10”, a process of assigning a server in charge of distribution is executed. The Then, between “19:00 on May 19, 2003 and 10:00 on May 26, 2003”, when the value obtained by dividing the content access index by the set threshold “10” exceeds the number of replicas, 1 replica is set. The process for adding one is executed. Furthermore, if the content access index falls below the set threshold “1” between “19:00 on May 19, 2003 and 10:00 on May 26, 2003”, the number of servers in charge of distribution is reduced by “1”. (In this example, the parameters are set in the order of the threshold value and the reduction number.)
Here, during the period when the fixed policy is not set for the content, it is possible to delete the content from the disc and reduce the number of servers in charge of distribution according to the first embodiment or the second embodiment. . Further, as in the first embodiment and the second embodiment, it is assumed that a content mirror or deletion instruction can be given from the management terminal 2006 as needed. Further, the content access index used by each policy may be the same index or a different index.
[0260]
There are various types of policies other than the above, but the essence of the present embodiment is that the following policies are prepared, and these policies are used in the first embodiment and second policy. This is in that it is executed using the autonomous mechanism for increasing / decreasing replicas and distribution allocation servers shown in the embodiment.
(1) Policies for preventing deletion from disk and reduction of the number of servers in charge of distribution
(2) Policy for preparing the first content distribution means in the distribution system
(3) Policy when adding content distribution means to the distribution system
(4) Policy for autonomously deleting from the distribution system when popularity is lost
Note that these policies do not necessarily need to be performed for each content, and may be set for each group after the content is grouped.
[0261]
(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. The configuration of the system according to the fourth embodiment is the same as that of the first, second, or third embodiment.
[0262]
In the fourth embodiment, a management method necessary for performing flexible and autonomous content arrangement management according to the server load and the disk load is realized. Up to the third embodiment, the policy setting is based on content, but here, autonomous content placement according to the state of the server or disk is realized.
[0263]
First, autonomous content placement according to the state of the disc in the fourth embodiment will be described.
[0264]
First, the disk status monitoring device 2027 monitors whether there is any disk whose observed disk load exceeds the disk limit load. If there is a disc exceeding the disc limit load, the content with the largest content access index is selected. In the case of the same configuration as that of the first embodiment, another replica creation process is executed according to the procedure shown in FIG. Further, in the case of the same configuration as that of the second embodiment, the process of creating one replica is executed according to the procedure shown in FIG.
[0265]
Next, autonomous content placement according to the server status in the fourth embodiment will be described.
[0266]
First, the server status monitoring apparatus 2027 monitors whether there is a server whose observed server load exceeds the server limit load. If there is a server that exceeds the server limit load, the content with the largest content access index is selected. In the case of the same configuration as that of the first embodiment, another replica creation process is executed according to the procedure shown in FIG. In the case of the same configuration as that of the second embodiment, the process of increasing the number of servers in charge of distribution by one is executed according to the processing flow shown in the operation of the second embodiment.
[0267]
(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. FIG. 20 shows the configuration of a system according to the fifth embodiment.
[0268]
A difference from the configuration of the embodiments described above is that a server 4050 is arranged on a network 2005 connected to an external content distribution system. A disk 4051 is connected to the server 4050 as a means for storing contents separately from the cluster. Although the configuration of the cluster 2020 shown in FIG. 20 is the same as that of the cluster in the first embodiment, it may be a cluster configuration having a shared disk as in the configuration of the second embodiment.
[0269]
By the way, in the embodiments so far, autonomous load balancing according to popularity has been realized, so for popular content and individually mirrored content, a quick response from the server of the distribution system High-throughput distribution can be realized. However, until it is confirmed that the content has become popular, specifically, since the replica is not created on the distribution system during the period of updating the content access index, even if the content suddenly becomes popular, Each request had to be directed to an external content distribution system and distributed from there. In such a situation, it would be more desirable if delivery with a fast response and high throughput could be achieved.
[0270]
With the configuration according to the fifth embodiment, it is possible to realize a fast response and high-throughput distribution even for content that has become popular during the period of updating the content access index.
[0271]
First, as an operation different from the above-described embodiments, there is a point that a request that has been guided to an external content distribution system so far is guided to the server 4050 by the load balancer 2012.
[0272]
The server 4050 operates as a so-called proxy cache server, and acquires content from an external content distribution system when receiving a request for content not held on a connected disk from a user. The acquired content is distributed to the clients 2001 to 2003 after being stored in the disk 4051 or while being stored.
[0273]
A request for the held content is not acquired from an external content distribution system and relayed, but is read from the disk 4051 and distributed to the clients 2001 to 2003. When the disk 4051 becomes full, replacement is executed by a replacement algorithm or the like in a general cache server.
[0274]
In this way, by installing the server 4050 that functions as a so-called proxy cache, it is possible to realize fast response and high-throughput distribution even for content that has become popular during the period of updating the content access index. Become.
[0275]
Furthermore, the utilization efficiency of the disk 4051 can be improved by linking the content arrangement state management apparatus 2014 and the server 4050. For content whose content access index has been updated and a replica has been created according to the set policy, since the request is directed to the cluster server thereafter, it is not directed to the server 4050. Disappear.
[0276]
Therefore, at the same time as detecting that a replica has been created for the cluster, the content arrangement status management apparatus 2014 instructs the server 4050 to delete the content from the disk 4051. The server 4050 deletes the content from the disk 4051.
[0277]
By doing so, the available disk space can be secured in advance, and the overhead of content replacement processing can be reduced.
[0278]
(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described. The configuration and operation of the system according to the sixth embodiment follows the configuration and operation of any of the first to fourth embodiments, but each server 2021 to 2023 operates as a proxy cache server. The point is different. Accordingly, the functions and operations of the load distribution device 2012 and the content arrangement control device 2030 are changed.
[0279]
Hereinafter, a sixth embodiment based on the first embodiment will be described with reference to the configuration shown in FIG.
[0280]
The configuration other than the function is the same as that of the first embodiment as shown in FIG. However, the server 2021, the server 2022, and the server 2023 of the cluster 2020 function as proxy cache servers.
[0281]
As described in the fifth embodiment, the function of the proxy server is that when a content request is received from a user, content that is not held in the connected disc is content from an external content distribution system. To get. Then, the acquired content is distributed to the client after being stored on the disk or while being stored. The stored content is read from the disc and distributed to the client.
[0282]
The load balancer 2012 in the first to fourth embodiments stores the requested content on a disk connected to a server in the cluster, and if the distribution is possible from that server, the request is sent there. If the requested content is not stored on a disk connected to a server in the cluster or there is no server that can deliver the content, Was directing requests to other content delivery systems.
[0283]
In the sixth embodiment, the load distribution apparatus 2012 issues a guidance destination update process to the content arrangement control apparatus 2030, waits for the completion of this process, and then the first to fourth embodiments. It is characterized by performing the operation in.
[0284]
In the first to fourth embodiments, the content placement control apparatus 2030 that controls the placement of content is a content that creates a replica (copy) on a server in the cluster based on the access status or the like, It had a function to select contents to delete replicas (copy) from the server, and to execute contents mirroring and deletion instructions.
[0285]
The sixth embodiment has a function of receiving a content destination update request from the load balancer 2012. This function and the functions possessed in the first to fourth embodiments can be used in combination, or only the functions shown below can be used.
[0286]
Since the sixth embodiment and the first embodiment described below function only in parallel when used in combination, the new function will be described below with reference to the flowchart of FIG.
[0287]
When the load balancer 2012 guides the user's request, the load balancer 2012 requests the content arrangement control device 2030 to update the content guide destination.
[0288]
The content arrangement determination unit 2031 receives this update request and relays it to the arrangement target selection unit 2032 (step 4500). The arrangement target selection unit 2032 receives the content access index from the access monitoring apparatus 2011 and further refers to the management information of the policy management apparatus 2015. Then, the arrangement target selecting unit 2032 determines whether it is necessary to add a content destination, and calculates the number of servers to be added when the addition is necessary (step 4501).
[0289]
For example, as shown in FIG. 7, in the content A, when the content access index is “200” or more and the number of replicas existing in the cluster is “0”, any server in the cluster Assume that one is set to create a replica. Then, as shown in FIG. 24, it is assumed that the content access index of content A is “300”. At this time, the arrangement target selection unit 2032 determines to add one server as a request guide destination of the content A.
[0290]
If it is determined that addition is necessary (step 4502), the guide destination determination server list is initialized (empty state), and the number of guide destination determination servers is set to “0” (step 4502).
[0291]
Then, the content arrangement determining unit 2031 registers a server that meets the set criteria in the candidate server list while referring to the management information of the server status monitoring apparatus 2027 and the management information of the content arrangement status management apparatus 2014 (step 4504). ). As an example of the reference set here, it is possible to use a server that does not contain content to which a guidance destination is added and is active, and the like.
[0292]
If it is not determined that addition is necessary, a notification of completion of update processing is sent to the load balancer 2012 (step 4531).
The following processing is executed until the candidate server list becomes empty (step 4505) or the number of guidance destination determination servers reaches the additional number instructed from the arrangement target selection unit 2032 (step 4506).
[0293]
That is, when the candidate server list becomes empty or the number of guidance destination determination servers reaches the additional number instructed from the arrangement target selection unit 2032, the content arrangement determination unit 2031 completes the guidance destination update processing in the load balancer 2012 Is notified (step 4521).
[0294]
First, the server selection unit 2034 selects one server from the candidate server list (step 4507). As a selection method at this time, there are the following methods.
[0295]
Method 1: Select the server with the smallest observation server load value.
[0296]
Method 2: Select the server with the smallest observed disk load value.
[0297]
Method 3: Select from among servers whose residual disk load is larger than the content disk load of the mirrored content by Method 1 or Method 2. If there is no server with enough disk space to place content, the server is selected by either method 1 or method 2.
[0298]
A server is selected according to any of the above methods or some other algorithm.
[0299]
The content arrangement determination unit 2031 inquires of the disk status monitoring apparatus 2028 about the disk status of the server determined in step 4505 and the content arrangement status management apparatus 2014 about the content disk load of the mirrored content (step 4508).
[0300]
Depending on whether the content is placed on the server or not expected to exceed the disk limit load value, the total of the observed disk load and the mirrored content disk load exceeds the disk limit load value. The process branches depending on whether or not it exceeds (step 4509).
[0301]
If it is predicted that the disk limit load value will not be exceeded, the process proceeds to step 4524 described later. When it is predicted that the disk limit load value will be exceeded, the content arrangement determination unit 2031 inquires of the deletion target selection unit 2033 whether there is content that can be deleted (step 4510). If it is determined that there is no content that can be deleted (step 4511), the process proceeds to step 4530 and later.
[0302]
If it is instructed that there is content that can be deleted, the server load will not exceed the server load limit value after the content is deleted and a replica of the content to be added to the destination is created on the server. Is predicted (step 4512).
[0303]
If it is determined that the server load does not exceed the server load limit value (step 4513), the content selected by the deletion target selection unit 2033 as a deletion target is deleted from the server (step 4514). Note that the processing in step 4512 and step 4513 is not necessarily performed.
[0304]
Then, the server load status is updated (step 4515). However, step 4515 which is the update process does not necessarily have to be performed, and as shown in the description of the configuration, the server status monitoring apparatus 2027 may periodically update the observation value.
[0305]
Next, the arrangement server information, which is the management information of the content arrangement state management apparatus 2014, is updated in accordance with the deletion or addition of the guidance destination (step 4516). In addition, the number of replicas is also updated. At this time, the content disc score is also updated.
[0306]
Next, the disk status is updated (step 4517). Note that the disk load need not always be updated, and as shown in the description of the configuration, the disk status monitoring apparatus 2028 may periodically update the disk load using observation values.
[0307]
Next, management information in the load balancer 2012 is also updated to maintain consistency with the content arrangement status (step 4518). Specifically, a server in the guidance destination determination server list is added to the guidance target. For the deleted content, the server from which the replica has been deleted is deleted from the request destination list of the load balancer 2012 in accordance with the deletion implementation status. If the number of replicas becomes “0” and there are no content replicas on the content distribution system 2010, another content distribution system or another system that determines the guide destination is registered as the guide destination.
[0308]
Then, “1” is added to the number of guidance destination determination servers (step 4519), the servers selected from the candidate server list are excluded (step 4520), and the process proceeds to step 4503 and subsequent steps.
[0309]
The sixth embodiment differs greatly from the first to fifth embodiments in that the cache server autonomously obtains content rather than issuing a content mirror instruction. is there. By introducing a cache server and guiding user requests with the load balancer, content mirroring can be performed autonomously without any explicit support.
[0310]
(Seventh embodiment)
Next, a seventh embodiment of the present invention will be described. The configuration of the seventh embodiment may be any of the configurations of the first to sixth embodiments.
[0311]
The system of the seventh embodiment is different from the first to sixth embodiments in the management of content access indexes.
[0312]
The content access index is an index indicating a client request for the content or a delivery result accompanying the request. If this index is simply managed, the value is “0” until a request for content or a distribution result occurs for the first time. For this reason, the newly generated content may become less popular than content that has a long content access index.
[0313]
For example, a case is assumed where the number of requests for each content since the content distribution system is activated is managed as a content access index. It is assumed that the content A accessed “1000” in the last week, but this week has 0 requests. On the other hand, it is assumed that the content B generated yesterday has already been accessed 500 times.
[0314]
In this case, if the access index is simply managed by integrating the number of requests, it is determined that the content B is less valuable than the content A, and the content A is given priority in the case of replacement. The problem of being left on the disk occurs.
[0315]
In order to solve such a problem, it is only necessary to raise the value of the content access index that is set when a request for content or a delivery result occurs for the first time.
[0316]
As a method for selecting the value to be raised, the content access index of the content that has been deleted from the disk in the content distribution system most recently is raised, with the value of the smallest content access index in the content distribution system being raised. And the content access index of the content most recently removed from the server in charge of the content distribution system is set as the bottom-up value.
[0317]
(Eighth embodiment)
An eighth embodiment will be described. The configuration of the system according to the eighth embodiment may be any of the configurations of the first to seventh embodiments.
[0318]
The difference of the eighth embodiment from the first to seventh embodiments is the management of the observation server load.
[0319]
In the distribution system, the server is down and restarted due to system troubles and maintenance, or a new server is added to increase the system processing capacity. At this time, if it is simply performed, the observation server load takes “0” as an initial value. Since the observation server load is low, there is a possibility that content distribution will be assigned to this server all at once.
[0320]
In order to solve such a problem, the initial value of the server load may be raised. As a method for selecting the value to be raised, there is a method in which the lowest server load value in the content distribution system is set as the raised value.
[0321]
(Ninth embodiment)
A ninth embodiment will be described. The configuration of the system according to the ninth embodiment may be any of the configurations of the first to eighth embodiments.
[0322]
The ninth embodiment is different from the first to eighth embodiments in the management of the observation disk load.
[0323]
In the distribution system, the disk is stopped and recovered due to system trouble or maintenance, or a new disk is added to increase the system processing capacity. At this time, if it is simply performed, the observation disk load takes “0” as an initial value. Since the observation disk load is low, content replicas may be created on this disk all at once.
[0324]
In order to solve such a problem, the initial value of the disk load may be raised. As a method for selecting the value to be raised, there is a method in which the lowest disk load value in the content distribution system is set as the raised value.
[0325]
Although the present invention has been described with reference to the preferred embodiments and examples, the present invention is not necessarily limited to the above-described embodiments and examples, and various modifications can be made within the scope of the technical idea. Can be implemented.
[0326]
【The invention's effect】
As described above, according to the content arrangement management system in the content distribution system of the present invention, the following excellent effects can be obtained.
[0327]
The first effect of the present invention is that the cost required for content arrangement management can be greatly reduced. The reason is that based on the content access index of the access monitoring device, the content distribution person in charge is allocated to the clustered servers in the content distribution system, and replicas are created and deleted on the disk autonomously. This is because content placement management within the content distribution system is performed autonomously.
[0328]
The second effect of the present invention is to improve the utilization efficiency of an external storage device such as a disk. In addition, it is possible to improve the utilization efficiency of an external storage device such as a disk by realizing arrangement management in accordance with the popularity of content and the load applied to the system. The reason is that location management is performed according to the popularity of content not in units of servers but in units of content distribution systems.
[0329]
A third effect of the present invention is that it is possible to realize arrangement management in units of contents. By setting arrangement management content called a policy for each content unit, it is possible to perform arrangement management for each content unit.
[0330]
The fourth effect of the present invention is that the operating rate of the server is increased and a standby standby server is not required. The reason is that a plurality of content groups or contents can be assigned to the same server, so that load distribution with higher granularity than conventional can be realized. Furthermore, this is because there is a function of reassigning a server of a content group with a low load to a server of a content group with a high load, such as when there is no spare server.
[0331]
The fifth effect of the present invention is that it can cope with contents whose load is increased on the system due to sudden popularity. The reason is that the proxy cache server and the content distribution system are used together. Conventionally, requests that have been made to other content distribution systems and distributions that have been made from other content distribution systems can be performed with a proxy cache server. High-response and high-throughput distribution is possible even for content that goes out and has a heavy load on the system.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a system according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of contents of an observation server load and a server limit load value managed by the server status monitoring apparatus according to the first embodiment.
FIG. 3 is a diagram illustrating an example of contents of an observed disk load and a disk limit load value managed by the disk status monitoring apparatus according to the first embodiment.
FIG. 4 is a block diagram for explaining request guidance processing according to the first embodiment;
FIG. 5 is a diagram illustrating a content example of management information in the load distribution apparatus according to the first embodiment;
FIG. 6 is a diagram illustrating an example of content management information managed by the content arrangement status management apparatus according to the first embodiment;
FIG. 7 is a diagram illustrating an example of the contents of management information in the policy management apparatus according to the first embodiment.
FIG. 8 is a flowchart showing processing contents of content arrangement control (mirror) in the first embodiment;
FIG. 9 is a flowchart showing processing details of selection of a deletion target in the first embodiment.
FIG. 10 is a flowchart showing processing details of content arrangement control (deletion) in the first embodiment.
FIG. 11 is a block diagram showing a configuration of a system according to a second embodiment of the present invention.
FIG. 12 is a diagram illustrating an example of contents of an observed disk load and a disk limit load value managed by the disk status monitoring apparatus according to the second embodiment.
FIG. 13 is a diagram illustrating an example of content management information managed by the content arrangement status management apparatus according to the second embodiment.
FIG. 14 is a diagram illustrating an example of contents of management information in a policy management apparatus according to the second embodiment.
FIG. 15 is a flowchart illustrating processing contents of content arrangement control (mirror and server allocation) in the second embodiment;
FIG. 16 is a flowchart showing processing details of deletion target selection in the second embodiment;
FIG. 17 is a flowchart showing processing contents of content arrangement control (distribution server reduction) in the second embodiment;
FIG. 18 is a flowchart illustrating processing contents of content arrangement control (deletion) in the second embodiment.
FIG. 19 is a diagram illustrating an example of a policy which is a set of conditions and actions according to the third embodiment.
FIG. 20 is a block diagram showing a configuration of a system according to a fifth embodiment of the present invention.
FIG. 21 is a block diagram for explaining distribution (PULL type) from another content system in the first embodiment;
FIG. 22 is a block diagram for explaining distribution (PUSH type) from another content system in the first embodiment;
FIG. 23 is a block diagram for explaining creation of a replica in the content system in the first embodiment;
FIG. 24 is a diagram illustrating an example of the contents of management information managed by the access status monitoring apparatus according to the first embodiment.
FIG. 25 is a flowchart showing the processing contents of content arrangement control (guidance destination update) in the sixth embodiment.
FIG. 26 is a block diagram showing a configuration diagram of a conventional content distribution system.
FIG. 27 is a block diagram showing a configuration of a conventional content distribution system using a shared storage.
FIG. 28 is a block diagram showing a configuration of a conventional wide area content distribution system.
FIG. 29 is a block diagram showing a configuration of a content arrangement management system in a conventional content distribution system.
FIG. 30 is a diagram illustrating a content example of a content management table in a conventional content arrangement management system.
[Explanation of symbols]
2001-2003: Client
2004: Access network
2005: Network
2006: Management terminal
2010: Content distribution system
2011: Access monitoring device
2012: Load balancer
2013: Local network
2014: Content arrangement state management device
2015: Policy management device
2020: Cluster
2021-2023: Server
2024-2026: Disc
2027: Server status monitoring device
2028: Disk status monitoring device
2030: Content arrangement control device
2031: Content arrangement determination unit
2032: Arrangement target selection unit
2033: Deletion target selection unit
2034: Arrangement server selection unit
3120: Cluster
3124, 3125: Disk
3140: Local network
4050: Server
4051: Disc

Claims (41)

A content placement management system for managing content placement in a content distribution system,
An access monitoring device that manages, in units of content distribution systems, requests for content from the client, content distribution results associated with the requests, and content access indexes that indicate the load that the content converted from them has on the system ,
Based on the content access index of the access monitoring device, the allocation of content distribution to clustered servers in the content distribution system, autonomous creation and deletion of replicas to the disk, Content placement management system. The content arrangement management system according to claim 1, wherein the number of replicas of the content in the content distribution system is calculated from the content access index. The content arrangement management system according to claim 2, wherein a threshold value used for calculating the number of replicas of the content in the content distribution system from a content access index is set for each content. The threshold value used when calculating the number of replicas of the content in the content distribution system from a content access index is set for each content differently depending on the number of replicas in the content distribution system. The content arrangement management system according to claim 2 or claim 3. 5. The content arrangement according to claim 1, wherein the number of servers in charge of distributing the content to the client in the content distribution system is calculated from the content access index. Management system. The threshold value used when calculating the number of servers in charge of distributing the content to clients in the content distribution system from the content access index is set for each content. Content placement management system. Depending on the number of servers in charge of delivery to clients in the content distribution system, the threshold used when calculating the number of servers in charge of delivery of the content to clients in the content delivery system from the content access index 7. The content arrangement management system according to claim 5, wherein a different value is set for each content. When creating a replica of the content on a predetermined disk in the content distribution system, if it is determined that the allowable load value of the disk is exceeded, it is deleted from the disk in order to reduce the load on the disk The content replica is determined on the basis of a content access index that is an index representing an access status to the content managed in units of the content distribution system. The content arrangement management system described. When creating a replica of the content on a predetermined disk in the content distribution system, it is determined that the load on the disk exceeds an allowable value, and the content of the content from the predetermined disk is reduced to reduce the load on the disk. When selecting and deleting a replica,
For content replicas on the predetermined disc, the content access index, which is an index indicating the access status to the content managed in units of the content distribution system, is divided by the number of replicas in the content distribution system. Calculate the disk score,
For the content that is to be created as a replica, a copy of the content access index, which is an index representing the access status to the content managed in the content distribution system unit, is created from the number of replicas in the content distribution system. Calculate the value of the predicted content disc score divided by the number you want to add,
When replicas of content whose content disc score is lower than the predicted content disc score are extracted as deletion candidates, and all deletion candidates are deleted, it is expected that the disk load will not exceed the allowable value even if replicas are created ,
The replica of the content is created after deleting the replica until it is predicted that the load on the disc does not exceed an allowable value even if the replica is created in order from the lowest content disk core. The content arrangement management system according to any one of claims 1 to 8. When creating a replica of content on a predetermined disk in the content distribution system, selecting the disk with the lowest load on the disk or the largest allowable disk load as the replica creation destination. The content arrangement management system according to claim 9, wherein: When allocating a predetermined server in the content distribution system as a person in charge of content distribution, if it is determined that the load on the server exceeds an allowable value, the server is removed from the server distribution person to reduce the server load 11. The content replica is determined based on a content access index that is an index representing an access status to content managed in units of the content distribution system. The content arrangement management system described. When allocating a predetermined server in the content distribution system as a person in charge of content distribution, if it is determined that the load on the server exceeds an allowable value, the server is removed from the server distribution person in order to reduce the server load When determining a replica of content based on a content access index that is an index representing an access status to content managed in units of content distribution systems,
For content replicas that have already been assigned to the server, calculate the value of the content server score by dividing the content access index by the number of servers to which content delivery is assigned,
For the content to be created as a replica, a value that is a predicted content server score by dividing the content access index by the number of servers to which content distribution is allocated plus the number of servers to be allocated To calculate
If the content server score lower than the predicted content server score is extracted as a deletion candidate, and the deletion candidate is deleted, the server load is expected not to exceed the allowable value even if the replica is created, The content replica is created after the replica is deleted until it is predicted that the load on the server does not exceed an allowable value even if the replica is created in order of increasing content server score. The content arrangement management system according to any one of claims 1 to 11. When allocating a predetermined server in the content distribution system as a person in charge of content distribution, a server having the lowest server load or the largest allowable value of the remaining server load is selected as a distribution allocation destination. The content arrangement management system according to claim 12. The content arrangement management system according to any one of claims 1 to 13, wherein a disk is connected to each of a plurality of servers in the content distribution system. The content arrangement management system according to any one of claims 1 to 14, wherein a disk in the content distribution system is connected as a shared disk that can be referred to from a plurality of servers. The content arrangement management system according to any one of claims 1 to 15, wherein the autonomous arrangement management content of content is set as a policy that is a combination of a condition and processing content at that time. The content arrangement management system according to claim 16, wherein the policy is set for each content. Creating a replica of content on a server in the cluster or deleting a replica from a server in the cluster while referring to the policy management apparatus that sets the policy for each content, the content access index and the policy The content arrangement management system according to claim 16, further comprising: a content arrangement control device that controls arrangement of contents. A content arrangement status management device for managing content location information is provided.
19. The content arrangement management system according to claim 18, wherein the content arrangement control device controls the arrangement of the content with reference to the content location information of the content arrangement state management device. Set autonomous content management content as a policy that is a combination of conditions and processing content at that time,
(1) Fixed policy (condition: when selected as a deletion candidate during a set period / action: not to be deleted from the disk or to be reduced to the number of servers in charge of distribution),
(2) New replica creation policy (Condition: The content access index exceeds the set threshold during the set period, and the content replica is not in the distribution management system / Action: Process for creating the replica is executed )
(3) Replica expansion policy (Condition: When the value obtained by dividing the content access index by the set threshold exceeds the number of replicas during the set period / Action: Create another replica in the distribution management system Execute the process for)
(4) Replica deletion policy (Condition: When the content access index falls below the set threshold during the set period / Action: Delete the set number of replicas from the disk),
(5) Replica eviction policy (condition: when selected as a deletion candidate during a set period / action: deleted from disk),
(6) New distribution server allocation policy (Condition: The content access index exceeds the set threshold during the set period, and the content distribution server is not in the distribution management system / Action: Allocate distribution server) Execute the process.
(7) Distribution server expansion policy (Condition: When the value divided by the set threshold for the content access index exceeds the number of distribution servers during the set period / Action: Distribution of content in the distribution management system Execute the process to increase the number of servers in charge by one),
(8) Distribution charge reduction policy (Condition: When the content access index falls below the set threshold during the set period / Action: Reduce the number of distribution charge servers by the set number)
20. The content arrangement management system according to claim 1, wherein the content arrangement management system is selected and set from any one of the policies. 21. The content arrangement management system according to claim 1, wherein a proxy cache server is provided in the content distribution system, and a part of content is distributed via the proxy cache server. . 21. The server according to claim 1, wherein a server in the content distribution system functions as a proxy cache server, and a content replica creation or deletion instruction is indirectly executed by guidance by a load balancer. The content arrangement management system according to claim 1. The disk load is monitored, and if there is a disk whose observed disk load exceeds the allowable value of the disk load, a replica of the content with the largest content access index is created in another content distribution system. The content arrangement management system according to any one of claims 1 to 22. The server load is monitored, and when there is a server whose observed server load exceeds an allowable value of the server load, the number of servers in charge of distributing a replica of a content having the largest content access index is increased. The content arrangement management system according to any one of claims 1 to 23. 25. The initial value of a content access index that is set when a request for content or a delivery result occurs for the first time is set as a dynamically changing reference value. Content placement management system. The content placement management according to any one of claims 1 to 25, wherein an initial value of an observation server load of a restarted or newly added server is set as a dynamically changing reference value. system. 27. The content arrangement according to claim 1, wherein an initial value of an observed disk load of a restarted or newly added disk is set as a dynamically changing reference value. Management system. A content placement management program that is executed on a computer and performs content placement management in a content distribution system,
A content access index, which is an index indicating a load from the client on the content, a distribution result of the content accompanying the request, and a load that the content converted from the request gives to the system is managed in units of content distribution systems,
Based on the content access index, a function for autonomously allocating content distribution personnel to clustered servers in the content distribution system and creating or deleting replicas on a disk is executed. Content placement management program. A function of calculating the number of replicas of the content in the content distribution system from the content access index;
The threshold value used when calculating the number of replicas of the content in the content distribution system from a content access index has a function of setting a different value for each content depending on the number of replicas in the content distribution system. The content arrangement management program according to claim 28. A function for calculating the number of servers in charge of distributing the content in the content distribution system to clients from the content access index;
Depending on the number of servers in charge of distribution to clients in the content distribution system, the threshold used when calculating the number of servers in charge of distribution of the content to clients in the content distribution system from the content access index The content arrangement management program according to claim 28, having a function of setting different values for each content. When creating a replica of the content on a predetermined disk in the content distribution system, if it is determined that the allowable load value of the disk is exceeded, it is deleted from the disk in order to reduce the load on the disk 31. The function according to claim 28, further comprising a function of determining a content replica based on a content access index that is an index representing an access status to content managed in units of the content distribution system. The content arrangement management program according to item 1. When creating a replica of the content on a predetermined disk in the content distribution system, it is determined that the load on the disk exceeds an allowable value, and the content of the content from the predetermined disk is reduced to reduce the load on the disk. When selecting and deleting a replica,
For content replicas on the predetermined disc, the content access index, which is an index indicating the access status to the content managed in units of the content distribution system, is divided by the number of replicas in the content distribution system. A function to calculate the value of the disk score;
For the content that is to be created as a replica, a copy of the content access index, which is an index representing the access status to the content managed in the content distribution system unit, is created from the number of replicas in the content distribution system. A function to calculate a value that is a predicted content disc score divided by the value to be added, and
When replicas of content whose content disc score is lower than the predicted content disc score are extracted as deletion candidates, and all deletion candidates are deleted, it is expected that the disk load will not exceed the allowable value even if replicas are created ,
The content disk core has a function of creating a replica of the content after deleting the replica until it is expected that the disk load will not exceed an allowable value even if the replica is created in order from the lowest. The content arrangement management program according to any one of claims 28 to 31. When allocating a predetermined server in the content distribution system as a person in charge of content distribution, if it is determined that the load on the server exceeds an allowable value, the server is removed from the server distribution person to reduce the server load 33. The function according to claim 28, further comprising a function of determining a content replica based on a content access index that is an index representing an access status to content managed in units of the content distribution system. The content arrangement management program according to item 1. When allocating a predetermined server in the content distribution system as a person in charge of content distribution, if it is determined that the load on the server exceeds an allowable value, the server is removed from the server distribution person in order to reduce the server load When determining a replica of content based on a content access index that is an index representing an access status to content managed in units of content distribution systems,
A function of calculating a value as a content server score by dividing the content access index by the number of servers to which content distribution is assigned, for a replica of the content already assigned to the server;
For the content that is to be created as a replica, a value that is a predicted content server score by dividing the content access index by the number of servers to which content distribution is allocated plus the number of servers that are to be allocated A function for calculating
If the content server score is lower than the predicted content server score is extracted as a deletion candidate, and the deletion candidate is deleted, the server load is expected not to exceed the allowable value even if a replica is created, The content server score has a function of creating a replica of a content after deleting the replica until it is predicted that the load of the server does not exceed an allowable value even if the replica is created in order from the lowest The content arrangement management program according to any one of claims 28 to 33. When allocating a predetermined server in the content distribution system as a person in charge of content distribution, the server has a function of selecting a server with the lowest server load or the largest allowable server load as a distribution allocation destination The content arrangement management program according to claim 34, wherein: 36. The content management apparatus according to claim 28, further comprising: a function for setting content autonomous location management content for each content as a policy that is a combination of a condition and processing content at that time. The content arrangement management program described. Set autonomous content management content as a policy that is a combination of conditions and processing content at that time,
(1) Fixed policy (condition: when selected as a deletion candidate during a set period / action: not to be deleted from the disk or to be reduced to the number of servers in charge of distribution),
(2) New replica creation policy (Condition: The content access index exceeds the set threshold during the set period, and the content replica is not in the distribution management system / Action: Process for creating the replica is executed )
(3) Replica expansion policy (Condition: When the value obtained by dividing the content access index by the set threshold exceeds the number of replicas during the set period / Action: Create another replica in the distribution management system Execute the process for)
(4) Replica deletion policy (Condition: When the content access index falls below the set threshold during the set period / Action: Delete the set number of replicas from the disk),
(5) New distribution server allocation policy (Condition: The content access index exceeds the set threshold during the set period, and the content distribution server is not in the distribution management system / Action: Allocate distribution server Execute the process.
(6) Distribution server expansion policy (Condition: When the value divided by the set threshold for the content access index exceeds the number of distribution servers during the set period / Action: Distribution of content in the distribution management system Execute the process to increase the number of servers in charge by one),
(7) Distribution charge reduction policy (Condition: When the content access index falls below the set threshold during the set period / Action: Reduce the number of distribution charge servers by the set number)
37. The content arrangement management program according to claim 28, further comprising a function of selecting and setting from any one of the policies. The content distribution management program according to any one of claims 28 to 37, wherein a proxy cache server is provided in the content distribution system, and a part of content is distributed via the proxy cache server. . The server in the content distribution system functions as a proxy cache server, and content replica creation and deletion instructions are indirectly executed by guidance by a load balancer. The content arrangement management program according to claim 1. The disk load is monitored, and if there is a disk whose observed disk load exceeds the allowable value of the disk load, a replica of the content with the largest content access index is created in another content distribution system. The content arrangement management program according to any one of claims 28 to 39. The server load is monitored, and when there is a server whose observed server load exceeds an allowable value of the server load, the number of servers in charge of distributing a replica of a content having the largest content access index is increased. The content arrangement management program according to any one of claims 28 to 40.

Images