CN113676514B

CN113676514B - File source returning method and device

Info

Publication number: CN113676514B
Application number: CN202110816461.6A
Authority: CN
Inventors: 贾鹏; 吴祖龙
Original assignee: Beijing QIYI Century Science and Technology Co Ltd
Current assignee: Beijing QIYI Century Science and Technology Co Ltd
Priority date: 2021-07-19
Filing date: 2021-07-19
Publication date: 2023-05-02
Anticipated expiration: 2041-07-19
Also published as: CN113676514A

Abstract

The embodiment of the invention provides a method and a device for file source return, which are applied to an agent end in a file source return system and comprise the following steps: acquiring a file downloading request, wherein the file downloading request comprises a file identifier; under the condition that the proxy end does not store the file corresponding to the file identifier, determining a second proxy end from all the first proxy ends according to the weight value of the first proxy end contained in the last level of the level where the proxy end is located and a preset load balancing algorithm; and forwarding the file downloading request to the second proxy end. In the invention, the weight value of the first proxy end is configured according to the performance index of the first proxy end and is used for controlling the probability of the first proxy end to process the file downloading request in the load balancing algorithm, so that the proxy end with better performance index can be determined from the first proxy end according to the weight value to serve as the second proxy end, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher and ensuring the service quality of the file source returning process.

Description

File source returning method and device

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a method and an apparatus for file source retrieval.

Background

With the rapid development of internet technology, content delivery network (Content Distribution Network, CDN) technology is widely used for delivering video files to provide video streaming services.

The CDN network includes a plurality of video servers as cascaded nodes, and the client may send a file download request to an edge node in the CDN network to request a specified video file. After receiving the file downloading request, the edge node sends the video file to the client if the video file corresponding to the file downloading request is stored in a storage medium local to the edge node, and further forwards the file downloading request to a certain video server in the upper cascade level (namely an upstream machine room) if the video file is not stored, so as to request to acquire the video file. To ensure that the number of file download requests received by the plurality of video servers in the upstream room is uniform, a consistent hashing algorithm may be used to determine a target server from the plurality of servers in the upstream room and forward the file download request to the target server, and if the video file is still not stored in the target server, further forward the target download request to a server in the upstream room of the target server until the video file requested by the client is found, a process referred to as source-back.

However, in the prior art, the uniformity of the number of file downloading requests received by multiple video servers in an upstream machine room cannot be guaranteed in a shorter time by adopting a consistent hash method, so that the number of file downloading requests received by individual video servers is large, the service pressure of the video servers is large, and if the load of the video servers is heavy, the service pressure of the video servers is further increased, thereby influencing the working performance of the servers.

Disclosure of Invention

The embodiment of the invention aims to provide a file source returning method and device, which are used for solving the problem that in the prior art, when file source returning is carried out, the number of file downloading requests received by a plurality of video servers of an upstream machine room is uneven, so that the working performance of individual video servers is affected. The specific technical scheme is as follows:

in a first aspect of the present invention, there is provided a file source-returning method, applied to a proxy end in a file source-returning system, where the file source-returning system further includes a server end, the proxy end is deployed at a node in a content distribution network, and the nodes in the content distribution network form a multi-level tree structure, and the method includes:

Acquiring a file downloading request, wherein the file downloading request comprises a file identifier;

under the condition that the proxy end does not store the file corresponding to the file identifier, determining a second proxy end from all the first proxy ends according to a weight value of the first proxy end contained in the previous level of the level where the proxy end is located and a preset load balancing algorithm;

forwarding the file downloading request to the second proxy end;

the weight value of the first proxy end is configured by the server end according to the performance index of the first proxy end and is used for controlling the probability of the first proxy end to process the file downloading request in the load balancing algorithm.

In a second aspect of the present invention, a method for file source-returning is provided, which is applied to a server in a file source-returning system, where the file source-returning system further includes a proxy, the proxy is deployed at a node in a content distribution network, and the nodes in the content distribution network form a multi-level tree structure, and the method includes:

according to the structure of the content distribution network, determining a first proxy end contained in the upper layer of the layer where the proxy end is located, and acquiring a performance index of the first proxy end;

Determining a weight value of the first proxy end according to the performance index of the first proxy end, and sending the weight value to the proxy end;

the weight value of the first proxy end is used for controlling the probability of the first proxy end processing the file downloading request in a load balancing algorithm, and the probability of the first proxy end processing the file downloading request is in direct proportion to the superiority of the performance index of the first proxy end.

In a third aspect of the present invention, there is further provided a file source-returning device, applied to a proxy end in a file source-returning system, where the file source-returning system further includes a server end, the proxy end is deployed at a node in a content distribution network, and the nodes in the content distribution network form a multi-level tree structure, and the device includes:

the first acquisition module is used for acquiring a file downloading request, wherein the file downloading request comprises a file identifier;

the first determining module is used for determining a second proxy end from all the first proxy ends according to a weight value of the first proxy end contained in the previous level of the level where the proxy end is located and a preset load balancing algorithm under the condition that the proxy end does not store the file corresponding to the file identifier;

The forwarding module is used for forwarding the file downloading request to the second proxy end;

In a fourth aspect of the present invention, there is also provided a file source-returning device applied to a server in a file source-returning system, the file source-returning system further including a proxy, the proxy being disposed at a node in a content distribution network, the nodes of the content distribution network forming a multi-level tree structure, the device comprising:

a third obtaining module, configured to determine, according to a structure of the content distribution network, a first proxy end included in a level above a level where the proxy end is located, and obtain a performance index of the first proxy end;

a fourth determining module, configured to determine a weight value of the first proxy according to a performance index of the first proxy, and send the weight value to the proxy;

In a fifth aspect of the present invention, there is also provided a file source-returning system, including: the system comprises a server and a proxy, wherein the proxy is deployed at nodes in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure;

the server determines a first proxy end contained in the upper layer of the layer where the proxy end is located according to the structure of the content distribution network, and determines a weight value of the first proxy end according to the acquired performance index of the first proxy end, wherein the weight value of the first proxy end is used for controlling the probability of the first proxy end to process a file downloading request in a load balancing algorithm, and the probability of the first proxy end to process the file downloading request is in direct proportion to the superiority of the performance index of the first proxy end;

the server side sends the weight value of the first proxy side to the proxy side;

the proxy end receives a weight value of a first proxy end sent by the server end, and acquires the file downloading request, wherein the file downloading request comprises a file identifier;

the agent end determines a second agent end from all the first agent ends according to the weight value of the first agent end and the load balancing algorithm under the condition that the agent end does not store the file corresponding to the file identifier;

And the proxy forwards the file downloading request to the second proxy.

In yet another aspect of the present invention, there is also provided an electronic device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

the memory is used for storing a computer program;

the processor is configured to execute the file source returning method according to any one of the above when executing the program stored in the memory.

In yet another aspect of the present invention, there is also provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements any of the file source retrieval methods described above.

Compared with the prior art, the file source returning method and device have the following advantages:

the embodiment of the invention provides a file source returning method and a device, which are applied to an agent end in a file source returning system, wherein the file source returning system also comprises a server end, the agent end is deployed at a node in a content distribution network, the node of the content distribution network forms a multi-level tree structure, and the method comprises the following steps: acquiring a file downloading request, wherein the file downloading request comprises a file identifier; under the condition that the proxy end does not store the file corresponding to the file identifier, determining a second proxy end from all the first proxy ends according to the weight value of the first proxy end contained in the last level of the level where the proxy end is located and a preset load balancing algorithm; forwarding the file downloading request to a second proxy end; the weight value of the first proxy terminal is configured by the server terminal according to the performance index of the first proxy terminal and is used for controlling the probability of the first proxy terminal to process the file downloading request in the load balancing algorithm. In the embodiment of the invention, in the process of file source returning, the proxy end can determine the second proxy end for source returning according to the weight value of each first proxy end and the load balancing algorithm, and because the weight value of the first proxy end is configured according to the performance index of the first proxy end and is used for controlling the probability that the first proxy end processes the file downloading request in the load balancing algorithm, the proxy end with better performance index can be determined from the first proxy end according to the weight value and used as the second proxy end, namely, the determined conditions of all performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher and ensuring the service quality of the file source returning process.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.

FIG. 1 is a flowchart illustrating steps of a method for file source retrieval according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a content distribution network according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a file source returning process according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating steps of a method for file source retrieval according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating steps of another method for file source retrieval according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating steps of another method for file source retrieval according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a consistent hashing algorithm in accordance with an embodiment of the present invention;

FIG. 8 is a flowchart illustrating the interaction steps of a file source returning method according to an embodiment of the present invention;

FIG. 9 is a block diagram illustrating a file source device according to an embodiment of the present invention;

FIG. 10 is a block diagram illustrating another exemplary embodiment of a file source device;

FIG. 11 is a schematic diagram of a file source return system according to an embodiment of the present invention;

fig. 12 is a block diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

Referring to fig. 1, a flowchart of steps of a file source returning method according to an embodiment of the present invention is shown.

The file source returning method provided by the embodiment of the invention is applied to a proxy end in a file source returning system, wherein the file source returning system also comprises a server end, the proxy end is deployed at a node in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure.

In the embodiment of the present invention, fig. 2 shows a schematic structural diagram of a content distribution network according to the embodiment of the present invention, where a video content distribution network refers to a set of systems capable of providing video services, and generally includes systems such as storage, synchronization, source-returning, scheduling, and video file parsing. Referring to fig. 2, the client 10 views a video, and the downloading of a video stream file is performed in the following manner: after the client 10 initiates a file downloading request, it first takes a video file analysis list from the video file analysis server, and then downloads the files in the list in turn. When downloading each file, a request needs to be made to the dispatcher, and the file is downloaded by the appointed video server positioned at the CDN edge node according to the result returned by the dispatcher. For example, when a file download request initiated by the client 10 reaches the video server 2, if the file is stored in the video server 2, the data is directly output to the client; if the video server 2 does not store the file, the video server 2 needs to send a file downloading request to the video server included in the previous hierarchy, in general, the video server 2 selects one video server from the video server 5, the video server 6 and the video server 7 included in the previous hierarchy to perform file source returning, for example, the video server 5 can be determined as a source node so as to forward the file downloading request to the video server 5, if the file is stored in the video server 5, the video server 5 sends the file to the video server 2, after the video server 2 receives the file returned by the video server 5, data is spitted to the client 10, and meanwhile, the file is stored locally, and if the same file downloading request exists next time, the video server 2 can directly return the file; if the video server 5 does not store the file either, a file download request needs to be further sent to the video server 5 above and including the video server, for example, the video server 8 may be determined as a back source node from the video server 8 and the video server 9, and the file download request may be forwarded to the video server 8 to further determine whether the file is stored in the video server 8 until the root node video server 10, where the video server 10 has to store the file. The file downloading manner described above is referred to as file back source.

The links of the requests finally form a CDN system with a multi-level tree structure, and each video server comprises a proxy end for specifically executing each step, so that the proxy ends are deployed at nodes in the CDN system.

Step 101, obtaining a file downloading request, wherein the file downloading request comprises a file identifier.

In this step, the proxy may obtain a file download request, where the file download request includes a file identifier, so that it may be determined, according to the file identifier, whether the local area of the proxy stores a file corresponding to the file identifier.

Fig. 3 shows a schematic diagram of a file source returning process according to an embodiment of the present invention, as shown in fig. 3, where an agent 1, an agent 2, an agent 3, and an agent 4 in a downstream machine room are located at edge nodes in a CDN system, where any one agent may directly obtain a file downloading request from a client 10; any one of the proxy end 5, the proxy end 6 and the proxy end 7 in the upstream machine room needs to acquire the file downloading request from a certain proxy end in the downstream machine room (i.e. a child node of the proxy end in the upstream machine room).

Step 102, determining a second proxy end from all the first proxy ends according to a weight value of the first proxy end and a preset load balancing algorithm contained in a previous level of the level where the proxy end is located under the condition that the proxy end does not store the file corresponding to the file identifier.

In the step, after the agent terminal obtains the file downloading request, the agent terminal can firstly judge whether the agent terminal stores the file corresponding to the file identifier, if the agent terminal stores the file, the agent terminal can directly send the file to the client terminal, and the downloading of the file is completed; if the proxy end does not store the file, the file downloading request can be forwarded to one proxy end in the proxy end contained in the previous hierarchy of the proxy end through the file return source so as to acquire the file.

Specifically, referring to fig. 3, if the proxy end that currently receives the file downloading request is the proxy end 3, after the proxy end 3 obtains the file downloading request from the client end 10, it may determine whether the proxy end 3 stores a file corresponding to the file identifier according to the file identifier in the file downloading request, if so, the proxy end sends the file to the client end 10, and the downloading of the file is completed; if not, a second proxy is determined from the first proxy 30 included in the previous hierarchy of the proxy 3, and the file download request is forwarded to the second proxy. Further, whether a file corresponding to the file identifier is stored in the second proxy end can be determined, if so, the file is sent to the proxy end 3 so that the proxy end 3 can store the file, and the file is sent to the client end 10 to finish downloading the file; if not, determining that a certain proxy end forwards the file downloading request from the proxy end contained in the upper layer of the second proxy end until the file corresponding to the file identifier is obtained.

Further, in the case that the proxy 3 does not store the file corresponding to the file identifier, the process of determining the second proxy from the first proxy 30 may be that the proxy 3 determines according to the weight value of the first proxy 30 and a preset load balancing algorithm.

The weight value of the first proxy end is configured by a server end in the file source system according to the performance index of the first proxy end, and is used for controlling the probability of the first proxy end to process the file downloading request in a load balancing algorithm.

Referring to fig. 3, the file source-returning system includes a proxy end and a server end 20, where the server end 20 may obtain a performance index of the first proxy end 30 through a message middleware 40, and further may determine a weight value of the first proxy end 30 according to the performance index of the first proxy end 30. The server 20 may be deployed in one or two servers in a centralized manner, and if deployed in two servers, the server may achieve high availability. Message middleware 40 is a separate system software or service program, which is software that connects two separate applications or separate systems. The performance index of the first proxy 30 may include any one or more of a load capacity of the CPU of the first proxy 30, a disk spare capacity, a memory, and a consumption ratio of waiting for disk reading and writing to the CPU, so as to represent a service pressure of a server corresponding to the first proxy 30, where the greater the load capacity of the CPU of the first proxy 30, the less the disk spare capacity, the less the memory, and the greater the consumption ratio of waiting for disk reading and writing to the CPU, the worse the superiority of the performance index of the first proxy 30 is illustrated, and the greater the service pressure thereof is. Therefore, according to the performance index of each first proxy end 30, a proxy end with a better performance index is selected from the plurality of first proxy ends 30 as a second proxy end, and the second proxy end is adopted to process the current file downloading request because the current service pressure of the second proxy end is smaller, so that the influence on the working performance of the second proxy end due to the larger service pressure of the second proxy end can be avoided, and the service quality of the current file source returning process is ensured. Therefore, even if the agent ends cannot be ensured to acquire and process file downloading requests uniformly in a short time, each agent end can be ensured not to influence the working performance of the agent end due to overlarge service pressure.

Specifically, according to the performance index of each first proxy end, selecting a proxy end with a better performance index from a plurality of first proxy ends as a second proxy end, and specifically, determining a weight value of the first proxy end according to the performance index of the first proxy end, and determining the second proxy end from the first proxy ends according to the weight value of the first proxy end and a load balancing algorithm.

The load balancing algorithm is to distribute data or requests to a plurality of operation units for execution, referring to fig. 3, that is, distribute a plurality of file download requests acquired by an agent in a downstream machine room to a plurality of first agent terminals 30 for processing, instead of distributing the plurality of file download requests to one of the first agent terminals 30 in a centralized manner for processing. The weight value of the first proxy end is used for controlling the probability that the first proxy end can process the file downloading request in a load balancing algorithm, namely, the higher the weight value of the first proxy end is, the higher the probability that the first proxy end can process the file downloading request is, and the lower the weight value of the first proxy end is, the lower the probability that the first proxy end can process the file downloading request is.

Therefore, after the server obtains the performance index of the first proxy end, the server can also process the file downloading request at the moment because the service pressure of the first proxy end with excellent performance index is smaller, so that a higher weight value can be configured for the first proxy end with excellent performance index, and the probability of the first proxy end receiving and processing the file downloading request is increased; because the service pressure of the first proxy end with poor performance index is higher, the first proxy end is not suitable for processing the file downloading request any more, and a lower weight value can be configured for the first proxy end with poor physical performance index, so that the probability that the first proxy end receives and processes the file downloading request is reduced.

And step 103, forwarding the file downloading request to the second proxy end.

In this step, after the second proxy end corresponding to the proxy end is determined, the file downloading request may be further forwarded to the second proxy end, so that the second proxy end further obtains the file corresponding to the file identifier, thereby completing the source returning and downloading process of the file.

For example, referring to fig. 3, after the agent 3 obtains a file download request from the client 10 and determines that a file corresponding to a file identifier is not stored in the agent 3, the agent 5, the agent 6, and the agent 7 included in a level above the level where the agent 3 is located may be first determined as the first agent 30, and the weight values of the agent 5, the agent 6, and the agent 7 are obtained from the server 20 through the message middleware 40, and if further according to the weight values of the agent 5, the agent 6, and the agent 7 and the load balancing algorithm, it is determined that the second agent is the agent 6 from the weight values of the agent 5, the agent 6, and the agent 7, then the file download request may be directly forwarded to the agent 6.

Referring to fig. 3, in a file source returning process based on the CDN system, generally, an upstream machine room has a plurality of first proxy ends, and if a proxy end in a downstream machine room does not find a file corresponding to a file download request, a load balancing algorithm is used to forward the file download request to a second proxy end in the plurality of first proxy ends in the upstream machine room. The load balancing algorithm may be a hash algorithm or a consistent hash algorithm, and when the consistent hash algorithm is used for file source returning, the consistent hash may be performed using a uniform resource identifier (Uniform Resource Identifier, URI) of the file as a key value. The consistent hash can ensure that file downloading requests forwarded by all agent ends in a downstream machine room can be uniformly forwarded to all first agent ends in an upstream machine room in a longer period of time, so that on one hand, the disc utilization rate of all the first agent ends is ensured to be uniform, and on the other hand, the read-write use of all the first agent ends is ensured to be uniform, thereby ensuring that the service pressure of a video server corresponding to each first agent end is uniform.

However, this consistent hashing method can ensure that file download requests can be uniformly distributed on each first proxy end in the upstream machine room for a longer period of time, but in a shorter time (such as the order of minutes), it is likely that some file download requests will be concentrated on individual first proxy ends, resulting in a higher service pressure of the video servers corresponding to these first proxy ends. For example, there is a resource file that needs to be transcoded in real time, which consumes resources of a central processing unit (central processing unit, CPU), and when there is an excessive amount of such file that is concentrated to some first proxy end, the load of the CPU of the video server corresponding to the first proxy end will rise, so as to affect the reverse proxy (ngix) service of the video server, so that the hypertext transfer protocol (Hypertext Transfer Protocol, HTTP) file request response of the video server is slow, and if the system level does not have a health policy, the health policy will find the video servers and take them off line. After the video servers are offline, the number of online video servers in the machine room is reduced, so that the situation is further deteriorated, more video servers are offline, and the short-time consistency hash is uneven, so that the service index of the whole machine room is reduced for a long time.

In summary, the method for file source returning provided by the embodiment of the present invention is applied to a proxy end in a file source returning system, where the file source returning system further includes a server end, the proxy end is disposed at a node in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure, and the method includes: acquiring a file downloading request, wherein the file downloading request comprises a file identifier; under the condition that the proxy end does not store the file corresponding to the file identifier, determining a second proxy end from all the first proxy ends according to the weight value of the first proxy end contained in the last level of the level where the proxy end is located and a preset load balancing algorithm; forwarding the file downloading request to a second proxy end; the weight value of the first proxy terminal is configured by the server terminal according to the performance index of the first proxy terminal and is used for controlling the probability of the first proxy terminal to process the file downloading request in the load balancing algorithm. In the embodiment of the invention, in the process of file source returning, the proxy end can determine the second proxy end for source returning according to the weight value of each first proxy end and the load balancing algorithm, and because the weight value of the first proxy end is configured according to the performance index of the first proxy end and is used for controlling the probability that the first proxy end processes the file downloading request in the load balancing algorithm, the proxy end can be used as the second proxy end with better performance index determination from the first proxy end according to the weight value, namely, the determined conditions of all performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher and ensuring the service quality of the file source returning process.

Referring to fig. 4, a flowchart illustrating specific steps of a file source returning method according to an embodiment of the present invention is shown.

Step 201, obtaining a performance index of the proxy end.

In this step, since the server determines the weight value according to the performance index of the proxy, the server may first obtain the performance index of each proxy.

Specifically, a new module for detecting the performance index change of each proxy end can be used in the proxy end, so that the performance index of the proxy end is reported in real time, and the timeliness of the performance index of the proxy end is improved; the server side can also obtain the performance index of the proxy side by sending a performance index detection request to the proxy side, that is, the proxy side can receive the performance index detection request sent by the server side and obtain the performance index of the proxy side according to the performance index detection request, so that resources can be saved.

Step 202, sending the performance index of the proxy end to the server end.

In this step, after the proxy end obtains the performance index, the performance index of the proxy end may be sent to the server end, so that the server end may determine the weight value according to the performance index of the proxy end, and correspondingly, a central control service may be added to the server end to collect the performance indexes of the proxy ends reported by the proxy ends.

In step 203, a file download request is obtained, the file download request includes a file identification.

The step may refer to step 101, and will not be described herein.

And 204, sending the file downloading request to other proxy ends which are positioned in the same machine room as the proxy end under the condition that the proxy end does not store the file, so that the other proxy ends feed back the file to the proxy end under the condition that the file is stored.

In this step, after the proxy end obtains the file downloading request in step 203 and determines that the file corresponding to the file identifier is not stored in the proxy end, if the proxy end directly returns to the first proxy end included in the upstream machine room (i.e. the previous level), the machine room return source bandwidth will be increased.

Therefore, after the agent acquires the file downloading request and determines that the file corresponding to the file identifier is not stored in the agent, the agent can firstly perform the same machine room source return, namely the agent can send the file downloading request to other agent terminals which are positioned in the same machine room with the agent terminal in a broadcasting mode, if the other agent terminals store the file, the file can be directly fed back to the agent terminal, namely the internal broadcasting search is preferably performed in the same machine room, if the file exists in the same machine room, the file source return is not required to be performed in an upstream machine room, the source return process between the machine rooms is avoided, and the source return bandwidth is saved.

In addition, if the internal broadcast search is performed in the same machine room, and the file is not stored in other agent terminals in the same machine room, step 205 may be further performed to perform file source recovery on the upstream machine room.

Step 205, determining a second proxy end from all the first proxy ends according to a weight value of the first proxy end and a preset load balancing algorithm contained in a previous level of the level where the proxy end is located under the condition that the proxy end does not store the file corresponding to the file identifier.

The step may refer to step 102, and will not be described herein.

And step 206, forwarding the file downloading request to the second proxy end.

This step can be specifically referred to step 103, and will not be described herein.

Step 207, receiving and storing the file returned by the second proxy end under the condition that the second proxy end stores the file.

After step 206, it may further be determined whether the second proxy end stores the file corresponding to the file identifier, if so, step 207 is executed, and if not, step 209 or 210 is executed.

In this step, if it is determined that the second proxy end stores the file corresponding to the file identifier, the second proxy end may directly return the file to the proxy end, so that the proxy end stores the file.

And step 208, the file is sent to the client or the proxy corresponding to the child node of the proxy.

In this step, after the proxy acquires and stores the file, the proxy may further send the file to the client or the proxy corresponding to the child node of the proxy.

Specifically, if the client is located at an edge node in the CDN system, the proxy may directly send the file to the client that generates the file download request, so as to complete downloading the file; if the client is not located at an edge node in the CDN system, that is, the proxy end has a child node in a next level in the CDN system, the proxy end may send the file to a proxy end corresponding to the child node of the proxy end, so that the proxy end corresponding to the child node may further send the file to the client that generates the file download request, thereby completing downloading the file.

Step 209, determining a fourth proxy from all third proxy terminals according to a weight value of the third proxy terminal included in a previous level of the level where the second proxy terminal is located and a preset load balancing algorithm, so that the second proxy terminal forwards the file downloading request to the fourth proxy terminal.

In this step, in step 206, the proxy end forwards the file downloading request to the second proxy end, that is, after the second proxy end receives the file downloading request, and determines that the second proxy end does not store the file corresponding to the file identifier, then the second proxy end may further determine a third proxy end located in a layer above the layer where the second proxy end is located and included in the layer, obtain a weight value of the third proxy end from the server end, and further determine a fourth proxy end from all the third proxy ends according to the weight value of the third proxy end and a preset load balancing algorithm, so that the second proxy end forwards the file downloading request to the fourth proxy end.

The weight value of the third proxy end is configured by the server end according to the performance index of the third proxy end, and is used for controlling the probability of the third proxy end to process the file downloading request in the load balancing algorithm.

The process of determining the fourth proxy from all the third proxy is similar to the process of determining the second proxy from the first proxy, and will not be repeated here.

Further, after receiving the file downloading request, if the fourth proxy end stores the file corresponding to the file identifier, the fourth proxy end can forward the file to the second proxy end, the second proxy end receives and stores the file, and forwards the file to the proxy end again, and the proxy end receives and stores the file, and sends the file to the client end, so as to complete the file downloading process.

Step 210, when the second proxy end does not store the file, the second proxy end sends the file downloading request to other first proxy ends except the second proxy end in all the first proxy ends, so that the other first proxy ends feed back the file to the second proxy end when the other first proxy ends store the file.

In this step, in step 206, the proxy terminal forwards the file downloading request to the second proxy terminal, that is, after the second proxy terminal receives the file downloading request and determines that the file corresponding to the file identifier is not stored in the second proxy terminal, if the second proxy terminal directly returns to the third generation terminal included in the upstream machine room (i.e., the previous level), the machine room return source bandwidth is increased.

Therefore, after the second proxy end receives the file downloading request and determines that the file corresponding to the file identifier is not stored in the second proxy end, the same machine room source returning can be performed first, that is, the second proxy end can send the file downloading request to other first proxy ends except the second proxy end in all first proxy ends in a broadcasting mode, if the other first proxy ends store the file, the file can be directly fed back to the second proxy end, that is, internal broadcasting searching is performed in the same machine room preferentially, if the file exists in the same machine room, file source returning is not required to be performed in an upstream machine room, a source returning process between the machine rooms is avoided, and source returning bandwidth is saved.

In addition, if the internal broadcast search is performed in the same machine room, and the file is not stored in other agent terminals in the same machine room, step 209 may be further performed to perform file source recovery on the upstream machine room.

Step 211, determining a time range in which the weight value of the first proxy is adjusted.

In the step, in the process of returning the file, as the weight value of the first proxy end in the process of returning the file is dynamically adjusted, the proxy end sends the file downloading request to the second proxy end determined from all the first proxy ends, which can lead to that some files which should not be returned to the second proxy end fall onto the video server corresponding to the second proxy end, so that the files stored in the disk of the video server corresponding to the second proxy end are increased, thereby causing uneven file distribution on the disks of a plurality of first proxy ends. Thus, some elimination policies may be used to ensure that these files are deleted in time.

For example, a least recently used (Least recently used, LRU) policy may be adopted, and each time a preset duration is set, a disk cleaning module set in the proxy end may be used to delete a file in a disk corresponding to the proxy end, so as to ensure that the disk is restored to be normal.

Specifically, the weight value of the first proxy end can be monitored in real time, and the time range in which the weight value of the first proxy end is adjusted is determined, so that files which are not stored in the first proxy end but are stored in the first proxy end due to the fact that the weight value of the first proxy end is adjusted can be deleted in a smaller number of times in a preset time interval, and the fact that files which are not stored in the first proxy end can not cause excessive files stored on a disk of the first proxy end is ensured, and therefore uniformity of file distribution on the disk in each proxy end is ensured.

In addition, files stored in the first proxy end can be not distinguished, namely, all files stored in the first proxy end in the time range of adjusting the weight value of the first proxy end are deleted with fewer times of use in each interval preset time length, so that the uniformity of file distribution on the disk in each proxy end is achieved.

For example, if the weight values of the first proxy ends are not adjusted, that is, the file downloading request is uniformly distributed to the four first proxy ends according to the load balancing algorithm, the initial weight values of the four first proxy ends are 25%, further, after the performance index of one of the first proxy ends is obtained, it is determined that the performance index of the first proxy end is better, a larger weight value may be configured for the first proxy end, for example, the weight value of the first proxy end is adjusted from the initial 25% to 40%, so that the probability of receiving and processing the file downloading request is increased, and meanwhile, the time range of detecting that the weight value of the first proxy end is adjusted is 9 am to 9 pm, that is, some files which should not respond to the source and are stored in the first proxy end in the time range, which results in the increase of files stored in the disk of the video server corresponding to the first proxy end.

Step 212, determining files stored in the time range by the first proxy end every preset time length.

In this step, files stored in the time range by the first proxy end may be determined every preset time period, for example, 2 hours or 10 hours, so that deletion of files is performed according to the LRU policy in the stored files.

And step 213, deleting the files with the use times less than the preset times.

In this step, the file stored in the disk of the video server corresponding to the first proxy end may be deleted according to the LRU policy.

For example, files with the use times less than the preset times in the files determined in the steps can be deleted, that is, files stored in the first agent end in the preset time due to the adjustment of the weight value can be screened every preset time, files with the use times less than the preset times are determined, and the files can be deleted due to the fact that the use times of the screened files are less, so that the disk cannot be abnormal due to the fact that the weight value of the corresponding agent end is adjusted.

For example, if the time range of the adjustment of the weight value of the first proxy end is detected to be 9 am to 9 pm, and the preset duration is 2 hours, the files stored in the first proxy end can be deleted according to the LRU policy every 2 hours in the time range of the adjustment of the weight value of the first proxy end. Specifically, files stored in the first proxy end and used for less than the preset times in the 2 hours can be deleted every 2 hours, so that the storage amount of the files in the first proxy end is ensured not to be too much.

It should be noted that, in the whole process of file source returning, files stored in the disks of the video server corresponding to all the proxy ends in the CDN system are deleted according to the LRU policy at preset intervals, so as to ensure that the least recently used files in the disks corresponding to the multiple proxy ends can be deleted in time, and ensure that the files stored in the disks corresponding to the multiple proxy ends are uniformly distributed.

Wherein, the preset times can be 3 times or 5 times or the like which are preset.

In summary, the method for file source returning provided by the embodiment of the present invention is applied to a proxy end in a file source returning system, where the file source returning system further includes a server end, the proxy end is disposed at a node in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure, and the method includes: acquiring a file downloading request, wherein the file downloading request comprises a file identifier; under the condition that the proxy end does not store the file corresponding to the file identifier, determining a second proxy end from all the first proxy ends according to the weight value of the first proxy end contained in the last level of the level where the proxy end is located and a preset load balancing algorithm; forwarding the file downloading request to a second proxy end; the weight value of the first proxy terminal is configured by the server terminal according to the performance index of the first proxy terminal and is used for controlling the probability of the first proxy terminal to process the file downloading request in the load balancing algorithm. In the embodiment of the invention, in the process of file source returning, the proxy end can determine the second proxy end for source returning according to the weight value of each first proxy end and the load balancing algorithm, and because the weight value of the first proxy end is configured according to the performance index of the first proxy end and is used for controlling the probability that the first proxy end processes the file downloading request in the load balancing algorithm, the proxy end with better performance index can be determined from the first proxy end according to the weight value and used as the second proxy end, namely, the determined conditions of all performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher and ensuring the service quality of the file source returning process.

In addition, in the case that the agent side obtains the file downloading request and does not store the file,

after the agent acquires the file downloading request and determines that the file corresponding to the file identifier is not stored in the agent, the agent can firstly perform the same machine room source return, namely the agent can send the file downloading request to other agent terminals which are positioned in the same machine room with the agent terminal in a broadcasting mode, if the file is stored in the other agent terminals, the file can be directly fed back to the agent terminal, namely the internal broadcasting search is preferably performed in the same machine room, if the file is stored in the same machine room, the file source return is not required to be performed in an upstream machine room, the source return process between the machine rooms is avoided, and the source return bandwidth is saved.

Referring to fig. 5, a flowchart illustrating steps of another file source returning method according to an embodiment of the present invention is shown.

The file source returning method provided by the embodiment of the invention is applied to a server side in a file source returning system, and the file source returning system also comprises a proxy side, wherein the proxy side is deployed at nodes in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure.

Step 301, determining a first proxy end included in a layer above a layer where the proxy end is located according to a structure of the content distribution network, and obtaining a performance index of the first proxy end.

In this step, the server may acquire the structure of the content distribution network in advance, so that the first proxy end included in the upper layer of the layer where the proxy end is located may be determined according to the structure of the content distribution network, so as to further acquire the performance index of the first proxy end.

Referring to fig. 2, since each node in the CDN is deployed with each proxy, the nodes of the CDN form a multi-level tree structure. Therefore, if the proxy end that obtains the file download request is the proxy end 2, according to the structure of the CDN, it may be determined that the first proxy end 30 corresponding to the proxy end 2 includes the proxy end 5, the proxy end 6, and the proxy end 7; if the agent end that obtains the file downloading request is the agent end 5, according to the CDN structure, it may be determined that the first agent end corresponding to the agent end 5 includes: proxy 8 and proxy 9.

Further, after determining the first proxy corresponding to the proxy, referring to fig. 3, the server 20 may obtain, through the message middleware, the performance index of each first proxy 30.

Step 302, determining a weight value of the first proxy according to the performance index of the first proxy, and sending the weight value to the proxy.

In this step, after the server side obtains the performance index of the first proxy side, the server side may determine the weight value of the first proxy side according to the performance index of the first proxy side, and send the weight value to the proxy side.

Specifically, the performance index of the first proxy end may include any one or more of a load capacity of the CPU of the first proxy end, a disk spare capacity, a memory and a consumption ratio of waiting for a disk to read and write on the CPU, so as to represent a service pressure of a server corresponding to the first proxy end. For example, the greater the load capacity of the CPU of the first proxy end, the smaller the disk free capacity, the smaller the memory, and the greater the consumption ratio of waiting for the disk to read/write to the CPU, the smaller the superiority of the performance index of the first proxy end is, the greater the service pressure is, that is, the lower the capability level of the first proxy end for processing the file download request is, the smaller weight value can be configured for the first proxy end, for example, the weight value of the first proxy end is adjusted from the initial 25% to 10%, so that the probability of receiving and processing the file download request is reduced; the smaller the load capacity of the CPU of the first proxy end, the larger the spare capacity of the disk, the larger the memory, and the smaller the consumption ratio of the waiting disk read-write to the CPU, the better the performance index of the first proxy end is, the smaller the service pressure is, that is, the higher the capability level of the first proxy end for processing the file download request is, the larger weight value can be configured for the first proxy end, for example, the weight value of the first proxy end is adjusted from the initial 25% to 40%, so that the probability of receiving and processing the file download request is increased.

Further, the server can send the determined weight value of the first proxy end to the proxy end, so that the proxy end can select the proxy end with better performance index from the plurality of first proxy ends as the second proxy end according to the weight value of each first proxy end, and the second proxy end is adopted to process the current file downloading request because the current service pressure of the second proxy end is smaller, thereby avoiding the influence on the working performance of the second proxy end due to the larger service pressure of the second proxy end and ensuring the service quality of the current file source returning process. Therefore, even if the agent ends cannot be ensured to acquire and process file downloading requests uniformly in a short time, each agent end can be ensured not to influence the working performance of the agent end due to overlarge service pressure.

In summary, the method for file source returning provided in the embodiment of the present invention is applied to a server in a file source returning system, where the file source returning system further includes a proxy, the proxy is disposed at a node in a content distribution network, and the nodes in the content distribution network form a multi-level tree structure, and includes: according to the structure of the content distribution network, determining a first proxy end contained in the upper layer of the layer where the proxy end is located, and acquiring a performance index of the first proxy end; determining a weight value of a first proxy end according to the performance index of the first proxy end, and sending the weight value to the proxy end; the weight value of the first proxy end is used for controlling the probability of the first proxy end to process the file downloading request in a load balancing algorithm. In the embodiment of the invention, in the process of file source return, the second proxy end for receiving and processing the file downloading request is determined according to the weight values of all the first proxy ends and the load balancing algorithm, and the weight values of the first proxy ends are configured according to the performance indexes of the first proxy ends and are used for controlling the probability of the first proxy ends for processing the file downloading request in the load balancing algorithm, so that the proxy end with better performance indexes can be determined from the first proxy ends according to the weight values to serve as the second proxy end, namely, the determined conditions of all the performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher, and ensuring the service quality of the file source return process.

Referring to fig. 6, a flowchart illustrating specific steps of another file source-returning method according to an embodiment of the present invention is shown.

Step 401, determining a first proxy end included in a level above a level where the proxy end is located according to a structure of the content distribution network.

This step may refer to step 301 specifically, and will not be described herein.

Step 402, determining a performance index change amplitude corresponding to the first proxy terminal according to the performance index of the first proxy terminal under the condition that the load balancing algorithm is a consistent hash algorithm.

In this step, the server may obtain the performance index of the first proxy in advance, and the process that the server obtains the performance index of the first proxy may report the performance index to the server in real time for the first proxy, or may obtain the performance index of the first proxy and feed back the performance index to the server after the first proxy receives the performance index detection request after the server sends the performance index detection request to the first proxy. After receiving the performance index of the first proxy, the server may further determine the weight value of the first proxy according to the performance index of the first proxy.

In this step, in the case that the load balancing algorithm is a consistent hash algorithm, the weight value of the first proxy end determined by the server end may be related to the number of virtual nodes corresponding to each first proxy end in the calculation process of the consistent hash algorithm.

Specifically, the server may first determine, according to the received performance index of the first proxy, a change amplitude of the performance index corresponding to the first proxy.

For example, if the performance index of the first proxy end is the load capacity of the CPU, if the load capacity of the CPU of the first proxy end is changed from 30% to 50%, that is, the change amplitude of the load capacity of the CPU of the first proxy end is 20%, the service pressure of the first proxy end is increased; and under the condition that the performance index of the first proxy end is the consumption proportion of the waiting disk read-write to the CPU, if the consumption proportion of the waiting disk read-write to the CPU of the first proxy end is changed from 30% to 15%, namely the change amplitude of the consumption proportion of the waiting disk read-write to the CPU of the first proxy end is-15%, the service pressure of the first proxy end is reduced.

Step 403, determining the number of virtual nodes corresponding to the first proxy end according to the performance index change amplitude when the performance index change amplitude is greater than a preset amplitude.

In this step, after the performance index change amplitude of the first proxy end is determined, the number of virtual nodes corresponding to the first proxy end may be determined according to the performance index change amplitude of the first proxy end when the performance index change amplitude of the first proxy end is greater than a preset amplitude.

For example, if the number of initial virtual nodes corresponding to the first proxy end is 500, if the consumption ratio of the first proxy end waiting for the disk read-write to the CPU changes from 30% to 15%, that is, the change amplitude of the consumption ratio of the first proxy end waiting for the disk read-write to the CPU is-15%, which is greater than the preset amplitude ±10%, that is, the service pressure of the first proxy end is reduced to a larger extent, more virtual nodes can be allocated to the first proxy end, for example, the number of virtual nodes corresponding to the first proxy end is increased from initial 500 to 600, so that the probability of receiving and processing a file downloading request by the first proxy end is improved; if the load of the CPU of the first proxy terminal changes from 30% to 50%, that is, the change amplitude of the load of the CPU of the first proxy terminal is 20%, which is greater than the preset amplitude ±10%, that is, the amplitude of the service pressure increase of the first proxy terminal is greater, fewer virtual nodes may be allocated to the first proxy terminal, for example, the number of virtual nodes corresponding to the first proxy terminal is reduced from initial 500 to 400, so that the probability of the first proxy terminal receiving and processing the file download request is reduced.

The virtual nodes are virtual nodes which are arranged when the proxy end calculates the consistent hash algorithm for the first proxy end and are used for processing the file downloading request, and the number of the virtual nodes corresponding to the first proxy end can represent the probability that the first proxy end receives and processes the file downloading request in the calculation process of the consistent hash algorithm.

Specifically, FIG. 7 is a schematic diagram of a consistent hashing algorithm, which is a special hashing algorithm, for solving the problem of distributed caching, according to an embodiment of the present inventionThe title comprises determining hash value of each first proxy end, and configuring it to 0-2 ³² Referring to fig. 7, for example, two first proxy ends exist corresponding to the proxy end: the first proxy 1 and the first proxy 2 may perform hash value calculation according to the internet protocol (Internet Protocol, IP) addresses, or the media access control (Media Access Control, MAC) addresses, of the first proxy 1 and the first proxy 2, respectively, to determine the location on the continuum, and may map the file download request on the continuum, and specifically may perform hash value calculation using the file identifier included in the file download request as a key value, determine the location of the file download request on the continuum, and then perform a clockwise search from the location mapped to by the file download request, so as to determine the found first proxy as the second proxy to receive and process the file download request.

Referring to fig. 7, the file download request 1 corresponds to the first proxy end 2 clockwise, and the file download request 2 corresponds to the first proxy end 1 clockwise, so that the proxy end may forward the file download request 1 to the first proxy end 2 for processing, and forward the file download request 2 to the first proxy end 1 for processing.

Further, when the number of the first proxy ends is small in consideration of the calculation of the consistent hash algorithm, the problem of data inclination caused by uneven node distribution of the first proxy ends on the continuum is easy to occur, so that a virtual node mechanism can be introduced to set a plurality of virtual nodes for each first proxy end, and referring to fig. 7, three virtual nodes can be additionally set for the first proxy end 1, namely a first proxy end 1#1, a first proxy end 1#2 and a first proxy end 1#3; two virtual nodes are additionally set for the first proxy end 2, namely a first proxy end 2#1 and a first proxy end 2#2.

In the embodiment of the invention, if the performance index of the first proxy terminal is detected to be poor, the number of the virtual nodes corresponding to the first proxy terminal can be reduced, so that the probability that the first proxy terminal receives and processes the file downloading request is reduced; if the performance index of the first proxy end is detected to be excellent, the number of virtual nodes corresponding to the first proxy end can be increased, so that the probability that the first proxy end receives and processes the file downloading request is increased.

Step 404, determining a weight value of the first proxy according to the number of virtual nodes corresponding to the first proxy.

In this step, after determining the number of virtual nodes corresponding to the first proxy, the server may further determine the weight value of the first proxy according to the number of virtual nodes corresponding to the first proxy.

For example, if there are four first proxy ends: the virtual nodes corresponding to the first proxy end 1 are 200, the virtual nodes corresponding to the first proxy end 2 are 600, the virtual nodes corresponding to the first proxy end 3 are 250 and the virtual nodes corresponding to the first proxy end 4 are 550. In general, the probability that the four first proxy ends receive the file downloading request is the same, that is, the weight value of each first proxy end is 25%, and according to the number of the corresponding virtual nodes, it can be determined that the weight value corresponding to the first proxy end 1 is 12.5%, the weight value corresponding to the first proxy end 2 is 37.5%, the weight value corresponding to the first proxy end 3 is 15.6%, and the weight value corresponding to the first proxy end 4 is 34.4%.

And step 405, sending the weight value to the proxy end.

In this step, after determining the weight value of the first proxy, the server may further send the weight value of the first proxy to the proxy, so that the proxy determines, according to the weight value of the first proxy and the consistent hash algorithm, a second proxy for receiving and processing the file download request from all the first proxy.

Referring to fig. 8, a flowchart of interaction steps of a file source returning method according to an embodiment of the present invention is shown.

The file source returning method provided by the embodiment of the invention is applied to a file source returning system, and the file source returning system comprises the following steps: the system comprises a service end and a proxy end, wherein the proxy end is deployed at a node in a content distribution network, and the node of the content distribution network forms a multi-level tree structure.

Step 501, the server determines, according to the structure of the content distribution network, a first proxy included in a level above the level where the proxy is located, and determines, according to the obtained performance index of the first proxy, a weight value of the first proxy.

This step is specifically referred to

steps

301 and 302, and will not be described in detail herein.

Step 502, the server sends the weight value of the first proxy to the proxy.

This step is specifically referred to step 302, and will not be described herein.

In step 503, the proxy end receives the weight value of the first proxy end sent by the server end, and obtains the file downloading request, where the file downloading request includes a file identifier.

In this step, the proxy end may receive the weight value of the first proxy end sent by the server end, and further receive the file downloading request sent by the proxy end of the client end or the child node, so that the proxy end may determine, from all the first proxy ends, the second proxy end according to the weight value of the first proxy end and the load balancing algorithm, and forward the file downloading request.

Step 504, the proxy determines, under the condition that the file corresponding to the file identifier is not stored, a second proxy from all the first proxy according to the weight value of the first proxy and the load balancing algorithm.

The step may refer to step 102, and will not be described herein.

Step 505, the proxy forwards the file downloading request to the second proxy.

In summary, the method for file source return provided by the embodiment of the present invention is applied to a file source return system, where the file source return system includes: the system comprises a service end and a proxy end, wherein the proxy end is deployed at a node in a content distribution network, and the node of the content distribution network forms a multi-level tree structure and comprises: the server determines a first proxy end contained in the upper layer of the layer where the proxy end is located according to the structure of the content distribution network, and determines a weight value of the first proxy end according to the acquired performance index of the first proxy end, wherein the weight value of the first proxy end is used for controlling the probability that the first proxy end processes a file downloading request in a load balancing algorithm; the server side sends the weight value of the first proxy side to the proxy side; the proxy end receives a weight value of a first proxy end sent by the server end, and acquires a file downloading request, wherein the file downloading request comprises a file identifier; under the condition that the proxy end does not store the file corresponding to the file identification, determining a second proxy end from all the first proxy ends according to the weight value and the load balancing algorithm of the first proxy end; and the proxy terminal forwards the file downloading request to the second proxy terminal. In the embodiment of the invention, in the process of file source returning, the proxy end can determine the second proxy end for source returning according to the weight value of each first proxy end and the load balancing algorithm, and because the weight value of the first proxy end is configured according to the performance index of the first proxy end and is used for controlling the probability that the first proxy end processes the file downloading request in the load balancing algorithm, the proxy end with better performance index can be determined from the first proxy end according to the weight value and used as the second proxy end, namely, the determined conditions of all performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher and ensuring the service quality of the file source returning process.

On the basis of the embodiment, the embodiment of the invention also provides a file source returning device.

Referring to fig. 9, a block diagram of a file source-returning device 600 according to an embodiment of the present invention is shown and applied to a proxy end in a file source-returning system, where the file source-returning system further includes a server end, the proxy end is deployed at a node in a content distribution network, and the nodes in the content distribution network form a multi-level tree structure, and may specifically include the following modules:

a first obtaining module 601, configured to obtain a file downloading request, where the file downloading request includes a file identifier;

a first determining module 602, configured to determine, when the proxy end does not store a file corresponding to the file identifier, a second proxy end from all the first proxy ends according to a weight value of the first proxy end included in a level previous to the level where the proxy end is located and a preset load balancing algorithm;

a forwarding module 603, configured to forward the file downloading request to the second proxy end;

Optionally, the apparatus 600 further includes:

the second acquisition module is used for acquiring the performance index of the proxy end;

and the first sending module is used for sending the performance index of the proxy end to the server end.

Optionally, the apparatus 600 further includes:

and the second sending module is used for sending the file downloading request to other agent terminals which are positioned in the same machine room as the agent terminal, so that the other agent terminals can feed back the file to the agent terminal under the condition that the other agent terminals store the file.

Optionally, the apparatus 600 further includes:

the second determining module is used for determining a time range in which the weight value of the first proxy end is adjusted;

a third determining module, configured to determine, for each interval of a preset duration, a file stored in the time range by the first proxy end;

and the deleting module is used for deleting the files with the use times less than the preset times in the files.

In summary, the file source returning device provided in the present application is applied to an agent end in a file source returning system, where the file source returning system further includes a server end, the agent end is disposed at a node in a content distribution network, and the node in the content distribution network forms a multi-level tree structure, and includes: acquiring a file downloading request, wherein the file downloading request comprises a file identifier; under the condition that the proxy end does not store the file corresponding to the file identifier, determining a second proxy end from all the first proxy ends according to the weight value of the first proxy end contained in the last level of the level where the proxy end is located and a preset load balancing algorithm; forwarding the file downloading request to a second proxy end; the weight value of the first proxy terminal is configured by the server terminal according to the performance index of the first proxy terminal and is used for controlling the probability of the first proxy terminal to process the file downloading request in the load balancing algorithm. In the embodiment of the invention, in the process of file source returning, the proxy end can determine the second proxy end for source returning according to the weight value of each first proxy end and the load balancing algorithm, and because the weight value of the first proxy end is configured according to the performance index of the first proxy end and is used for controlling the probability that the first proxy end processes the file downloading request in the load balancing algorithm, the proxy end with better performance index can be determined from the first proxy end according to the weight value to serve as the second proxy end, namely, the determined conditions of all performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher and ensuring the service quality of the file source returning process.

Referring to fig. 10, a block diagram of another file source-returning device 700 according to an embodiment of the present invention is shown and applied to a proxy end in a file source-returning system, where the file source-returning system further includes a server end, the proxy end is deployed at a node in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure, and may specifically include the following modules:

a third obtaining module 701, configured to determine, according to a structure of the content distribution network, a first proxy end included in a level above a level where the proxy end is located, and obtain a performance index of the first proxy end;

a fourth determining module 702, configured to determine a weight value of the first proxy according to a performance index of the first proxy, and send the weight value to the proxy;

Optionally, in the case where the load balancing algorithm is a consistent hash algorithm, the fourth determining module 702 specifically includes:

The first determining submodule is used for determining a performance index change amplitude corresponding to the first proxy end according to the performance index of the first proxy end;

the second determining submodule is used for determining the number of virtual nodes corresponding to the first proxy end according to the performance index change amplitude when the performance index change amplitude is larger than a preset amplitude, wherein the virtual nodes are virtual nodes which are set when the proxy end performs consistent hash algorithm calculation on the first proxy end and are used for processing the file downloading request;

and the third determining submodule is used for determining the weight value of the first proxy end according to the number of the virtual nodes corresponding to the first proxy end.

In summary, the file source returning device provided in the present application is applied to a server in a file source returning system, where the file source returning system further includes a proxy end, the proxy end is disposed at a node in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure, and the method includes: according to the structure of the content distribution network, determining a first proxy end contained in the upper layer of the layer where the proxy end is located, and acquiring a performance index of the first proxy end; determining a weight value of a first proxy end according to the performance index of the first proxy end, and sending the weight value to the proxy end; the weight value of the first proxy end is used for controlling the probability of the first proxy end to process the file downloading request in a load balancing algorithm. In the embodiment of the invention, in the process of file source return, the second proxy end for receiving and processing the file downloading request is determined according to the weight values of all the first proxy ends and the load balancing algorithm, and the weight values of the first proxy ends are configured according to the performance indexes of the first proxy ends and are used for controlling the probability of the first proxy ends for processing the file downloading request in the load balancing algorithm, so that the proxy end with better performance indexes can be determined from the first proxy ends according to the weight values to serve as the second proxy end, namely, the determined conditions of all the performance indexes of the second proxy end are better, thereby avoiding influencing the working performance of the second proxy end when the service pressure of the second proxy end is higher, and ensuring the service quality of the file source return process.

Referring to fig. 11, a schematic diagram of a file source return system according to an embodiment of the present invention is shown, where the file source return system includes: the system comprises a service end and a proxy end, wherein the proxy end is deployed at a node in a content distribution network, and the node of the content distribution network forms a multi-level tree structure. The service end is in communication connection with each proxy end in each hierarchy in the content distribution network, so that data transmission is realized.

The embodiment of the present invention also provides an electronic device, as shown in fig. 12, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804,

a memory 803 for storing a computer program;

the processor 801, when executing the program stored in the memory 803, implements the following steps:

the file source returning method is applied to a proxy end in a file source returning system, the file source returning system also comprises a server end, the proxy end is deployed at a node in a content distribution network, the nodes of the content distribution network form a multi-level tree structure, and the method comprises the following steps:

forwarding the file downloading request to the second proxy end;

The file source returning method is applied to a server side in a file source returning system, the file source returning system further comprises a proxy side, the proxy side is deployed at nodes in a content distribution network, the nodes of the content distribution network form a multi-level tree structure, and the method comprises the following steps:

A file back source system, the file back source system comprising: the system comprises a server and a proxy, wherein the proxy is deployed at nodes in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure;

and the proxy forwards the file downloading request to the second proxy.

The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the terminal and other devices.

The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In yet another embodiment of the present invention, a computer readable storage medium is provided, where instructions are stored, which when executed on a computer, cause the computer to perform the file source back method according to any of the above embodiments.

In yet another embodiment of the present invention, a computer program product containing instructions that, when executed on a computer, cause the computer to perform the file source back method of any of the above embodiments is also provided.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. The file source returning method is characterized by being applied to a proxy end in a file source returning system, wherein the file source returning system also comprises a server end, the proxy end is deployed at a node in a content distribution network, the nodes of the content distribution network form a multi-level tree structure, and the method comprises the following steps:

forwarding the file downloading request to the second proxy end;

the weight value of the first proxy end is configured by the server end according to the performance index of the first proxy end and is used for controlling the probability of the first proxy end to process the file downloading request in the load balancing algorithm;

Determining a time range in which the weight value of the first proxy end is adjusted;

determining files stored in the time range by the first proxy end every preset time length;

and deleting the files with the use times less than the preset times.

2. The method according to claim 1, wherein before the step of determining the second agent from all the first agent according to the weight value of the first agent and a preset load balancing algorithm included in a level previous to the level in which the agent is located, the method further comprises:

acquiring performance indexes of the proxy end;

and sending the performance index of the proxy end to the server end.

3. The method of claim 1, wherein in the event that the proxy does not store the file, prior to the step of determining a second proxy from all of the first proxy, the method further comprises:

and sending the file downloading request to other agent terminals which are positioned in the same machine room as the agent terminal, so that the other agent terminals feed back the file to the agent terminal under the condition that the file is stored in the agent terminal.

4. The file source returning method is characterized by being applied to a server side in a file source returning system, wherein the file source returning system further comprises a proxy side, the proxy side is deployed at nodes in a content distribution network, the nodes of the content distribution network form a multi-level tree structure, and the method comprises the following steps:

determining a weight value of the first proxy end according to the performance index of the first proxy end, and sending the weight value to the proxy end; after the agent end determines the time range in which the weight value of the first agent end is adjusted, determining files stored in the time range by the first agent end every preset time length, and deleting files with the use times less than the preset times in the files;

5. The method according to claim 4, wherein in the case that the load balancing algorithm is a consistent hashing algorithm, the step of determining the weight value of the first proxy according to the performance index of the first proxy specifically includes:

determining a performance index change amplitude corresponding to the first proxy end according to the performance index of the first proxy end;

under the condition that the performance index change amplitude is larger than a preset amplitude, adjusting the number of virtual nodes corresponding to the first proxy end according to the performance index change amplitude, wherein the virtual nodes are virtual nodes which are set by the proxy end when the proxy end performs consistent hash algorithm calculation on the first proxy end and are used for processing the file downloading request;

and determining the weight value of the first proxy terminal according to the number of the virtual nodes corresponding to the first proxy terminal.

6. The utility model provides a file returns source device which characterized in that is applied to the proxy end in a file returns source system, file returns source system still includes the server end, the node in the content distribution network is disposed to the proxy end, the node of content distribution network constitutes the hierarchical tree structure, the device includes:

7. The utility model provides a file returns source device which characterized in that is applied to the service end in a file returns source system, file returns source system still includes the proxy end, the node in the content distribution network is disposed to the proxy end, the node of content distribution network constitutes the hierarchical tree structure, the device includes:

a fourth determining module, configured to determine a weight value of the first proxy according to a performance index of the first proxy, and send the weight value to the proxy; after the agent end determines the time range in which the weight value of the first agent end is adjusted, determining files stored in the time range by the first agent end every preset time length, and deleting files with the use times less than the preset times in the files;

8. A file back source system, the file back source system comprising: the system comprises a server and a proxy, wherein the proxy is deployed at nodes in a content distribution network, and the nodes of the content distribution network form a multi-level tree structure;

the proxy terminal forwards the file downloading request to the second proxy terminal;

the agent end determines a time range in which the weight value of the first agent end is adjusted; determining files stored in the time range by the first proxy end every preset time length; and deleting the files with the use times less than the preset times.

9. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the communication bus;

the memory is used for storing a computer program;

the processor is configured to implement the method steps of any one of claims 1-5 when executing a program stored on the memory.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-5.