CN117424907A

CN117424907A - File acquisition method, device, node equipment and computer readable storage medium

Info

Publication number: CN117424907A
Application number: CN202311356331.4A
Authority: CN
Inventors: 白飞飞; 徐文义
Original assignee: Guangzhou Huya Technology Co Ltd
Current assignee: Guangzhou Huya Technology Co Ltd
Priority date: 2023-10-18
Filing date: 2023-10-18
Publication date: 2024-01-19

Abstract

The invention relates to the technical field of peer-to-peer networks, in particular to a file acquisition method, a device, node equipment and a computer readable storage medium, wherein the method comprises the following steps: receiving a file acquisition request sent by a request client; determining a first index node to which a partition to which a shared file to be acquired belongs according to a file identifier and first retrieval information in a file acquisition request; the file identification is sent to the first index node, so that the first index node searches the second index node according to the file identification and the second search information, and the second index node is returned to the request node when the search is successful; if the returned second index node is received, the file identification is sent to the second index node, so that the second index node obtains the shared file to be obtained according to the file identification and the third retrieval information, and finally the shared file is returned to the request client. The invention improves the acquisition efficiency of the shared file in the P2P network.

Description

File acquisition method, device, node equipment and computer readable storage medium

Technical Field

The present invention relates to the technical field of peer-to-peer networks, and in particular, to a method, an apparatus, a node device, and a computer readable storage medium for acquiring a file.

Background

In recent years, competition in the live video-on-demand industry is increasingly stronger, and high bandwidth cost becomes a problem that all companies in the industry have to face, besides live streaming, various scenes such as gift packages, resource packages, video-on-demand files and the like can also generate huge bandwidth cost.

Peer-to-Peer (P2P) technology, also known as Peer-to-Peer technology, relies on computing power and bandwidth of participants in the network instead of aggregating dependencies on fewer servers, providing a good solution for application scenarios of massive users and massive files.

How to improve the efficiency of obtaining the shared file in the P2P network is a problem to be solved by those skilled in the art.

Disclosure of Invention

The invention aims to provide a file acquisition method, a device, node equipment and a computer readable storage medium, which can improve the acquisition efficiency of shared files in a P2P network.

Embodiments of the invention may be implemented as follows:

in a first aspect, the present invention provides a file obtaining method, applied to a request node in a plurality of nodes in a peer-to-peer P2P network, at least one of the nodes is communicatively connected to at least one client, at least one client stores a shared file that can be shared by the P2P network, all the shared files stored by the client are divided into at least one partition in advance, each of the nodes stores first search information, second search information and third search information, the first search information is used for searching a node to which the partition belongs, the second search information is used for searching a node storing third search information of the shared file in the partition of the node, the third search information is used for searching a client to which the shared file stored by the node belongs, and the request node is communicatively connected to the request client, the method includes:

Receiving a file acquisition request sent by the request client, wherein the file acquisition request comprises a file identifier of a shared file to be acquired;

determining a first index node to which a partition to which the shared file to be acquired belongs according to the file identifier and the local first retrieval information;

the file identification is sent to the first index node, so that the first index node searches a second index node according to the file identification and local second search information, and returns the second index node to the request node when the search is successful, wherein the second index node is a node storing third search information of the shared file to be acquired;

and if the returned second index node is received, sending a file identification to the second index node, so that the second index node acquires the shared file to be acquired according to the file identification and the local third retrieval information, and returns the shared file to be acquired to the request node so as to return to the request client through the request node.

In an alternative embodiment, the method further comprises:

if the returned second index node is not received, randomly selecting a preset number of candidate nodes from the plurality of nodes;

Broadcasting the file identification to the candidate node so that the candidate node can search the target client which stores the shared file to be acquired according to the third search information stored locally, acquire the shared file to be acquired when the search is successful, and return the shared file to be acquired to the request node so as to return to the request client through the request node.

In an optional embodiment, the first inode retrieves a second inode according to the file identifier and the local second retrieval information by:

if the first index node fails to read the local second retrieval information, judging that the retrieval fails;

if the first index node successfully reads the local second retrieval information, the first index node determines the second index node according to the file identification and the local second retrieval information.

In an alternative embodiment, the second retrieval information of the first inode is updated by:

when any node receives a first file synchronization request sent by a client, determining a synchronization node to which a partition to which a first file to be synchronized belongs according to a first file identifier to be synchronized carried by the first file synchronization request and local first retrieval information;

And if the synchronous node is the first index node, sending the first file to be synchronized and the node receiving the first file synchronous request to the first index node so that the first index node updates the second retrieval information of the first index node according to the first file to be synchronized and the node receiving the first file synchronous request.

In an alternative embodiment, the third search information of the candidate node is updated by:

when any client sends a second file synchronization request to the candidate node, updating third retrieval information of the candidate node according to a second file identification to be synchronized carried by the second file synchronization request and the corresponding client.

In an optional embodiment, the second inode obtains the manner of obtaining the shared file to be obtained according to the file identifier and the local third search information:

determining a target client for storing the shared file to be acquired according to the file identification and the local third retrieval information;

sending the file identification to the target client;

the target client obtains the sharing priority of the shared file to be obtained according to the file identification, and sends the shared file to be obtained to the second index node according to the sharing priority, wherein the sharing priority is set according to the heat of the shared file to be obtained, the heat of the shared file to be obtained is obtained from a statistical node in the plurality of nodes, and the second retrieval information stored by the statistical node comprises a node storing third retrieval information of the shared file to be obtained.

In an optional embodiment, the statistics node counts the number of times of downloading the shared file to be obtained, and the heat calculation mode of the shared file to be obtained is as follows:

the statistical node counts the number of the clients storing the shared file to be acquired according to the local second retrieval information;

and the statistical node calculates the heat of the shared file to be acquired according to the downloading times and the number.

In a second aspect, the present invention provides a file obtaining apparatus applied to a requesting node among a plurality of nodes in a peer-to-peer P2P network, at least one of the nodes being communicatively connected to at least one client, at least one of the clients storing a shared file sharable through the P2P network, all of the shared files stored by the clients being divided into at least one partition in advance, each of the nodes storing first search information for searching a node to which the partition belongs, second search information for searching a node storing third search information of the shared file in the partition of the node, and third search information for searching a client to which the shared file stored by the node belongs, the requesting node being communicatively connected to the requesting client, the apparatus comprising:

The receiving module is used for receiving a file acquisition request sent by the request client, wherein the file acquisition request comprises a file identifier of a shared file to be acquired;

the determining module is used for determining a first index node to which the partition to which the shared file to be acquired belongs according to the file identifier and the local first retrieval information;

the sending module is used for sending the file identifier to the first index node so that the first index node searches a second index node according to the file identifier and local second search information, and returns the second index node to the request node when the search is successful, wherein the second index node is a node storing third search information of the shared file to be acquired;

and the return module is used for sending a file identifier to the second index node if the returned second index node is received, so that the second index node acquires the shared file to be acquired according to the file identifier and the local third retrieval information and returns the shared file to be acquired to the request node so as to return to the request client through the request node.

In a third aspect, the present invention provides a node device comprising a processor and a memory, the memory being configured to store a program, the processor being configured to implement the file acquisition method according to any one of the preceding embodiments when the program is executed.

In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a file acquisition method according to any of the preceding embodiments.

Compared with the prior art, the embodiment of the invention provides a file acquisition method, a device, node equipment and a computer readable storage medium, wherein nodes of a P2P network are in communication connection with at least one client, at least one client stores shared files which can be shared through the P2P network, all the shared files are divided into at least one partition in advance, each node stores first retrieval information, second retrieval information and third retrieval information, the first retrieval information is used for retrieving nodes belonging to the partition, the second retrieval information is used for retrieving nodes of the third retrieval information of the shared files stored in the partition of the node, the third retrieval information is used for retrieving the clients belonging to the shared files stored in the node, when the clients are required to request the shared files to be acquired, a file acquisition request is sent to the requesting node, the requesting node determines a first index node to which the partition belonging to the shared files to be acquired according to the file identifier in the file acquisition request and the local first retrieval information, the first index node is sent to the first index node, the first index node retrieves the second index node according to the file identifier and the local second retrieval information, the second index node is retrieved to the second index node, the second index node is used for retrieving the second index node, the second index node is sent to the second index node is distributed to the second index node when the second index node is required to the shared files to be acquired by the requesting the shared files to be acquired, and the shared files are fully distributed in the shared by the second node, and the shared nodes can be fully distributed on the P node, and the shared nodes can be distributed by the nodes according to the second index node and the shared node can be fully can be acquired according to the shared by the shared information.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an exemplary diagram of an application scenario provided in an embodiment of the present invention.

Fig. 2 is a block diagram of a node device according to an embodiment of the present invention.

Fig. 3 is an exemplary diagram of updating the first search information according to the embodiment of the present invention.

Fig. 4 is an exemplary diagram of updating third search information according to an embodiment of the present invention.

Fig. 5 is an exemplary diagram of updating the second search information according to the embodiment of the present invention.

Fig. 6 is a flowchart illustrating a file obtaining method according to an embodiment of the present invention.

Fig. 7 is a diagram illustrating a process of searching according to the first search information, the second search information, and the third search information according to an embodiment of the present invention.

Fig. 8 is a diagram illustrating a process of searching in a broadcasting manner according to an embodiment of the present invention.

Fig. 9 is a block diagram of a file acquisition apparatus according to the present embodiment.

Icon: 10-node; 20-client; 30-node device; 31-a processor; 32-a memory; 33-bus; 100-a file acquisition device; 110-a receiving module; 120-determining a module; 130-a transmitting module; 140-return module.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.

In the description of the present invention, it should be noted that, if the terms "upper", "lower", "inner", "outer", and the like indicate an azimuth or a positional relationship based on the azimuth or the positional relationship shown in the drawings, or the azimuth or the positional relationship in which the inventive product is conventionally put in use, it is merely for convenience of describing the present invention and simplifying the description, and it is not indicated or implied that the apparatus or element referred to must have a specific azimuth, be configured and operated in a specific azimuth, and thus it should not be construed as limiting the present invention.

Furthermore, the terms "first," "second," and the like, if any, are used merely for distinguishing between descriptions and not for indicating or implying a relative importance.

It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.

Related schemes for quickly discovering clients with shared files in a P2P network are generally classified into two types, centered and non-centered. The central scheme is simple, but has the risk of single-point failure, the capacity of the central server is limited, users and files of the P2P network cannot be managed and scheduled, and the scenes of massive files of massive users cannot be met; the centerless scheme is complex, the fault tolerance is high, capacity limitation can be broken through, but the implementation scheme of the related algorithm such as the distributed hash table has defects in the aspects of stability, time delay and the like, and is difficult to meet commercial requirements.

In view of this, embodiments of the present invention provide a method, an apparatus, a node device, and a computer readable storage medium for file acquisition to improve the efficiency of acquiring a shared file in a P2P network, which will be described in detail below.

Referring to fig. 1, fig. 1 is an exemplary diagram of an application scenario provided by an embodiment of the present invention, in fig. 1, a P2P network includes a plurality of nodes 10, at least one node 10 is communicatively connected to at least one client 20, at least one client 20 stores a shared file that can be shared through the P2P network, and the shared file stored by one client 20 can be shared to other clients 20 through the P2P network. The shared file stored by all clients 20 in the P2P network is divided into at least one partition, in order to reduce the amount of search data and ensure the search efficiency, the number of the partitions may be greater than the number of nodes, each node is responsible for managing at least one partition, each node stores first search information, second search information and third search information, each shared file has a file identifier, the partition to which the shared file belongs can be obtained according to the file identifier, the node can search the node to which the partition belongs through the first search information, each node stores second search information, a node for searching the third search information of the shared file stored in the partition of the node is stored, the third search information is used for searching the client to which the shared file stored in the node belongs, for example, a client a and a node 1 are in communication connection, the client a sends a sharing request for obtaining the shared file x according to the shared file x, the node 1 obtains the shared file x, the partition 1 is managed by the node 2, the node 1 sends the file identifier of the shared file x to the node 2, the node 2 has the second search information to the node 2, the node 2 has the third search information to the node 4 and the node 4 is returned to the node 1 according to the shared file stored in the partition, and the node 4 is obtained from the node 3 and the node 4 is returned to the node 4 to the node 1 according to the shared file stored in the node 1.

The node 10 may be a server or a server group or a server cluster formed by a plurality of servers, and the node 10 may be a virtual server or an entity server.

The client 20 includes, but is not limited to, a mobile terminal or wireless terminal such as a cell phone, tablet computer, wearable device, notebook computer, etc.

On the basis of fig. 1, the embodiment of the present invention provides a block example diagram of a node device 30, where the node device 30 may be any node 10 in fig. 1 that is communicatively connected to a client 20 that needs to request a shared file, and is also referred to as a requesting node in the following embodiments, and it is understood that any node 10 in fig. 1 may be a requesting node. Referring to fig. 2, fig. 2 is a block diagram of a node device 30 according to an embodiment of the present invention, where the node device 30 includes a processor 31, a memory 32, and a bus 33, and the processor 31, the memory 32, and the bus 33 are connected.

The processor 31 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the file acquiring method of the present embodiment, or the steps of the file acquiring method of the present embodiment, may be completed by an integrated logic circuit of hardware in the processor 31 or an instruction in the form of software. The processor 31 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a Digital Signal Processor (DSP), application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.

The memory 32 is used to store programs such as the file acquisition device in the present embodiment. The file acquisition means comprise at least one software functional module which may be stored in the memory 32 in the form of software or firmware (firmware) or cured in an Operating System (OS) of the node device 30. The processor 31 executes a program to implement the file acquisition device method disclosed in the above embodiment after receiving an execution instruction.

In order to more clearly explain the file acquisition method provided in the present embodiment, the present embodiment first describes the updating process of the first search information, the second search information, and the third search information, and then describes the implementation process of the file acquisition method based on the first search information, the second search information, and the third search information.

For a scene of massive users and massive files, how to find a group of other users with the file identification through the file identification is a very complex matter, and as users increase and files increase, the P2P network can face the problem of capacity expansion, and the maintenance and capacity expansion processing of index information for indexing the users with the specified files also become more complex. On the other hand, the data size of the index information is very large, for example, for an application scene of 1 million users and 10 million files, the data size of the index data even reaches the TB level, and the index data can be frequently changed and queried, so that the maintenance of the index data is quite complex, thereby greatly influencing the file acquisition efficiency.

Therefore, the embodiment adopts the three-level indexing mechanism of the first retrieval information, the second retrieval information and the third retrieval information, which is used for improving the file acquisition efficiency, and meanwhile, the first retrieval information, the second retrieval information and the third retrieval information are distributed on each node in the P2P network in a scattered way, so that the maintenance and the capacity expansion processing of index data are simplified.

In this embodiment, the first search information is used to search the nodes to which the partition belongs, each node in the P2P network stores the first search information, and the first search information may simply be the correspondence between the partition and the node, where one node corresponds to at least one partition. As an implementation manner, the server independent of the P2P network may be used to manage the first search information, that is, the server determines the correspondence between the partition and the node, and synchronizes the correspondence to each node in the P2P network, and when a new node is added to the P2P network, or an existing node is deleted from the P2P network, or the node goes online or offline, the server updates the first search information, and timely synchronizes the updated first search information to each node in the P2P network. For example, the P2P network has 10 nodes, and when a new node is added, the server updates the first search information so that each node corresponds to at least one partition, and synchronizes the updated first search information to each node, thereby distributing the second search information of the node for searching the third search information of the shared file in the partition storing the node approximately uniformly to all the nodes. Referring to fig. 3, fig. 3 is an exemplary diagram of updating first search information provided by the embodiment of the present invention, where the original 3 nodes and 4 partitions of the P2P network, the first search information determined by the server is: node 1-partition 1 and partition 2; node 2-partition 3; node 3-partition 4, now, node 4 adds P2P network, the server updates the first search information, and the updated first search information is: node 1-partitioning; node 2-partition 3; node 3-partition 4, node 4-partition 2.

It should be noted that fig. 3 is only an example of updating the first search information, and in an actual scenario, the server that manages the first search information may update the correspondence between the nodes and the partitions according to a preset rule, for example, the node 1 and the partition 2 may be corresponding, and the node 4 and the partition 1 may be corresponding, depending on the specific preset rule.

It should be noted that fig. 3 is only for illustrating synchronization of the first search information between the server and the nodes in the P2P network, and the client 20 in the P2P network and the communication connection relationship between the nodes are not shown in fig. 3, but these are not represented in the actual application scenario.

After the correspondence between the nodes and the partitions is determined, each node needs to store second search information of the shared file in the partition corresponding to the node, where the second search information is used for searching nodes storing third search information of the shared file in the partition of the node, that is, the second search information characterizes the correspondence between the shared file in the partition of the node and the node storing the third search information of the shared file.

In view of the readability of the description, the present embodiment first describes third search information for searching the client to which the shared file stored in the node belongs, and since the user typically performs file sharing by logging in the client, the user who owns the shared file can be regarded as the client to which the user logs in, and in this case, the user and the client can be equivalent. The client synchronizes the locally stored shared file to a node in communication connection with the locally stored shared file, the node associates the locally stored shared file with the client, and at this time, the corresponding relationship between the locally stored shared file and the locally stored shared file are third search information. In addition, when a plurality of users log in the same client, the client can synchronize the shared file and the user identifier to the node, and at this time, the corresponding relationship between the shared file and the client and the user identifier is the third retrieval information. When a local stored shared file of a client changes, the client needs to inform the node of the changed shared file, and the node updates the local third search information, please refer to fig. 4, fig. 4 is an exemplary diagram of updating the third search information provided by the embodiment of the present invention, the client a is communicatively connected to the node 1, and the client a originally stores the shared files x, y, z, where the third search information of the node 1 is: a- (x, y, z), then y is deleted, and the client a informs that the shared file stored by the node 1 itself is changed to x, z, and at this time, the node 1 updates the third search information, and the updated third search information is a- (x, z).

The node that updated the third search information needs to synchronize information with the node that stores the third search information to update the second search information in the node that stores the third search information. Referring to fig. 5, fig. 5 is an exemplary diagram of updating the second search information provided by the embodiment of the present invention, in fig. 5, the third search information of the node 1 is a client a-shared file x, the node 1 determines, according to the file identifier of the shared file x, a partition a to which the shared file belongs, determines, according to the first search information local to the node 1, that the node to which the partition a belongs is node 2, and synchronizes information with the node 2 by the node 1, so that the node 2 updates the local second search information: (a, x) -node 1.

When the client B wants to acquire the shared file x, the client B sends an acquisition request to a node 3 in communication connection with the client B, the node 3 determines a partition a to which the shared file belongs according to the file identification of the shared file x, determines a node to which the partition a belongs as a node 2 according to the first search information of the local area of the node 3, the node 3 sends the file identification of the shared file x to the node 2, the node 2 determines a third search information for storing the shared file x as a node 1 according to the file identification of the shared file x and the second search information of the local area of the node 2, and returns the node 1 to the node 3, the node 3 sends a request to the node 1, and the node 1 acquires the shared file x from the client a and returns the shared file to the node 3 and returns the shared file to the client B through the node 3.

In addition to the updating of the first search information, the second search information, and the third search information, the online-offline of the node may affect the migration of a part of the second search information from one node to another node, for example, in fig. 3, the second search information related to the shared file in partition 2 of node 1 may be migrated to node 4.

Based on the first search information, the second search information, and the third search information, the present embodiment further provides a file obtaining method, please refer to fig. 6, fig. 6 is a flowchart illustrating a file obtaining method according to an embodiment of the present invention, the method includes the following steps:

step S101, a file acquisition request sent by a request client is received, where the file acquisition request includes a file identifier of a shared file to be acquired.

In this embodiment, the requesting client may be any client 20 in fig. 1 that needs to obtain the shared file, and the requesting node may be the node 10 in fig. 1 that is communicatively connected to the requesting client. The file identifier is also called a file ID, and is used for uniquely identifying a file entity, and the file identifier of the shared file to be acquired is used for uniquely identifying the shared file to be acquired.

Step S102, determining a first index node to which a partition to which the shared file belongs is attributed according to the file identification and the local first retrieval information.

In this embodiment, as an implementation manner, a hash value is calculated according to a file identifier and a preset hash function, a partition to which a shared file to be obtained belongs is determined according to the hash value, and a first index node to which the partition to which the shared file to be obtained belongs is determined according to the partition to which the shared file to be obtained belongs and first search information. As another implementation manner, the partition to which the shared file to be acquired belongs may also be determined according to the file identifier and a preset mapping function, where the preset mapping function characterizes a correspondence between the file identifier and the partition.

Step S103, the file identification is sent to the first index node, so that the first index node searches the second index node according to the file identification and the local second search information, and returns the second index node to the request node when the search is successful, wherein the second index node is a node storing the third search information of the shared file to be acquired.

In this embodiment, the search may be successful or may fail, for example, when the first index node is abnormal, the search may fail, when the first index node is normal but the second search information stored locally in the first index node is read abnormally, the search may fail, when the network between the request node and the first index node is abnormal, the search may fail, and the second index node may be obtained only if the search is successful.

Step S104, if the returned second index node is received, the file identification is sent to the second index node, so that the second index node obtains the shared file to be obtained according to the file identification and the local third retrieval information, and returns the shared file to be obtained to the request node, so that the shared file to be obtained is returned to the request client through the request node.

In this embodiment, the request node sends the file identifier to the second index node, and as an implementation manner, the second index node obtains the file to be obtained according to the received file identifier and the third search information stored locally and returns the file to the request node, so that the file to be obtained is returned to the request client through the request node; in another implementation manner, the second index node may first send a message about whether to be ready for receiving to the request node, and send the shared file to be obtained to the request node after the request node returns the ready acknowledgement message.

According to the method provided by the embodiment, the shared file is managed according to the partition, and the first search information, the second search information and the third search information are distributed on the plurality of nodes of the P2P network based on the partition, so that the processing capacity of all the nodes in the P2P network can be fully utilized when the shared file is searched, and the acquisition efficiency of the shared file is improved.

In an alternative embodiment, as an implementation of retrieving the second inode may be:

In order to more clearly illustrate the above searching process, please refer to fig. 7, fig. 7 is an exemplary diagram of a process of searching according to the first searching information, the second searching information and the third searching information provided by the embodiment of the present invention, in fig. 7, the requesting client sends an obtaining request to the node a, the node a determines, according to the file identifier in the obtaining request and the local first searching information, that the second searching information of the partition to which the file belongs is stored in the node x, the node a sends the file identifier to the node x, as shown by the dashed line with reference number 1 in fig. 7, the node x determines, according to the file identifier and the local second searching information, that the node storing the third searching information of the file is the node b and the node c, as shown by the dashed line with reference number 2 in fig. 7, the node a sends the file identifier to the node b and the node c, as shown by the dashed line with reference number 3 in fig. 7, the node b obtains the file according to the local third searching information and returns to the node a, as shown by the solid line with reference number 4 in fig. 7, the node a returns the file to the requesting client, the node c finds that the load is higher, as shown by the dashed line with reference number 1, and thus the node c cannot respond to the node a in response to the current overload request, as shown by the node a, as shown by the dashed line with reference numeral 5.

It should be noted that if the node b and the node c can both normally respond to the node a, the node a can also receive information from the node b and the node c at the same time according to the current load condition of the node b and the node c, or only need to receive information from one of the node b and the node c, thereby realizing a better flow control effect.

In an alternative embodiment, when the second index node fails to be retrieved or the second retrieval information in the second index node is temporarily unavailable in the migration process, in order to reduce the influence on the file acquisition as much as possible, the embodiment further provides an implementation manner of file acquisition in this scenario:

if the returned second index node is not received, randomly selecting a preset number of candidate nodes from a plurality of nodes;

broadcasting the file identification to the candidate nodes so that the candidate nodes can search the shared file to be acquired according to the third search information stored locally, acquire the shared file to be acquired when the search is successful, and return the shared file to be acquired to the request node so as to return to the request client through the request node.

In this embodiment, since the broadcast has a larger uncertainty, in order to reduce the influence of the broadcast on the network communication bandwidth as much as possible, a preset number of candidate nodes are randomly selected from the plurality of nodes, and the preset number may be preset according to the bearing capacity of the network communication bandwidth.

In order to more clearly illustrate the above-mentioned searching process, please refer to fig. 8, fig. 8 illustrates a process example of searching by broadcasting provided by the embodiment of the present invention, in which node a is a requesting node, a requesting client sends an acquisition request to node a, the request includes a file identifier, node a determines a node storing second search information according to the file identifier and the first search information stored locally by node a, and as a result finds that the second search information fails, node a determines a node needing broadcasting from the P2P network, namely node b-e, and sends the file identifier to node b-e in a broadcasting manner, as indicated by a dotted line with reference number 1 in fig. 8, node d finds a third search information without the file identifier locally, does not respond to node a, nodes b, c, e find a request capable of responding normally, as indicated by a dotted line with reference number 2 in fig. 8, node e finds a request capable of responding normally, as indicated by a dotted line with reference number 2, and thus node b can not respond normally, as indicated by node b, node b can receive the request 2, as indicated by a dotted line with reference number 3, and node b can not respond normally, and node b can receive the request 1 in response to node b, as indicated by a request 1 can not see if node b can respond normally, and node b can receive the request 1, and node b can respond normally, as indicated by node b can only has a request 1, and node b can receive the request 1 can be received in response as indicated by node b, and node b can be determined by node b can respond normally as indicated by node b has a request. Node a returns it to the requesting client.

As described above in this embodiment, the first search information, the second search information, and the third search information are updated partially along with the update of the files stored in the online and offline nodes or the clients in the P2P network, so this embodiment further provides an update manner of the second search information of the first index node:

if the synchronization node is the first index node, the first file to be synchronized and the node receiving the first file synchronization request are sent to the first index node, so that the first index node updates the second retrieval information of the first index node according to the first file to be synchronized and the node receiving the first file synchronization request.

The embodiment also provides an updating mode of the third retrieval information of the candidate node:

Since the updating manner of the second search information and the third search information of any node has been described above, the description thereof is omitted.

In this embodiment, in order to ensure that a shared file to be acquired with a vigorous demand can be shared to a required client in time through a P2P network, the embodiment further provides a manner that the second index node acquires the shared file to be acquired according to the file identifier and the local third search information:

firstly, determining a target client for storing a shared file to be acquired according to a file identifier and local third retrieval information;

and secondly, the target client acquires the sharing priority of the shared file to be acquired according to the file identification, and sends the shared file to be acquired to the second index node according to the sharing priority, wherein the sharing priority is set according to the heat of the shared file to be acquired, the heat of the shared file to be acquired is acquired from a statistical node in a plurality of nodes, and the second retrieval information stored by the statistical node comprises a node storing third retrieval information of the shared file to be acquired.

In this embodiment, the higher the heat, the higher the sharing priority, and in addition to scheduling the shared files according to the sharing priority, the files may be set to be downloaded, uploaded and left in the cache preferentially, so as to ensure that the files with high heat are shared efficiently preferentially.

In this embodiment, when the statistics node receives an acquisition request of a shared file to be acquired, the statistics node may count the number of times of downloading the shared file to be acquired, and as a statistical manner, may count the number of times of downloading the shared file to be acquired in a preset duration.

In an alternative embodiment, a method for calculating the heat of the shared file to be acquired is as follows:

firstly, counting the number of clients storing the shared file to be acquired by the counting node according to local second retrieval information;

and secondly, calculating the heat of the shared file to be acquired by the statistical node according to the downloading times and the number.

In this embodiment, as one implementation, the popularity of the shared file=the number of downloads of the shared file ² The number of clients storing the shared file.

In order to perform the above-described embodiments and the corresponding steps in each possible implementation, an implementation of the file obtaining apparatus 100 is given below, which is applied to the node 10 in fig. 1 for requesting a shared file and the node device 30 in fig. 2, respectively. Referring to fig. 9, fig. 9 is a block diagram of a file obtaining apparatus 100 according to an embodiment of the present invention, and it should be noted that the basic principle and the technical effects of the file obtaining apparatus 100 according to the present embodiment are the same as those of the corresponding embodiment, and the description of the embodiment is omitted.

The file obtaining apparatus 100 includes a receiving module 110, a determining module 120, a sending module 130, and a returning module 140.

A receiving module 110, configured to receive a file acquisition request sent by a request client, where the file acquisition request includes a file identifier of a shared file to be acquired;

the determining module 120 is configured to determine, according to the file identifier and the local first search information, a first inode to which a partition to which the shared file belongs is to be acquired;

the sending module 130 is configured to send the file identifier to a first inode, so that the first inode retrieves a second inode according to the file identifier and local second retrieval information, and returns the second inode to the request node when the retrieval is successful, where the second inode is a node storing third retrieval information of the shared file to be obtained;

and the return module 140 is configured to send a file identifier to the second inode if the returned second inode is received, so that the second inode obtains the shared file to be obtained according to the file identifier and the local third search information, and returns the shared file to be obtained to the request node, so as to return to the request client through the request node.

In an alternative embodiment, the sending module 130 is further configured to:

In an alternative embodiment, the first inode retrieves the second inode according to the file identifier and the local second retrieval information by:

In an optional embodiment, the second inode obtains the shared file to be obtained according to the file identifier and the local third search information by:

transmitting the file identification to a target client;

the target client obtains the sharing priority of the shared file to be obtained according to the file identification, and sends the shared file to be obtained to the second index node according to the sharing priority, wherein the sharing priority is set according to the heat of the shared file to be obtained, the heat of the shared file to be obtained is obtained from a statistical node in a plurality of nodes, and the second retrieval information stored by the statistical node comprises a node storing third retrieval information of the shared file to be obtained.

In an alternative embodiment, the statistics node counts the number of downloads of the shared file to be obtained, and the calculation mode of the heat of the shared file to be obtained is as follows:

the statistical node counts the number of clients storing the shared file to be acquired according to the local second retrieval information;

An embodiment of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a file acquisition method according to any one of the preceding embodiments.

In summary, the embodiments of the present invention provide a method, an apparatus, a node device, and a computer readable storage medium for obtaining a file, where a node of a P2P network is communicatively connected to at least one client, at least one client stores a shared file that may be shared by the P2P network, all the shared files are pre-divided into at least one partition, each node stores first search information, second search information, and third search information, the first search information is used for searching a node to which the partition belongs, the second search information is used for searching a node storing the third search information of the shared file in the partition of the node, the third search information is used for searching a client to which the shared file stored in the node belongs, when a request client needs to request for obtaining the shared file, the request node determines a first index node to which the partition to which the shared file to be obtained belongs according to a file identifier in the request for obtaining the file and the local first search information, and the first index node searches the second index node according to the file identifier and the local second search information, and returns the first index node to the second index node to the request for obtaining the shared file, and the first index node to the request is implemented as compared with the prior art, and the second index node is returned to the request for obtaining the shared file from the second index node to the client: (1) The shared file is managed according to the partition, and the first search information, the second search information and the third search information are distributed on a plurality of nodes of the P2P network based on the partition, so that the processing capacity of all the nodes in the P2P network can be fully utilized when the shared file is searched, and the acquisition efficiency of the shared file is improved; (2) When the node is on-line or off-line or the second retrieval information is unavailable, acquiring the file through a broadcasting mechanism, so that the availability of acquiring the shared file through the P2P network is increased; (3) When a broadcasting mechanism is adopted, the number of broadcasting nodes is determined, and the influence of the broadcasting mechanism on the P2P network bandwidth is reduced as much as possible; (4) The flow control mechanism is utilized when the shared file is acquired, so that unnecessary bandwidth occupation is avoided; (5) And determining the priority in the downloading, uploading and temporary storage buffer of the shared file by using the shared file heat, and ensuring that the shared file with high heat is shared by the P2P network.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present invention should be included in the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1. A file acquisition method, applied to a requesting node among a plurality of nodes in a peer-to-peer P2P network, at least one of the nodes being communicatively connected to at least one client, at least one client storing a shared file sharable through the P2P network, all the shared files stored by the client being divided into at least one partition in advance, each of the nodes storing first search information for searching a node to which the partition belongs, second search information for searching a node storing third search information for searching a shared file in the partition of the node, and third search information for searching a client to which the shared file stored by the node belongs, the requesting node being communicatively connected to the requesting client, the method comprising:

2. The file acquisition method according to claim 1, wherein the method further comprises:

broadcasting the file identification to the candidate nodes so that the candidate nodes can search the shared file to be obtained according to the third search information stored locally, and when the search is successful, obtaining the shared file to be obtained and returning the shared file to be obtained to the request node so as to return to the request client through the request node.

3. The method of claim 1, wherein the first inode retrieves the second inode according to the file identifier and the local second retrieval information by:

4. The file acquisition method according to claim 1, wherein the second retrieval information of the first inode is updated by:

5. The file acquisition method according to claim 2, wherein the third retrieval information of the candidate node is updated by:

6. The method of claim 1, wherein the second inode obtains the shared file to be obtained according to the file identifier and the local third search information by:

sending the file identification to the target client;

7. The method of claim 6, wherein the statistics node counts the number of downloads of the shared file to be obtained, and the heat calculation method of the shared file to be obtained is as follows:

8. A file acquisition device, applied to a requesting node among a plurality of nodes in a peer-to-peer P2P network, at least one of the nodes being communicatively connected to at least one client, at least one client storing a shared file sharable through the P2P network, all the shared files stored by the client being divided into at least one partition in advance, each of the nodes storing first search information for searching a node to which the partition belongs, second search information for searching a node storing third search information for sharing files in the partition of the node, and third search information for searching a client to which the shared file stored by the node belongs, the requesting node being communicatively connected to the requesting client, the device comprising:

9. A node device comprising a processor and a memory, the memory for storing a program, the processor being configured to implement the file acquisition method of any one of claims 1-7 when the program is executed.

10. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements a file acquisition method as claimed in any one of claims 1-7.