CN116599973A - Data transmission method and device based on BT service acceleration component - Google Patents

Data transmission method and device based on BT service acceleration component Download PDF

Info

Publication number
CN116599973A
CN116599973A CN202310841537.XA CN202310841537A CN116599973A CN 116599973 A CN116599973 A CN 116599973A CN 202310841537 A CN202310841537 A CN 202310841537A CN 116599973 A CN116599973 A CN 116599973A
Authority
CN
China
Prior art keywords
system memory
data
client
specified
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310841537.XA
Other languages
Chinese (zh)
Other versions
CN116599973B (en
Inventor
朱勇
顾勤
齐俊星
张作宸
卢伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Zhuolang Kunlun Cloud Software Technology Co ltd
Original Assignee
Tianjin Zhuolang Kunlun Cloud Software Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Zhuolang Kunlun Cloud Software Technology Co ltd filed Critical Tianjin Zhuolang Kunlun Cloud Software Technology Co ltd
Priority to CN202310841537.XA priority Critical patent/CN116599973B/en
Publication of CN116599973A publication Critical patent/CN116599973A/en
Application granted granted Critical
Publication of CN116599973B publication Critical patent/CN116599973B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
    • H04L67/1078—Resource delivery mechanisms
    • H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
    • H04L67/1078—Resource delivery mechanisms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a data transmission method and device based on a BT service acceleration component, which relates to the technical field of data transmission, wherein the method deploys the BT service acceleration component and BT service simultaneously in each client in a P2P peer-to-peer network in advance, and each client sets a corresponding system memory, and after the BT service acceleration component of the current client receives a downloading request of a specified data sheet sent by a target client, if the specified data sheet exists in the system memory, the BT service acceleration component of the current client is directly read from the system memory; otherwise, reading from the system disk, and caching the appointed data sheet to the system memory. Compared with the method for reading the data sheet from the system disk in the prior art, the method for preferentially acquiring the data sheet from the system memory can effectively shorten the data reading time, thereby relieving the technical problem that the data transmission speed between the peers is lower when a large number of clients use the BT service to transmit the same large-capacity file.

Description

Data transmission method and device based on BT service acceleration component
Technical Field
The invention relates to the technical field of data transmission, in particular to a data transmission method and device based on a BT service acceleration component.
Background
BT (Bit Torrent) is an implementation of P2P, and P2P (Peer-to-Peer) is also called Peer-to-Peer network, which is a distributed application architecture that distributes tasks and workload among peers (peers), and is a network form formed by Peer-to-Peer computing model at application layer. In the P2P network environment, a plurality of computers connected to each other are in a peer-to-peer position, each computer has the same function, and has no master-slave division, each node not only serves as a server to provide services for other nodes, but also serves as a client to enjoy the services provided by other nodes.
However, if a client acting as a server (hereinafter referred to as a server for short) receives a large number of data download requests from the client in succession within a period of time, the server needs to read data from a system disk each time when responding to each data download request, and since the disk data reading speed is slow, in the prior art, there is a problem that the data transmission speed between peers is slow when transmitting the same large-capacity file.
Disclosure of Invention
The invention aims to provide a data transmission method and device based on a BT service acceleration component, so as to solve the technical problem that the data transmission speed between peers is low when a large number of clients use BT services to transmit the same large-capacity file.
In a first aspect, the present invention provides a data transmission method based on a BT service acceleration component, applied to the BT service acceleration component, where the BT service acceleration component and the BT service are deployed at the same time on each client in a P2P peer-to-peer network, and each client sets a corresponding system memory, where the method includes: receiving a downloading request for a specified data sheet sent by a target client in the P2P peer-to-peer network; wherein the target client represents any client in the P2P peer-to-peer network having a communication link with a current client; the downloading request carries the hash value of the appointed data sheet; judging whether the specified data sheet exists in the system memory of the current client based on the hash value of the specified data sheet; reading the specified data sheet from the system memory under the condition that the existence is determined; reading the appointed data sheet from the system disk of the current client under the condition that the appointed data sheet does not exist, and caching the appointed data sheet into the system memory; and sending the specified data sheet to the target client.
In an alternative embodiment, before determining whether the specified piece of data exists in the system memory of the current client based on the hash value of the specified piece of data, the method further includes: obtaining a hash value of each cache data slice in the system memory; converting the hash value of each cache data slice into binary number groups with preset length to obtain an array corresponding to each cache data slice; constructing a maintenance array corresponding to the system memory based on the arrays of all the cache data slices; wherein the maintenance array is a binary array with a preset length; if all the elements of the array corresponding to the cache data sheet on the target storage position are 0, the elements of the maintenance array on the target storage position are 0; if there is at least one array element in the target storage location is 1, the element of the maintaining array in the target storage location is 1.
In an alternative embodiment, determining whether the specified data slice exists in the system memory of the current client based on the hash value of the specified data slice includes: converting the hash value of the appointed data sheet into a binary number group with a preset length to obtain an appointed array; comparing the appointed array with a maintenance array corresponding to the system memory; determining that the specified data slice does not exist in the system memory under the condition that the elements on the first storage position of the maintenance array are not all 1; wherein, the first storage location represents a storage location with an element value of 1 in the specified array; retrieving a hash value of the specified chunk in the system memory if the element in the first storage location of the sustain array is all 1's; determining that the specified data slice does not exist in the system memory under the condition that the hash value of the specified data slice is not retrieved; and under the condition that the hash value of the specified data slice is retrieved, determining that the specified data slice exists in the system memory.
In an alternative embodiment, the system memory uses a b+ tree data structure to manage hash values for all buffered data slices.
In an alternative embodiment, before caching the specified data slice in the system memory, the method further includes: judging whether the residual storage space of the system memory reaches a preset threshold value or not; if yes, acquiring the service time of each cache data sheet in the system memory; wherein the usage time represents the time of last request for downloading; the buffered data slices with the earliest usage time are discarded.
In an alternative embodiment, in a case where all clients in the P2P peer-to-peer network download the specified seed file, the method further includes: acquiring the time of downloading the last data piece of the appointed seed file by each client to obtain a plurality of downloading ending time; the specified seed file is divided into a preset number of data pieces, and each data piece has a unique hash value; and clearing the cache data sheet in the system memory of each client based on the preset time length of the delay of the downloading end time corresponding to each client.
In a second aspect, the present invention provides a data transmission device based on a BT service acceleration component, applied to the BT service acceleration component, where the BT service acceleration component and the BT service are deployed at the same time on each client in a P2P peer-to-peer network, and each client sets a corresponding system memory, where the device includes: the receiving module is used for receiving a downloading request of the specified data sheet sent by the target client in the P2P peer-to-peer network; wherein the target client represents any client in the P2P peer-to-peer network having a communication link with a current client; the downloading request carries the hash value of the appointed data sheet; the first judging module is used for judging whether the specified data sheet exists in the system memory of the current client based on the hash value of the specified data sheet; the first reading module is used for reading the specified data sheet from the system memory under the condition that the existence is determined; the second reading module is used for reading the appointed data sheet from the system disk of the current client and caching the appointed data sheet into the system memory under the condition that the system disk does not exist; and the sending module is used for sending the appointed data sheet to the target client.
In an alternative embodiment, the apparatus further comprises: the first acquisition module is used for acquiring the hash value of each cache data slice in the system memory; the conversion module is used for converting the hash value of each cache data sheet into binary number groups with preset length to obtain an array corresponding to each cache data sheet; the construction module is used for constructing a maintenance array corresponding to the system memory based on the arrays of all the cache data slices; wherein the maintenance array is a binary array with a preset length; if all the elements of the array corresponding to the cache data sheet on the target storage position are 0, the elements of the maintenance array on the target storage position are 0; if there is at least one array element in the target storage location is 1, the element of the maintaining array in the target storage location is 1.
In a third aspect, the present invention provides an electronic device, including a memory, a processor, the memory storing a computer program executable on the processor, the processor implementing the steps of the BT service acceleration component-based data transmission method according to any of the preceding embodiments when the computer program is executed.
In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement a BT service acceleration component-based data transmission method according to any of the preceding embodiments.
The invention provides a data transmission method based on a BT service acceleration component, which comprises the steps of deploying the BT service acceleration component and BT service at the same time in each client in a P2P peer-to-peer network in advance, setting corresponding system memory for each client, and directly reading the appointed data sheet from the system memory if the appointed data sheet exists in the system memory after the BT service acceleration component of the current client receives a downloading request for the appointed data sheet sent by a target client; if the specified data sheet does not exist in the system memory, the specified data sheet is read from the system disk, and the specified data sheet is cached to the system memory. Compared with the method for reading the data sheet from the system disk in the prior art, the method for preferentially acquiring the data sheet from the system memory can effectively shorten the data reading time, thereby relieving the technical problem that the data transmission speed between the peers is lower when a large number of clients use the BT service to transmit the same large-capacity file.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a data transmission method based on a BT service acceleration component according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a P2P peer-to-peer network when a current client is acting as a server;
fig. 3 is a system architecture diagram of a client for deploying BT services and BT service acceleration components simultaneously, provided by an embodiment of the invention;
fig. 4 is a functional block diagram of a data transmission device based on a BT service acceleration component according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Some embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Example 1
The embodiment of the invention provides a data transmission method based on a BT service acceleration component, which is applied to the BT service acceleration component, wherein the BT service acceleration component and a BT service are deployed at each client in a P2P peer-to-peer network at the same time, and each client is provided with a corresponding system memory, and fig. 1 is a flow chart of the data transmission method based on the BT service acceleration component, as shown in fig. 1, and the method specifically comprises the following steps:
step S102, receiving a download request for a specified data slice sent by a target client in the P2P peer-to-peer network.
Wherein, the target client represents any client having a communication link with the current client in the P2P peer-to-peer network; the download request carries the hash value of the specified piece of data.
Specifically, when the BT initiates transmission, the BT is transmitted according to the BT seed file information. The main information in the BT seed file is the file fragment size (i.e., the total number of data fragments of the BT seed file that are split), and the hash value (unique identifier of data transmission) of each data fragment is transmitted according to the data fragments when the BT node transmits in the BT node. It is known that in a P2P network environment, each node acts as a server, providing services to other nodes, and also as a client, enjoying services provided by other nodes. Fig. 2 is therefore a schematic diagram of a P2P peer-to-peer network when a current client is acting as a server, for providing a download service of its existing data pieces to a client with which a communication link exists.
Fig. 3 is a system architecture diagram of a client for deploying BT service and BT service acceleration components simultaneously according to an embodiment of the present invention. As can be seen from fig. 3, the embodiment of the present invention not only deploys the BT service and the BT service acceleration component on each client at the same time, but also adds a system memory in each client. The data transmission between BT services is requested in accordance with the hash value of the data slice. Thus, as can be seen from fig. 3, when the BT service of the current client acquires a download request for a specified data piece, the BT service forwards the download request to the BT service acceleration component, and when the BT service acceleration component receives the download request, the BT service acceleration component may acquire a hash value of the specified data piece from the download request.
Step S104, judging whether the specified data sheet exists in the system memory of the current client based on the hash value of the specified data sheet.
Step S106, when the existence is determined, the appointed data sheet is read from the system memory.
Step S108, when the fact that the specified data piece does not exist is determined, the specified data piece is read from the system disk of the current client side, and the specified data piece is cached to the system memory.
Step S110, the designated data slice is sent to the target client.
After determining the specified data piece requested by the target client according to the hash value, the BT service acceleration component of the current client first judges whether the specified data piece exists in the system memory, and if the specified data piece corresponding to the hash value exists in the system memory, the BT service acceleration component of the current client directly reads the specified data piece from the system memory and then sends the specified data piece to the target client. However, if the specified data slice does not exist in the system memory, the specified data slice needs to be read from the system disk and then sent to the target client, and the hash value of the specified data slice is taken as an identifier and cached in the system memory.
In view of the fact that the data reading speed of the system memory is far greater than the data reading speed of the system disk, when a large number of clients request to download the data pieces from the current client, if the specified data pieces requested by the target client exist in the system memory of the current client, the method can effectively improve the data transmission speed between the clients compared with the method for reading the specified data pieces from the system disk in the prior art.
The invention provides a data transmission method based on a BT service acceleration component, which comprises the steps of deploying the BT service acceleration component and BT service at the same time in each client in a P2P peer-to-peer network in advance, setting corresponding system memory for each client, and directly reading the appointed data sheet from the system memory if the appointed data sheet exists in the system memory after the BT service acceleration component of the current client receives a downloading request for the appointed data sheet sent by a target client; if the specified data sheet does not exist in the system memory, the specified data sheet is read from the system disk, and the specified data sheet is cached to the system memory. Compared with the method for reading the data sheet from the system disk in the prior art, the method for preferentially acquiring the data sheet from the system memory can effectively shorten the data reading time, thereby solving the technical problem that the data transmission speed between the peers is lower when a large number of clients use the BT service to transmit the same large-capacity file (namely, the large-capacity data sheet).
In an alternative embodiment, before determining whether the specified data slice exists in the system memory of the current client based on the hash value of the specified data slice, the method of the present invention further includes the steps of:
Step S201, obtain the hash value of each cache data slice in the system memory.
Step S202, converting the hash value of each cache data slice into binary number groups with preset length to obtain an array corresponding to each cache data slice.
Step S203, a corresponding maintenance array of the system memory is constructed based on the arrays of all the cache data slices.
Wherein, the array is maintained as a binary array with preset length; if the elements of the array corresponding to all the cache data slices on the target storage position are 0, maintaining the elements of the array on the target storage position to be 0; if there is at least one array element at the target storage location that is 1, then the array element at the target storage location is maintained at 1. The target storage location represents any storage location of the array.
Specifically, in order to improve the timeliness of judging whether a specified data sheet exists in the system memory, the embodiment of the invention provides a method for constructing a maintenance array for the system memory. Firstly, obtaining hash values of all cache data slices in a system memory of a current client, wherein the cache data slices represent data slices requested to be downloaded by a target client in a specified historical time period, then converting the hash value of each cache data slice into binary number groups with preset length, and further obtaining an array corresponding to each cache data slice.
After obtaining the corresponding arrays of all the cache data slices, the embodiment of the invention stores the data with the element 1 in the array in the maintaining array according to the bit, and if a plurality of elements with the element 1 in the same storage position exist in the array, the element with the element 1 in the storage position of the maintaining data is still 1. If all the elements of the array in the same storage location are 0, then the elements of the data in that storage location are maintained as 0.
For ease of understanding, the following example assumes that the system memory includes 4 cache data slices and that the binary array is 64 bits long, the corresponding arrays are [1100 … 0 … ], [0110 … 0 … 00], [0011 … 0 … ] and [1010 … 0 … ] respectively, and the ellipses are all 0, and then based on the storage locations of element 1 in the 4 64-bit binary array, the constructed maintenance array is [1111 … 0 … ] as known from the storage locations of element 1.
In an optional embodiment, the step S104 is to determine whether the specified data slice exists in the system memory of the current client based on the hash value of the specified data slice, and specifically includes the following steps:
in step S1041, the hash value of the specified data slice is converted into a binary array with a preset length, so as to obtain the specified array.
In step S1042, the designated array is compared with the corresponding maintenance array of the system memory.
In step S1043, in the case where the elements in the first storage location of the array are not all 1, it is determined that the specified data slice does not exist in the system memory.
Wherein the first storage location represents a storage location in the specified array for which the element value is 1.
In step S1044, in the case where the elements in the first storage location of the array are maintained to be all 1' S, the hash value of the specified data slice is retrieved in the system memory.
In step S1045, if the hash value of the specified data piece is not retrieved, it is determined that the specified data piece does not exist in the system memory.
In step S1046, when the hash value of the specified data slice is retrieved, it is determined that the specified data slice exists in the system memory.
After the maintenance array corresponding to the system memory is pre-built, when judging whether the specified data piece exists in the system memory, firstly converting the hash value of the specified data piece into a binary array with a preset length by using the conversion method used in the step S202 to obtain the specified array, then comparing the specified array with the maintenance array, and if the comparison result is that the elements on the first storage position of the maintenance array are not all 1, then indicating that the specified data piece does not exist in the system memory.
By way of example, if the sustain array is known as [1111 … 0 … ] and the specified array is known as [0110 … 0 … 01], then it is known that the first memory location determined from the specified array is the 2 nd, 3 rd and 64 th memory bits, and by comparing the specified array with the sustain array, the elements in the first memory location of the sustain array are not all 1 (the elements in the 64 th memory bit are 0), then it is known that there is no specified data slice in the system memory according to the construction principle of the sustain array.
In the prior art, a method for judging whether specified data exists in a database is a training mode, and the method for training is applied to the method and the method are as follows: and sequentially comparing the hash value of the designated data slice with the hash values of all the cached data slices stored in the system memory, if the hash values which are consistent in comparison are found, feeding back the result with the designated data slice, and if the hash values which are consistent in comparison are compared to the last hash value which is not matched in comparison, feeding back the result without the designated data slice. Obviously, compared with the training method commonly used in the prior art, the method for comparing by using the maintenance array disclosed by the embodiment of the invention can effectively shorten the judgment time.
However, if the comparison reveals that the elements in the first storage location of the sustain array are all 1's, then further searching the hash value of the specified slice in the system memory is required to determine whether the specified slice exists in the system memory. This is because, according to the method for constructing the maintenance array, if the arrays corresponding to the 4 pieces of cache data in the system memory are [1100 … 0 … ], [0110 … 0 … 00], [0011 … 0 … ], [1010 … 0 … 00], the constructed maintenance array is [1111 … 0 … 00], and the designated array is [1110 … 0 … 00], the elements in the first storage location of the maintenance array are all 1, but the designated pieces of data are not present in the system memory in practice. In this case, it is therefore necessary to further retrieve the hash value of the specified piece of data in the system memory to determine whether the specified piece of data exists in the system memory.
The step of searching can be avoided by increasing the length of the binary array after conversion, namely, the positions of 1 in the array after conversion of different hash values are controlled to be different from each other, so that when the elements in the first storage position of the array are determined to be all 1, the existence of the designated data sheet in the system memory can be judged, but if the number of the data sheets cut by the seed file is more, the array length required by the method is correspondingly longer.
Thus, to balance array length with retrieval time, optionally, system memory uses a b+ tree data structure to manage hash values for all buffered slices. When determining that the hash value of the specified data slice needs to be retrieved in the system memory, the embodiment of the invention can quickly determine whether the specified data slice exists in the system memory according to the data query method of the b+ tree data structure.
In an alternative embodiment, the method of the present invention further comprises the steps of, prior to caching the specified data slice in the system memory:
step S301, determining whether the remaining storage space of the system memory reaches a preset threshold.
If yes, the following steps S302 to S303 are executed; if not, executing the step of caching the appointed data sheet into the system memory.
Step S302, the service time of each cache data slice in the system memory is obtained.
Wherein the usage time indicates the time of the last download request.
Step S303, discarding the cached data piece with the earliest use time.
In order to avoid the problem of reduced data transmission performance of a client caused by excessive occupation of a system memory by a cache data sheet, when the cache data sheet is added into the system memory, firstly judging whether the residual storage space of the system memory reaches a preset threshold value, and if the residual storage space of the system memory does not reach the preset threshold value, directly caching the appointed data sheet read from a system disk into the system memory; and if the preset threshold is determined to be reached, acquiring the service time of each cache data sheet in the system memory. In the embodiment of the invention, when each cache data sheet in the system memory is used, the service time of the cache data sheet needs to be refreshed. Assuming that the time when the other clients last sent the download request to the current client for the cached data piece a is T1, the service time of the cached data piece a is T1.
Assuming that total 4 pieces of cache data in the system memory are a, B, C and D respectively, the service time of the cache data piece a is T1, the service time of the cache data piece B is T2, the service time of the cache data piece C is T3, the service time of the cache data piece D is T4, and the time of T3 is the earliest according to the time sequencing, then the system memory needs to discard the cache data piece C with the earliest service time at this time, and then the designated data piece is cached in the system memory.
In an alternative embodiment, in case that all clients in the P2P peer-to-peer network download the specified seed file, the method of the present invention further comprises the steps of:
step S401, obtaining the time of downloading the last data slice of the specified seed file by each client, and obtaining a plurality of downloading end times.
The specified seed file is divided into a preset number of data pieces, and each data piece has a unique hash value.
Step S402, taking the preset time length of the downloading end time delay corresponding to each client as a reference, and clearing the cache data sheet in the system memory of each client.
When all clients in the P2P peer-to-peer network download the specified seed file, more or less cache data pieces are kept in the system memory of all clients, and the cache data pieces occupy too much memory to affect the performance of the clients, so that the system memory needs to be released timely, and in general, the downloading process of the clients for downloading all the cache data pieces belonging to the same seed file cannot exceed 1 hour, so that the system memory can be conveniently recovered by setting the expiration time of the cache data pieces in the system memory.
After the downloading of the specified seed file is completed, the system memory of each client needs to be cleared. In the embodiment of the invention, the time of the last data slice of the client-side downloading designated seed file is taken as the downloading ending time of the last data slice, according to the actual data transmission situation, the downloading ending time of different client-sides may be different, the downloading ending time of the client-side X1 is assumed to be T1, the downloading ending time of the client-side X2 is assumed to be T2, the downloading ending time of the client-side X3 is T3, and the preset duration is T, so that the client-side X1 will clear the cached data slice in the system memory at the time t1+T, the client-side X2 will clear the cached data slice in the system memory at the time t2+T, and the client-side X3 will clear the cached data slice in the system memory at the time t3+T.
In summary, the data transmission method based on the BT service acceleration component provided by the embodiment of the invention is applied to a scenario of simultaneously transmitting the same file to a large number of BT services of clients in the P2P network, and based on the policy that the method provides to obtain the specified data slice from the system memory preferentially, the single-node data transmission speed of the Bit Torrent single file can be effectively improved.
Example two
The embodiment of the invention also provides a data transmission device based on the BT service acceleration component, which is mainly used for executing the data transmission method based on the BT service acceleration component provided in the first embodiment, and the data transmission device based on the BT service acceleration component provided in the embodiment of the invention is specifically introduced below.
The data transmission device based on the BT service acceleration component provided in the embodiment of the present invention is further applied to the BT service acceleration component, where the BT service acceleration component and the BT service are deployed at each client in the P2P peer-to-peer network at the same time, and each client sets a corresponding system memory, and fig. 4 is a functional block diagram of the data transmission device based on the BT service acceleration component provided in the embodiment of the present invention, and as shown in fig. 4, the device mainly includes: the device comprises a receiving module 10, a first judging module 20, a first reading module 30, a second reading module 40 and a transmitting module 50, wherein:
a receiving module 10, configured to receive a download request for a specified data slice sent by a target client in a P2P peer-to-peer network; wherein, the target client represents any client having a communication link with the current client in the P2P peer-to-peer network; the download request carries the hash value of the specified piece of data.
The first determining module 20 is configured to determine whether the specified data slice exists in the system memory of the current client based on the hash value of the specified data slice.
The first reading module 30 is configured to read the specified data slice from the system memory if it is determined that the specified data slice exists.
And the second reading module 40 is configured to read the specified data piece from the system disk of the current client and cache the specified data piece to the system memory if it is determined that the specified data piece does not exist.
A sending module 50, configured to send the specified data slice to the target client.
The embodiment of the invention provides a data transmission device based on a BT service acceleration component, which comprises the steps that a BT service acceleration component and BT service are deployed at the same time in each client in a P2P peer-to-peer network in advance, each client is provided with a corresponding system memory, and after the BT service acceleration component of the current client receives a downloading request of a specified data sheet sent by a target client, if the specified data sheet exists in the system memory, the specified data sheet is directly read from the system memory; if the specified data sheet does not exist in the system memory, the specified data sheet is read from the system disk, and the specified data sheet is cached to the system memory. Compared with the method for reading the data sheet from the system disk in the prior art, the method for preferentially acquiring the data sheet from the system memory can effectively shorten the data reading time, thereby relieving the technical problem that the data transmission speed between the peers is lower when a large number of clients use the BT service to transmit the same large-capacity file.
Optionally, the apparatus further comprises:
the first acquisition module is used for acquiring the hash value of each cache data slice in the system memory.
The conversion module is used for converting the hash value of each cache data slice into binary number groups with preset length to obtain an array corresponding to each cache data slice.
And the construction module is used for constructing a maintenance array corresponding to the system memory based on the arrays of all the cache data slices.
Wherein, the array is maintained as a binary array with preset length; if the elements of the array corresponding to all the cache data slices on the target storage position are 0, maintaining the elements of the array on the target storage position to be 0; if there is at least one array element at the target storage location that is 1, then the array element at the target storage location is maintained at 1.
Optionally, the first judging module 20 is specifically configured to:
and converting the hash value of the appointed data sheet into a binary number group with a preset length to obtain the appointed array.
And comparing the appointed array with the maintenance array corresponding to the system memory.
Determining that the specified data slice does not exist in the system memory under the condition that the elements on the first storage position of the array are maintained to be not all 1; wherein the first storage location represents a storage location in the specified array for which the element value is 1.
In the case where the elements on the first storage location of the array are maintained as all 1's, the hash value of the specified chunk is retrieved in system memory.
In the event that the hash value of the specified piece of data is not retrieved, it is determined that the specified piece of data does not exist in the system memory.
In the case that the hash value of the specified piece of data is retrieved, it is determined that the specified piece of data exists in the system memory.
Optionally, the system memory uses a b+ tree data structure to manage the hash values of all buffered slices.
Optionally, the apparatus further comprises:
and the second judging module is used for judging whether the residual storage space of the system memory reaches a preset threshold value.
The second acquisition module is used for acquiring the service time of each cache data sheet in the system memory under the condition that the residual storage space of the system memory reaches a preset threshold value; wherein the usage time indicates the time of the last download request.
And the discarding module is used for discarding the cached data piece with the earliest use time.
Optionally, the apparatus further comprises:
the third acquisition module is used for acquiring the time of downloading the last data piece of the appointed seed file by each client to obtain a plurality of downloading ending time; the specified seed file is divided into a preset number of data pieces, and each data piece has a unique hash value.
And the clearing module is used for clearing the cache data sheet in the system memory of each client based on the preset time length of the downloading end time delay corresponding to each client.
Example III
Referring to fig. 5, an embodiment of the present invention provides an electronic device, including: a processor 60, a memory 61, a bus 62 and a communication interface 63, the processor 60, the communication interface 63 and the memory 61 being connected by the bus 62; the processor 60 is arranged to execute executable modules, such as computer programs, stored in the memory 61.
The memory 61 may include a high-speed random access memory (RAM, random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The communication connection between the system network element and at least one other network element is achieved via at least one communication interface 63 (which may be wired or wireless), and may use the internet, a wide area network, a local network, a metropolitan area network, etc.
Bus 62 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 5, but not only one bus or type of bus.
The memory 61 is configured to store a program, and the processor 60 executes the program after receiving an execution instruction, and the method executed by the apparatus for defining a process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 60 or implemented by the processor 60.
The processor 60 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware or instructions in software in the processor 60. The processor 60 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 61 and the processor 60 reads the information in the memory 61 and in combination with its hardware performs the steps of the method described above.
The embodiment of the invention provides a data transmission method and device based on a BT service acceleration component, which comprises a computer readable storage medium storing a non-volatile program code executable by a processor, wherein the program code comprises instructions for executing the method described in the previous method embodiment, and the specific implementation can be referred to the method embodiment and is not repeated herein.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
In the description of the present invention, it should be noted that, directions or positional relationships indicated by terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., are directions or positional relationships based on those shown in the drawings, or are directions or positional relationships conventionally put in use of the inventive product, are merely for convenience of describing the present invention and simplifying the description, and are not indicative or implying that the apparatus or element to be referred to must have a specific direction, be constructed and operated in a specific direction, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and should not be construed as indicating or implying relative importance.
Furthermore, the terms "horizontal," "vertical," "overhang," and the like do not denote a requirement that the component be absolutely horizontal or overhang, but rather may be slightly inclined. As "horizontal" merely means that its direction is more horizontal than "vertical", and does not mean that the structure must be perfectly horizontal, but may be slightly inclined.
In the description of the present invention, it should also be noted that, unless explicitly specified and limited otherwise, the terms "disposed," "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (10)

1. A BT service acceleration component-based data transmission method, applied to a BT service acceleration component, where the BT service acceleration component is deployed simultaneously with a BT service on each client in a P2P peer-to-peer network, and each client sets a corresponding system memory, the method includes:
Receiving a downloading request for a specified data sheet sent by a target client in the P2P peer-to-peer network; wherein the target client represents any client in the P2P peer-to-peer network having a communication link with a current client; the downloading request carries the hash value of the appointed data sheet;
judging whether the specified data sheet exists in the system memory of the current client based on the hash value of the specified data sheet;
reading the specified data sheet from the system memory under the condition that the existence is determined;
reading the appointed data sheet from the system disk of the current client under the condition that the appointed data sheet does not exist, and caching the appointed data sheet into the system memory;
and sending the specified data sheet to the target client.
2. The BT service acceleration component-based data transmission method of claim 1, wherein prior to determining whether the specified piece of data exists in the system memory of the current client based on the hash value of the specified piece of data, the method further comprises:
obtaining a hash value of each cache data slice in the system memory;
converting the hash value of each cache data slice into binary number groups with preset length to obtain an array corresponding to each cache data slice;
Constructing a maintenance array corresponding to the system memory based on the arrays of all the cache data slices;
wherein the maintenance array is a binary array with a preset length; if all the elements of the array corresponding to the cache data sheet on the target storage position are 0, the elements of the maintenance array on the target storage position are 0; if there is at least one array element in the target storage location is 1, the element of the maintaining array in the target storage location is 1.
3. The BT service acceleration component-based data transmission method of claim 2, wherein determining whether the specified piece of data exists in the system memory of the current client based on the hash value of the specified piece of data, comprises:
converting the hash value of the appointed data sheet into a binary number group with a preset length to obtain an appointed array;
comparing the appointed array with a maintenance array corresponding to the system memory;
determining that the specified data slice does not exist in the system memory under the condition that the elements on the first storage position of the maintenance array are not all 1; wherein, the first storage location represents a storage location with an element value of 1 in the specified array;
Retrieving a hash value of the specified chunk in the system memory if the element in the first storage location of the sustain array is all 1's;
determining that the specified data slice does not exist in the system memory under the condition that the hash value of the specified data slice is not retrieved;
and under the condition that the hash value of the specified data slice is retrieved, determining that the specified data slice exists in the system memory.
4. The BT service acceleration component based data transfer method of claim 3 wherein the system memory manages hash values of all buffered data slices using a b+ tree data structure.
5. The BT service acceleration component-based data transfer method of claim 1, wherein prior to caching the specified data slice into the system memory, the method further comprises:
judging whether the residual storage space of the system memory reaches a preset threshold value or not;
if yes, acquiring the service time of each cache data sheet in the system memory; wherein the usage time represents the time of last request for downloading;
the buffered data slices with the earliest usage time are discarded.
6. The BT service acceleration component-based data transfer method of claim 1, wherein in the event that all clients in the P2P peer-to-peer network download a specified seed file, the method further comprises:
acquiring the time of downloading the last data piece of the appointed seed file by each client to obtain a plurality of downloading ending time; the specified seed file is divided into a preset number of data pieces, and each data piece has a unique hash value;
and clearing the cache data sheet in the system memory of each client based on the preset time length of the delay of the downloading end time corresponding to each client.
7. A BT service acceleration component-based data transmission apparatus, applied to a BT service acceleration component, the BT service acceleration component being deployed simultaneously with a BT service at each client in a P2P peer-to-peer network, each of the clients setting a corresponding system memory, the apparatus comprising:
the receiving module is used for receiving a downloading request of the specified data sheet sent by the target client in the P2P peer-to-peer network; wherein the target client represents any client in the P2P peer-to-peer network having a communication link with a current client; the downloading request carries the hash value of the appointed data sheet;
The first judging module is used for judging whether the specified data sheet exists in the system memory of the current client based on the hash value of the specified data sheet;
the first reading module is used for reading the specified data sheet from the system memory under the condition that the existence is determined;
the second reading module is used for reading the appointed data sheet from the system disk of the current client and caching the appointed data sheet into the system memory under the condition that the system disk does not exist;
and the sending module is used for sending the appointed data sheet to the target client.
8. The BT service acceleration component-based data transfer apparatus of claim 7, further comprising:
the first acquisition module is used for acquiring the hash value of each cache data slice in the system memory;
the conversion module is used for converting the hash value of each cache data sheet into binary number groups with preset length to obtain an array corresponding to each cache data sheet;
the construction module is used for constructing a maintenance array corresponding to the system memory based on the arrays of all the cache data slices;
wherein the maintenance array is a binary array with a preset length; if all the elements of the array corresponding to the cache data sheet on the target storage position are 0, the elements of the maintenance array on the target storage position are 0; if there is at least one array element in the target storage location is 1, the element of the maintaining array in the target storage location is 1.
9. An electronic device comprising a memory, a processor, the memory having stored thereon a computer program executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the BT service acceleration component based data transfer method of any of claims 1 to 6.
10. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the BT service acceleration component-based data transmission method of any of claims 1 to 6.
CN202310841537.XA 2023-07-11 2023-07-11 Data transmission method and device based on BT service acceleration component Active CN116599973B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310841537.XA CN116599973B (en) 2023-07-11 2023-07-11 Data transmission method and device based on BT service acceleration component

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310841537.XA CN116599973B (en) 2023-07-11 2023-07-11 Data transmission method and device based on BT service acceleration component

Publications (2)

Publication Number Publication Date
CN116599973A true CN116599973A (en) 2023-08-15
CN116599973B CN116599973B (en) 2023-09-26

Family

ID=87606592

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310841537.XA Active CN116599973B (en) 2023-07-11 2023-07-11 Data transmission method and device based on BT service acceleration component

Country Status (1)

Country Link
CN (1) CN116599973B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075241A (en) * 2006-12-26 2007-11-21 腾讯科技（深圳）有限公司 Method and system for processing buffer
CN112950211A (en) * 2021-05-14 2021-06-11 腾讯科技（深圳）有限公司 Transaction duplication checking method, device, equipment and medium
CN114625762A (en) * 2020-11-27 2022-06-14 华为技术有限公司 Metadata acquisition method, network equipment and system
CN115033501A (en) * 2022-06-30 2022-09-09 苏州浪潮智能科技有限公司 System, method, equipment and medium for invalidation cache data
CN116049109A (en) * 2023-01-03 2023-05-02 重庆长安汽车股份有限公司 File verification method, system, equipment and medium based on filter

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075241A (en) * 2006-12-26 2007-11-21 腾讯科技（深圳）有限公司 Method and system for processing buffer
CN114625762A (en) * 2020-11-27 2022-06-14 华为技术有限公司 Metadata acquisition method, network equipment and system
CN112950211A (en) * 2021-05-14 2021-06-11 腾讯科技（深圳）有限公司 Transaction duplication checking method, device, equipment and medium
CN115033501A (en) * 2022-06-30 2022-09-09 苏州浪潮智能科技有限公司 System, method, equipment and medium for invalidation cache data
CN116049109A (en) * 2023-01-03 2023-05-02 重庆长安汽车股份有限公司 File verification method, system, equipment and medium based on filter

Also Published As

Publication number Publication date
CN116599973B (en) 2023-09-26

Similar Documents

Publication Publication Date Title
US9015269B2 (en) Methods and systems for notifying a server with cache information and for serving resources based on it
US8706884B2 (en) Method and system for generating and using an augmented bloom filter
US7558854B2 (en) Access relaying apparatus
JP5828760B2 (en) Method and system for cache optimization
US9471646B2 (en) Method and server device for exchanging information items with a plurality of client entities
US20140359066A1 (en) System, method and device for offline downloading resource and computer storage medium
US11146848B2 (en) Cache management in content delivery systems
WO2008025297A1 (en) A method for downloading files by adopting the p2p technique and a p2p downloading system
JP2008537619A5 (en)
CN108197160B (en) Picture loading method and device
US8086629B2 (en) Content delivery apparatus, content delivery method, and content delivery program
CN109873855B (en) Resource acquisition method and system based on block chain network
US20140156809A1 (en) Data downloading method, terminal, server, and system
CN111212301B (en) Video code rate matching method, storage medium and terminal equipment
CN109905479A (en) Document transmission method and device
CN116599973B (en) Data transmission method and device based on BT service acceleration component
CN113852665A (en) File uploading method and device, electronic equipment, storage medium and program product
CN111061682B (en) Data caching method, reading method, electronic equipment and storage medium
CN112491963B (en) Data transmission method, device, equipment and readable storage medium
JP6495777B2 (en) Transfer device, server device, and program for content distribution network
CN110474954B (en) Block chain-based file sharing method and device, computer equipment and readable storage medium
CN108762683B (en) Method and system for data transmission in distributed big data storage system
CN112839070A (en) Data processing method and device and node equipment in CDN (content delivery network)
CN110784775A (en) Video fragment caching method and device and video-on-demand system
CN112055044B (en) Data request method, server and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant