CN112612734B

CN112612734B - File transmission method, device, computer equipment and storage medium

Info

Publication number: CN112612734B
Application number: CN202011509142.2A
Authority: CN
Inventors: 田玉凯
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2020-12-18
Filing date: 2020-12-18
Publication date: 2023-09-26
Anticipated expiration: 2040-12-18
Also published as: WO2022126919A1; CN112612734A

Abstract

The application discloses a file transmission method, a file transmission device, computer equipment and a storage medium. The method comprises the following steps: judging whether the cache data stored in the server reaches a saturated state or not; if the cache data stored in the server reaches a saturated state, copying super block information of a file block group of the cache data into a memory of the server; the super block information stored in the memory of the server is remotely transmitted to the client through RDMA; acquiring a remote data reading request generated by a client based on super block information, wherein the remote data reading request comprises an address space of data to be read; and according to the remote data reading request, the data to be read is remotely transmitted to the client through the DMA. According to the scheme, only super block information of the cache data is copied to a server memory, so that the transmission pressure of the cache data is reduced; and the target data is remotely transmitted through the DMA, so that the processing of an operating system and a CPU is not involved, the transmission quantity of data copying is reduced, and the data transmission efficiency is improved.

Description

File transmission method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of data transmission technologies, and in particular, to a file transmission method, a file transmission device, a computer device, and a storage medium.

Background

In the existing buffer I/O mechanism, the DMA mode may directly read data from the disk to the page buffer, or directly write data from the page buffer back to the disk, but cannot directly transfer data between the application address space and the disk, so that multiple data copying operations need to be performed between the application address space and the page buffer during the data transfer, and the CPU (Central Processing Unit ) and memory overhead caused by these data copying operations are very large.

In order to improve the data reading efficiency, the Linux system level has been improved in many ways, for example: adding a memory sharing (mmap, method of memory mapping file); transferring file contents between a descriptor in_fd representing an input file and a descriptor out_fd representing an output file by using a Sendfile zero copy technology, directly transmitting the file descriptors to a Socket buffer through DMA, and copying data to the Socket buffer by a DMA engine; there are also Copy On Write (Copy-On-Write) techniques used when data must pass through user space.

In the above common zero-copy technologies, the CPU is involved in context switching and data copying, so that the CPU cannot be completely avoided from being involved in file transfer, which can cause occupation and consumption of CPU resources, and the zero-copy file mode can only be aimed at some specific file systems, and cannot meet the requirements of general file systems.

Disclosure of Invention

The embodiment of the application provides a file transmission method, a file transmission device, computer equipment and a storage medium, aiming at solving the problem of occupation of CUP resources during file transmission.

In a first aspect, an embodiment of the present application provides a file transfer method, including the following steps: judging whether the cache data stored in the server reaches a saturated state or not; if the cache data stored in the server reaches a saturated state, copying super block information of a file block group of the cache data into a server memory; remotely transmitting the superblock information stored in the server memory to a client through RDMA; acquiring a remote data reading request generated by the client based on the super block information, wherein the remote data reading request comprises an address space of data to be read; and according to the remote data reading request, the data to be read is remotely transmitted to a client through RDMA.

In a second aspect, an embodiment of the present application further provides a file transmission device, including: the buffer judgment unit is used for judging whether the buffer data stored in the server reaches a saturated state or not; the information copying unit is used for copying super block information of a file block group of the cache data to a server memory if the cache data stored in the server reaches a saturated state; the information transmission unit is used for remotely transmitting the super block information stored in the server memory to the client through RDMA; the request acquisition unit is used for acquiring a remote data reading request generated by the client based on the super block information, wherein the remote data reading request comprises an address space of data to be read; and the data transmission unit is used for remotely transmitting the data to be read to the client through DMA according to the remote data reading request.

In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the file transfer method when executing the computer program.

In a fourth aspect, embodiments of the present application also provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, implement the above-described file transfer method.

The embodiment of the application provides a file transmission method, a file transmission device, computer equipment and a storage medium. The method comprises the following steps: judging whether the cache data stored in the server reaches a saturated state or not; if the cache data stored in the server reaches a saturated state, copying super block information of a file block group of the cache data into a server memory; the super block information stored in the memory of the server is remotely transmitted to the client through RDMA; acquiring a remote data reading request generated by a client based on super block information, wherein the remote data reading request comprises an address space of data to be read; and according to the remote data reading request, the data to be read is remotely transmitted to the client through DMA. In the embodiment of the application, when the cache data is fully stored in the server, only superblock information of the cache data is copied to the memory of the server, the cache data transmission pressure is reduced, and the cache data is remotely transmitted to the client through RDMA (remote direct memory access), the client obtains the address space of the data to be read based on the superblock information analysis, and directly reads the target data from the memory of the server through DMA (direct memory access) remote transmission based on the address space, so that the processing of an operating system and a CPU (central processing unit) is not involved, the data transmission processing flow is simplified, the data copy transmission quantity is reduced, the data transmission efficiency is improved, and the universality of the scene of file remote transmission is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic view of an application scenario of a file transmission method according to an embodiment of the present application;

fig. 2 is a flow chart of a file transfer method according to an embodiment of the present application;

FIG. 3 is a schematic sub-flowchart of a file transfer method according to an embodiment of the present application;

FIG. 4 is a schematic sub-flowchart of a file transfer method according to an embodiment of the present application;

FIG. 5 is a schematic sub-flowchart of a file transfer method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a sub-flow of a file transfer method according to an embodiment of the present application;

FIG. 7 is a flowchart illustrating a file transfer method according to another embodiment of the present application;

FIG. 8 is a schematic block diagram of a file transfer device according to an embodiment of the present application;

FIG. 9 is a schematic block diagram of an information copy unit of a file transfer device according to an embodiment of the present application;

fig. 10 is a schematic block diagram of a data transmission unit of a file transmission device according to an embodiment of the present application;

FIG. 11 is a schematic block diagram of a data encryption module of a file transfer device according to an embodiment of the present application;

FIG. 12 is a schematic block diagram of a file transfer device according to another embodiment of the present application; and

fig. 13 is a schematic block diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

Referring to fig. 1 and fig. 2, fig. 1 is a schematic application scenario diagram of a file transmission method according to an embodiment of the present application. Fig. 2 is a schematic flowchart of a file transfer method according to an embodiment of the present application. The file transmission method is applied to the server, and the server and the client perform data interaction and realize file remote transmission. When the cache data stored by the server is saturated, the server copies super block information of the cache data into a server memory, and remotely transmits the super block information to the client through RDMA, the client analyzes the super block information to obtain an address space of data to be read, and obtains corresponding data from the server memory through RDMA remote transmission based on the address space. As shown in fig. 1, the client may include terminal devices such as a first terminal and a second terminal that may perform network data transmission.

Fig. 2 is a flow chart of a file transfer method according to an embodiment of the present application. As shown, the method includes the following steps S110-150.

S110, judging whether the cache data stored in the server reaches a saturated state or not.

In this embodiment, when the file system of the server is in normal use, the user process calls the sys_read () command to enter the kernel mode through the read () command, the process scheduler allocates a CPU (Central Processing Unit ), the CPU sends an IO (Input/Output) operation to the DMA, the DMA drives the hardware by requesting, the hardware caches the cache data, and the size of the cache data also determines the frequency of the interrupt. If the buffer data buffered by the server has reached a saturated state, the CPU interrupt is triggered, and further the DMA auxiliary data is triggered to the memory of the server.

Among them, DMA (Direct Memory Access ) is an important function of all modern computers, which allows hardware devices of different speeds to communicate without relying on a large interrupt load of the CPU.

And S120, if the cache data stored in the server reaches a saturated state, copying super block information of a file block group of the cache data into a memory of the server.

In this embodiment, the cache data is composed of a plurality of file block groups, each of which includes: inode, superblock, denry cache, and data block. According to the scheme, the super block information is copied into the server memory through DMA (direct memory access) without copying the rest Inode, denry cache and data block, so that the size of DMA copy data is reduced, and the DMA copy efficiency is further improved. The super block contains information for managing the whole file system, based on the super block information of the file block group, the Inode corresponding to each file in the server hard disk can be obtained, and the i_mapping and i_data fields in the Inode structure actually map the address space of each file in the server hard disk, so that all data in the hard disk can be obtained through the super block information.

The Inode, chinese is an Inode, and is configured to store file meta information, where each file has file meta information, and the file meta information includes information that describes a corresponding file, such as file creator information, file creation date information, file address information, and file size information. The file address information is i_mapping and i_data fields in the Inode structure.

Referring to FIG. 3, in one embodiment, step S120 includes steps S121-S123.

S121, analyzing the cache data to obtain a plurality of file block groups forming the cache data.

S122, super block information in each file block group is acquired.

S123, copying the super block information in all the file block groups into a server memory through DMA.

In this embodiment, before the superblock information is copied to the server memory through DMA, the cached data needs to be parsed to obtain the file block groups that compose the cached data, and then the superblock information of each file block group is read from the parsed file block groups and copied to the server memory through DMA technology, so that the client can directly obtain the superblock information through remote transmission by RDMA technology.

S130, the super block information stored in the memory of the server is remotely transmitted to the client through RDMA.

In this embodiment, the present solution uses RDMA technology, and the client may access the data in the server memory through RDMA remote, directly copy the file in the memory (that is, the superblock information described above) and transmit the file to the client, that is, the server remotely transmits the superblock information in the memory to the client through RDMA, and further processes the superblock information by the client, directly copies the superblock information in the server memory to the client through RDMA, without involving the processing of the operating system and the CPU, thereby simplifying the data transmission processing procedure, reducing the data copy transmission amount from the server to the client, and improving the data transmission efficiency.

The RDMA (Remote Direct Memory Access) technology is called remote direct data access, and is used for solving the delay of server data processing in network transmission. RDMA transfers data directly into the memory area of a computer over a network, and moves the data quickly from one system to a remote system memory without any impact on the operating system, thus eliminating the need for more or less computer processing functions. It eliminates the overhead of external memory copying and context switching, thus freeing up memory bandwidth and CPU cycles for improved application system performance.

S140, acquiring a remote data reading request generated by the client based on the super block information, wherein the remote data reading request comprises an address space of data to be read.

In this embodiment, the client analyzes the obtained superblock information to obtain an address space of all or part of the files in the server file system, and generates a remote data reading request according to its own needs and based on the obtained address space, where the remote data reading request includes the address space of the data to be read, and based on the remote data reading request, the data block and other file information in the corresponding address space can be directly copied remotely through DMA, so as to complete data remote transmission between the server and the client.

And S150, according to the remote data reading request, the data to be read is remotely transmitted to the client through the DMA.

In this embodiment, based on the remote data reading request of the client, the client may directly copy the data block and other file information in the corresponding address space on the server remotely through DMA, so as to complete the data remote transmission between the server and the client.

Specifically, the remote data reading request may include a plurality of address spaces for data to be read. For example, the remote data reading request may include address spaces of the data to be read a, the data to be read b and the data to be read c, and the client may directly remotely copy the data blocks and other file information corresponding to the data to be read a, the data to be read b and the data to be read c through DMA based on the remote data reading request, so as to complete data remote transmission between the server and the client.

Referring to FIG. 4, in one embodiment, step S150 includes steps S151-S153.

S151, acquiring data to be read corresponding to the address space according to the address space.

S152, encrypting the data to be read to obtain an encrypted data packet.

And S153, remotely transmitting the encrypted data packet to the client through DMA.

In this embodiment, in order to ensure the integrity and security of data, when the scheme copies and transmits the data to be read from the server to the client, the data to be obtained corresponding to the encryption is encrypted, and the data to be obtained is remotely and directly transmitted to the client through the DMA, so as to ensure the security of the data.

Referring to fig. 5, step S152 includes steps S1521-S1523.

S1521, encrypting the data to be read through a preset encryption algorithm to obtain encrypted data.

In this embodiment, after the data to be read is obtained, an encryption algorithm is used to encrypt the data to be read to obtain encrypted data, so as to ensure data security, where the encryption algorithm may use a conventional/non-conventional encryption algorithm such as a symmetric encryption algorithm, an asymmetric encryption algorithm, etc. to further improve data security.

S1522, packaging the encrypted data into a data packet.

In this embodiment, the encrypted data is encapsulated into a data packet, and the subsequent data transmission is performed in the form of the data packet, so that the transmission quality and efficiency of the encrypted data can be ensured when the data size of the encrypted data is relatively large.

Referring to fig. 6, in the present embodiment, step S1522 includes steps S1522a and S1522b.

S1522a, dividing the encrypted data into a plurality of file portions according to the information amount of the encrypted data.

S1522b, compressing the file parts in parallel to obtain a plurality of sub-data packets.

In this scheme, the encrypted data is packed, that is, the encrypted data is compressed, so as to obtain a data packet in a compressed packet format. In the scheme, the encrypted data is divided into a plurality of file parts according to the information quantity of the encrypted data, the plurality of file parts jointly form the encrypted data, and the file parts are compressed in parallel by preset compression software to obtain sub-data packets in a plurality of compression formats. The data packet in the compressed packet format is a compressed packet file in zip or rar format, and is transmitted after being packaged, so that the data stability of the encrypted data in the transmission process is effectively ensured.

S1523, adding the identification of the encryption algorithm into the packet header of the data packet to obtain the encrypted data packet.

In this embodiment, an identifier of an encryption algorithm is added to a packet header of a data packet, so that a state corresponding to a current data packet can be identified, which indicates that the data packet has been encrypted and encapsulated, and data transmission can be performed.

In the embodiment of the application, when the cache data of the server reaches a saturated state, only superblock information of the cache data is copied to a server memory, the cache data transmission pressure is reduced, and the cache data is remotely transmitted to the client through RDMA (remote direct memory access), the client obtains the address space of the data to be read based on the superblock information analysis, and directly reads the data from the memory of the server through DMA (direct memory access) remote transmission based on the address space, so that the processing of an operating system and a CPU (central processing unit) is not involved, the data transmission processing flow is simplified, the data copying transmission quantity is reduced, the data transmission efficiency is improved, and the universality of the scene of file remote transmission is improved.

Fig. 7 is a flowchart of a file transfer method according to another embodiment of the present application. As shown in fig. 7, the file transfer method of the present embodiment includes steps S210 to S270. Steps S210 to S250 are similar to steps S110 to S150 in the above embodiment, and are not described herein. Steps S260 to S270 added in the present embodiment are described in detail below.

S260, analyzing the super block information to obtain an address space corresponding to the data information data of the data stored in the server.

S270, acquiring an address space of the data to be read according to the data information, and generating a remote data reading request based on the address space of the data to be read.

In this embodiment, the client analyzes the obtained superblock information to obtain data information of all or part of the files in the server file system and address space associated with the data information, and the client determines data to be read according to the data information first, and generates a remote data reading request according to the address space associated with the data to be read, where the remote data reading request includes the address space of the data to be read, and based on the remote data reading request, the data block and other file information in the corresponding address space can be copied remotely and directly through DMA, so as to complete data remote transmission between the server and the client.

Fig. 8 is a schematic block diagram of a file transfer device according to an embodiment of the present application. As shown in fig. 8, the present application also provides a file transmission device corresponding to the above file transmission method. The file transfer device comprises a unit for executing the file transfer method, and the device can be configured in a desktop computer, a tablet computer, a portable computer, and the like. Specifically, referring to fig. 8, the file transfer apparatus 400 includes a buffer determination unit 10, an information copy unit 20, an information transfer unit 30, a request acquisition unit 40, and a data transfer unit 50.

And a buffer judgment unit 10 for judging whether the buffer data stored in the server has reached a saturated state.

In this embodiment, when the file system of the server is in normal use, the user process calls the sys_read () command to enter a kernel mode through the read () command, the process scheduler allocates a CPU (Central Processing Unit ), the CPU sends an IO (Input/Output) operation to the DMA, the DMA drives the hardware through a request, the hardware caches the cached data, and the cached data size also determines the frequency of the interrupt. When the server reaches a saturated state, the CPU interrupt is triggered, and further the DMA auxiliary data is triggered to the memory of the server.

And the information copying unit 20 is configured to copy super block information of a file block group of the cached data to a server memory if the cached data stored in the server has reached a saturated state.

Referring to fig. 9, the information copying unit 20 includes a data parsing module 21, an information acquisition module 22, and an information copying module 23.

The data parsing module 21 is configured to parse the cached data to obtain a plurality of file block groups that compose the cached data.

The information acquisition module 22 is configured to acquire superblock information in each file block group.

And the information copying module 23 is used for copying the super block information in all the file block groups into the memory of the server through DMA.

An information transmission unit 30, configured to remotely transmit the superblock information stored in the server memory to the client through RDMA.

The request acquiring unit 40 is configured to acquire a remote data reading request generated by the client based on the superblock information, where the remote data reading request includes an address space of data to be read.

The data transmission unit 50 is configured to remotely transmit data to be read to the client through DMA according to the remote data reading request.

Referring to fig. 10, in an embodiment, the data transmission unit 50 includes a data acquisition module 51, the data acquisition module 51, and a data transmission module 53.

The data obtaining module 51 is configured to obtain, according to the address space, data to be read corresponding to the address space.

The data encryption module 52 is configured to encrypt data to be read to obtain an encrypted data packet.

The data transmission module 53 is configured to remotely transmit the encrypted data packet to the client through DMA.

Referring to fig. 11, in one embodiment, the data encryption module 52 includes an encryption sub-module 521, an encapsulation sub-module 522, and an identification sub-module 523.

The encryption sub-module 521 is configured to encrypt the data to be read by using a preset encryption algorithm, so as to obtain encrypted data.

The encapsulation sub-module 522 is further configured to divide the encrypted data into a plurality of file portions according to the information amount of the encrypted data; and compressing the file parts in parallel to obtain a plurality of sub-data packets.

In this embodiment, the encrypted data is packed, that is, the encrypted data is compressed, so as to obtain a data packet in a compressed packet format. In the scheme, the encrypted data is divided into a plurality of file parts according to the information quantity of the encrypted data, the plurality of file parts jointly form the encrypted data, and the file parts are compressed in parallel by preset compression software to obtain sub-data packets in a plurality of compression formats. The data packet in the compressed packet format is a compressed packet file in zip or rar format, and is transmitted after being packaged, so that the data stability of the encrypted data in the transmission process is effectively ensured.

The identifier sub-module 523 is configured to add an identifier of an encryption algorithm to a header of the data packet, so as to obtain an encrypted data packet.

In the embodiment of the application, when the server reaches a saturated state, only superblock information of the cache data is copied to the memory of the server, the cache data transmission pressure is reduced, and the cache data is remotely transmitted to the client through RDMA (remote direct memory access), the client obtains the address space of the data to be read based on the superblock information analysis, and directly reads the data from the memory of the server through DMA (direct memory access) remote transmission based on the address space, so that the processing of an operating system and a CPU (central processing unit) is not involved, the data transmission processing flow is simplified, the data copying transmission quantity is reduced, the data transmission efficiency is improved, and the universality of the scene of file remote transmission is improved.

Fig. 12 is a schematic block diagram of a file transfer device according to another embodiment of the present application. As shown in fig. 12, the file transfer apparatus 400 of the present embodiment is an addition of the information parsing unit 60 and the request generating unit 70 to the above-described embodiments.

The information parsing unit 60 is configured to parse the super block information to obtain data information of the data stored in the server and an address space corresponding to the data.

The request generating unit 70 is configured to obtain an address space of data to be read according to the data information, and generate a remote data reading request based on the address space of the data to be read.

It should be noted that, as will be clearly understood by those skilled in the art, the specific implementation process of the above-mentioned file transfer device 400 and each unit may refer to the corresponding description in the foregoing method embodiments, and for convenience and brevity of description, the description is omitted here.

The file transfer means described above may be implemented in the form of a computer program which is executable on a computer device as shown in fig. 13.

Referring to fig. 13, fig. 13 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a terminal or a server, where the terminal may be an electronic device with a communication function, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster formed by a plurality of servers.

With reference to FIG. 8, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a file transfer method.

The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.

The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform a file transfer method.

The network interface 505 is used for network communication with other devices. It will be appreciated by those skilled in the art that the structure shown in FIG. 13 is merely a block diagram of some of the structures associated with the present inventive arrangements and does not constitute a limitation of the computer device 500 to which the present inventive arrangements may be applied, and that a particular computer device 500 may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

Wherein the processor 502 is adapted to run a computer program 5032 stored in a memory.

It should be appreciated that in an embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.

Accordingly, the present application also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions.

The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.

The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the application can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a terminal, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application.

While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims

1. A method of file transfer comprising the steps of:

judging whether the cache data stored in the server reaches a saturated state or not;

if the cache data stored in the server reaches a saturated state, copying super block information of a file block group of the cache data into a server memory;

remotely transmitting the superblock information stored in the server memory to a client through RDMA;

acquiring a remote data reading request generated by the client based on the super block information, wherein the remote data reading request comprises an address space of data to be read;

acquiring data to be read corresponding to the address space according to the address space;

encrypting the data to be read to obtain an encrypted data packet;

and remotely transmitting the encrypted data packet to the client through DMA.

2. The method according to claim 1, wherein the step of encrypting the data to be read to obtain an encrypted data packet includes:

encrypting the data to be read through a preset encryption algorithm to obtain encrypted data;

encapsulating the encrypted data into a data packet;

and adding the identification of the encryption algorithm into the packet header of the data packet to obtain the encrypted data packet.

3. The file transfer method according to claim 2, wherein the step of encapsulating the encrypted data into a data packet comprises:

dividing the encrypted data into a plurality of file portions according to the information amount of the encrypted data;

and compressing the file parts in parallel to obtain a plurality of sub data packets.

4. The file transfer method according to claim 1, wherein the step of acquiring the remote data read request generated by the client based on the superblock information includes, before:

analyzing the super block information to obtain data information of data stored in a server and an address space corresponding to and associated with the data;

and acquiring an address space of the data to be read according to the data information, and generating a remote data reading request based on the address space of the data to be read.

5. The method according to claim 1, wherein the step of copying super block information of the file block group of the cache data into a server memory includes:

analyzing the cache data to obtain a plurality of file block groups forming the cache data;

acquiring super block information in each file block group;

and copying the super block information in all the file block groups into a server memory through DMA.

6. A document transfer apparatus, comprising:

the buffer judgment unit is used for judging whether the buffer data stored in the server reaches a saturated state or not;

the information copying unit is used for copying super block information of a file block group of the cache data to a server memory if the cache data stored in the server reaches a saturated state;

the information transmission unit is used for remotely transmitting the super block information stored in the server memory to the client through RDMA;

the request acquisition unit is used for acquiring a remote data reading request generated by the client based on the super block information, wherein the remote data reading request comprises an address space of data to be read;

the data transmission unit is used for acquiring data to be read corresponding to the address space according to the address space; encrypting the data to be read to obtain an encrypted data packet; and remotely transmitting the encrypted data packet to the client through DMA.

7. The file transfer device of claim 6, wherein the information copying unit comprises a data parsing module, an information acquisition module, and an information copying module;

the data analysis module is used for analyzing the cache data to obtain a plurality of file block groups forming the cache data;

the information acquisition module is used for acquiring super block information in each file block group;

and the information copying module is used for copying the super block information in all the file block groups into a server memory through DMA.

8. A computer device, characterized in that it comprises a memory and a processor, on which a computer program is stored, which processor, when executing the computer program, implements the file transfer method according to any of claims 1-5.

9. A storage medium storing a computer program comprising program instructions which, when executed by a processor, implement the file transfer method of any of claims 1-5.