CN115361374A

CN115361374A - File transmission method and device and electronic equipment

Info

Publication number: CN115361374A
Application number: CN202210886059.XA
Authority: CN
Inventors: 高亚龙
Original assignee: Zhongdian Jinxin Software Co Ltd
Current assignee: Zhongdian Jinxin Software Co Ltd
Priority date: 2022-07-26
Filing date: 2022-07-26
Publication date: 2022-11-18

Abstract

The embodiment of the application provides a file transmission method and device, electronic equipment and related products, and relates to the field of data transmission. The method comprises the following steps: and responding to a transmission instruction carrying transmission parameters, traversing a first shared directory configured by the first client to determine a first original file to be transmitted, and determining a second node configured by a second client executing receiving operation. And determining a target processing mode according to the size of the first original file so as to process the first original file, obtain a first subfile convenient to transmit and transmit the first subfile. According to the method, before transmission, the processing mode of the file to be transmitted is determined, and the sub-file convenient to transmit is obtained through corresponding processing, so that even if the file to be transmitted is a large number of small files or super-large files, the file can be efficiently transmitted through the scheme.

Description

File transmission method and device and electronic equipment

Technical Field

The present application relates to the field of data transmission technologies, and in particular, to a file transmission method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

Background

In banking, file exchange (including the transfer of a file from one client to another) is a common workflow. In the prior art, a file transfer system is generally adopted to cooperate with two clients to process the workflow.

At present, when a file transmission system is used for file transmission operation, only serial and parallel transmission modes can be provided. For example, when a large amount of small files (millions of small data files, such as invoices) are transmitted, it is time consuming to transmit the files in a serial transmission manner. For example, when a very large file is faced (a file with hundreds of G data size, such as a package), the parallel transmission method puts high requirements on the bandwidth.

Therefore, how to satisfy the transmission of massive files or super-large files becomes a technical problem to be solved.

Disclosure of Invention

An embodiment of the present invention provides a file transmission method, a file transmission apparatus, an electronic device, a computer-readable storage medium, and a computer program product, so as to solve one of the above technical problems. In order to achieve the purpose, the embodiment of the application provides the following scheme.

In one aspect, an embodiment of the present application provides a file transmission method, where the method includes:

responding to a transmission instruction carrying transmission parameters, traversing a first shared directory configured by a first client to determine a first original file to be transmitted; the transmission parameters comprise an identifier of a second client; determining a target processing mode according to the storage space occupied by the first original file and/or the number of the files contained in the first original file; processing the first original file according to a target processing mode to obtain a first subfile to be transmitted; wherein the processing of the first original file comprises: splitting and merging; searching online nodes configured by the second client through the registry, and determining the second nodes from the online nodes; and transmitting the first subfile and the target processing mode to the second node.

Optionally, determining a target processing manner according to the storage space occupied by the first original file and/or the number of files included in the first original file includes:

calculating the total number of files in the first original file and the total storage space occupied by the first original file; if the total number is larger than a first threshold value, determining that the target processing mode is a merging processing mode; and if the total amount is smaller than or equal to the first threshold and the total storage space is larger than the second threshold, determining that the target processing mode is the splitting processing mode.

Optionally, after the step of calculating the total number of files in the first original file and the total storage space occupied by the first original file, the method further comprises:

if the total storage space is larger than a second threshold value, determining that the target processing mode is a splitting processing mode; and if the total storage space is less than or equal to the second threshold and the total amount is greater than the first threshold, determining that the target processing mode is a merging processing mode.

Optionally, processing the first original file according to the target processing manner to obtain a first subfile to be transmitted, including:

if the target processing mode is a merging processing mode, merging a plurality of files contained in the first original file according to the preset unit file quantity to obtain at least one first subfile; each first subfile is configured with a file sequence number corresponding to the merging processing sequence;

if the target processing mode is a splitting processing mode, splitting a plurality of files contained in the first original file according to a preset unit storage space to obtain at least one first subfile; each first subfile is configured with a file sequence number corresponding to the order of the splitting process.

Optionally, searching for an online node configured by the second client through the registry, and determining the second node from the online nodes, includes:

sending a request carrying the identifier of the second client to the registration center; the transmission parameter carries the identifier of the second client; and responding to the online list corresponding to the second client fed back by the registry, and determining the corresponding node in the online list, the working state of which is an idle state, as the second node.

Optionally, before traversing the first shared directory configured by the first client, the method further includes:

acquiring a first authentication result; the first authentication result represents whether the first node is configured with readable permission to the first shared directory; acquiring a second authentication result according to the identifier of the second client; the second authentication result represents whether the first client configures the writable right of the second shared directory configured by the second client.

And traversing the first shared directory configured by the first client if the first authentication result is configured with the readable permission and the second authentication result is configured with the writable permission.

Optionally, if a second subfile sent by a third node is received, the third node is a node configured by a third client; the method further comprises the following steps:

if the current node is the node for executing the sorting operation, counting all the second subfiles;

and if the corresponding target processing mode of the second subfiles is a merging processing mode, determining the path level of each second subfile according to the corresponding file sequence number of each second subfile, and recombining according to the path levels to obtain a second original file consisting of a preset number of files.

And if the target processing mode corresponding to the second subfile is a split processing mode, determining the path level of each second subfile according to the file sequence number corresponding to each second subfile, and performing merging processing according to the path levels to obtain a second original file.

On the other hand, the embodiment of the present application further provides a file transmission device, and the device includes:

and the scanning module is used for responding to a transmission instruction carrying transmission parameters, and traversing the first shared directory configured by the first client to determine a first original file to be transmitted.

And the first determining module is used for determining a target processing mode according to the storage space occupied by the first original file and/or the number of the files contained in the first original file.

The processing module is used for processing the first original file according to a target processing mode to obtain a first subfile to be transmitted; wherein, the processing of the first original file comprises: splitting and combining.

And the second determining module is used for searching the online nodes configured by the second client through the registry and determining the second nodes from the online nodes.

And the transceiver module is used for transmitting the first subfile and the target processing mode to the second node.

In another aspect, an embodiment of the present application provides an electronic device, including:

a memory, a processor and a computer program stored on the memory, the processor executing the computer program to perform the steps of the method shown in the above two aspects of the embodiments of the present application.

Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method shown in the above two aspects of the embodiments of the present application.

Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the method shown in the above two aspects of the embodiments of the present application are implemented.

The technical scheme provided by the embodiment of the application has the following beneficial effects:

the embodiment of the application provides a file transmission method, which comprises the following steps: and responding to a transmission instruction carrying transmission parameters, traversing a first shared directory configured by the first client to determine a first original file to be transmitted, and determining a second node configured by a second client executing receiving operation. Determining a target processing mode of the first original file according to the size (such as occupied storage space and file number) of the first original file so as to process the first original file, thereby obtaining a first subfile convenient to transmit; the first subfile is then transmitted to the second node. Before transmission, the method in the embodiment of the application determines a processing mode of a file to be transmitted (such as a first original file), and performs corresponding processing, so as to obtain a subfile (such as a first subfile) convenient to transmit, so as to perform transmission operation. For example, a large amount of small files are merged, and a large amount of files are split. Therefore, even if the files to be transmitted are massive small files or huge files, the files can be efficiently transmitted to the target client through the scheme.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.

Fig. 1 is a schematic diagram of a file transfer system according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a file transfer method according to an embodiment of the present disclosure;

fig. 3 is a schematic view of an application scenario of a file transmission method according to an embodiment of the present application;

fig. 4a is a schematic structural diagram of a document transportation device according to an embodiment of the present application;

fig. 4b is a schematic structural diagram of another document transportation device according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the embodiments set forth below in connection with the drawings are exemplary descriptions for explaining technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates at least one of the items defined by the term, e.g., "a and/or B" can be implemented as "a", or as "B", or as "a and B".

To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.

In the background art, the existing file transmission system generally adopts a serial transmission mode and a parallel transmission mode to transmit files. The serial transmission mode is to transmit a plurality of files to be transmitted in sequence, that is, to transmit one file at a time until all the files are transmitted. The parallel transmission mode is to allow at least two files of a plurality of files to be transmitted simultaneously, or all the files to be transmitted simultaneously. However, for a large amount of small files, the serial transmission method is time-consuming, or the serial transmission method puts requirements on bandwidth. For a single super-large file, both serial transmission and parallel transmission will have high requirements on bandwidth.

In order to solve at least one technical problem in the foregoing background, an embodiment of the present application provides a file transmission method. According to the method, the original file to be transmitted is split or combined on the first node of the first client, so that the subfiles convenient to transmit are obtained and transmitted. And after the second node of the second client receives the subfiles to be transmitted, finishing operation is executed according to the received processing mode subfiles to obtain the original file. The method can realize the purpose of efficient transmission when facing the original files comprising a large number of small files or the original files comprising a single super-large file.

The file transmission method provided by the embodiment of the application can be applied to any node in a distributed cluster configured on a client, namely a server in the distributed cluster. The client may be a first client, a second client, or a third client. The first client transmits the file to the second client, and the third client transmits the file to the first client. Wherein, each server provides a larger space for file storage operation.

Optionally, the method provided by the embodiment of the present application may be implemented as an independent application program or a functional module/plug-in of an application program, for example, the application program may be software or an application program for special file transfer, and the processing operation for the original file, which is required in the file transfer process, may be implemented by the application program to obtain the subfile, and the sorting operation for the subfile may be implemented to obtain the original file.

The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below through descriptions of several exemplary embodiments. It should be noted that the following embodiments may be referred to, referred to or combined with each other, and the description of the same terms, similar features, similar implementation steps and the like in different embodiments is not repeated.

Fig. 1 shows a schematic architecture diagram of a file transfer system provided in an embodiment of the present application, and as shown in fig. 1, a file transfer system 10 may include the following modules: an information entry center 110, a registry 120, and a management center 130.

And the information entry center 110 is used for entering the configuration information of the client. Specifically, the configuration information of the client may include: identification of the client, transmission relation authority and shared directory; the client is configured with a distributed cluster, and the configuration information of the client may further include: the access address of the distributed cluster, information of nodes serving the client in the distributed cluster, such as the number of nodes, the identification of the nodes, the access authority of each node to the shared directory, and the transmission mode of each node.

Optionally, the information entry module is further configured to enter configuration information of the node. Specifically, the configuration information of the node includes: the identification of the client to which the node belongs, the IP address and port number of the node, the identification of the node and the like.

Optionally, the information entry module 110 further stores the entered configuration information of the client and the configuration information of the node.

And the registry 120 is configured to determine that the node is an online node according to the received registration message.

Alternatively, the file transfer system 10 includes management of a plurality of areas, and a registry is provided in each area. Wherein the registry of each region is responsible for managing registration transactions for nodes of the respective region. Specifically, for example, a node a configured in the client 1 sends a request message to the file transfer system after the node a is started, so as to obtain configuration information related to the node a. Wherein the relevant configuration information includes: identification of client 1; information of the associated registry; shared directory of client 1; access rights, such as node a's readable and/or writeable rights to the shared directory; the transmission authority, the authority for transmitting the file in one way and the authority for transmitting the file in two ways, such as: the client 1 has the authority to transmit the file to the client 2, but the client 2 cannot transmit the file to the client 1, which can be called the authority to transmit the file in one way; the transmission modes include a one-to-one mode and a one-to-many mode, for example, when the node a performs a transmission operation, it selects to transmit to one node or to transmit to a plurality of nodes. In addition, the IP address and port number of node a are included.

The registry 120 arranges the nodes in order according to the client identifiers. Optionally, a corresponding online list of each client is generated, where the online list includes online nodes configured by the client.

And then, the node A sends a registration message to the registration center of the corresponding area according to the acquired relevant configuration information of the node A. After receiving the registration message of the node a, the registry of the corresponding area performs a registration operation, such as registering relevant configuration information of the node a, and setting the state of the node a to be an online state.

Optionally, a heartbeat mechanism may be established between the registry 120 of the corresponding area and the node a to confirm whether the node a is online and to determine the operating status of the node a. Specifically, the working state is a busy state or an idle state. The node a may be a first node, a second node, or a third node. The client 1 may be a first client, a second client or a third client

Optionally, if at least one node configured by the client becomes an online node, which indicates that the client is online, an operation of transferring a file may be performed between the online clients.

Exemplarily, a first client and a second client are taken as an example. If at least one node configured by the first client is an online node, the state of the first client is online, and at least one node configured by the second client is an online node, the state of the second client is online, and a process of transmitting files can be established between the first client and the second client. And if so, sending a transmission instruction carrying the identifier of the second client to the first client to determine to transmit the file to the second client.

And the control center 130 is used for controlling the transmission process. Optionally, after receiving the transmission instruction, the first node further sends a notification message to the management and control center; when the first node transmits the subfile to be transmitted to the second node, the first node also sends a notification message to the management and control module. And after receiving the notification message, the management and control center displays the notification message according to the content carried in the notification message and receives a control instruction input by a user based on the displayed content.

In order to understand how the file transfer system 10 participates in the file transfer process, the embodiment of the present application also provides a specific example. As shown in fig. 1, the file producer a corresponds to a first client, and the file consumer B corresponds to a second client.

In this example, the file producer a includes a function of generating a file of the type as file 1, and is configured with a plurality of servers. The file consumer B subscribes to the file 1 to the file transfer system, and once a new file 1 is generated in the file producer a, the file producer a sends the file 1 to the file consumer B. Therefore, there are many history files 1 in the file consumer B.

The first phase, the registration phase of the server.

After the server configured by the document producer a is started, registration information is sent to the registry of the document delivery system to register the server as being online. The same is true for the server of file consumer B.

In the second stage, the servers that are online are determined.

After generating a new file 1, the file producer a obtains configuration information of a file consumer who subscribes to the file 1, such as information of an online server of the file consumer B, from a registry of the file transmission system.

Wherein, the file consumer A determines whether the transmission authority with the file consumer B exists before acquiring the subscription message. E.g., whether file consumer a configures the right to transfer the file to file transferor B. And if so, acquiring the information of the online server.

The file producer a determines a server that needs to perform a receiving operation according to the information of the online server of the file consumer B. Any online server of the file producer a creates a connection with the online server of the file consumer B.

And in the third stage, transmitting the file.

The document producer a processes the document 1 to be transmitted. When the file producer A transmits the file, the information of the transmission process can be sent to a control center of the file transmission system. Such as: the name and occupied storage space of the currently transmitted original file, the total number of the contained files, the subfiles which are currently transmitted to file consumers and the transmission progress. And the management and control center of the file transmission system displays the data after receiving the data.

Fig. 2 shows a flowchart of a file transmission method according to an embodiment of the present application. As shown in fig. 2, the method includes steps S210 to S250.

S210, in response to the transmission instruction carrying the transmission parameter, traversing the first shared directory configured by the first client to determine a first original file to be transmitted.

The nodes configured by the first client are nodes in a distributed cluster, the distributed cluster comprises at least one server, and each node is a server. The nodes configured on the first client share a directory for storing files, namely a first shared directory.

When the first node is started, the first client establishes network connection with the first node. Through the established network connection, the first client may send a transmission instruction to the first node.

Alternatively, the user may set the access right of each first node to any subdirectory in the first shared directory. The access rights include read rights to read files in the subdirectory and write rights to write files to the subdirectory.

After traversing the first shared directory configured by the first client according to the identifier of the original file carried in the transmission parameter, the storage path corresponding to the first original file can be determined.

Optionally, the first original file may be a folder, where the folder includes a plurality of subfiles or subfolders; the first original file may also be a single file; the first original file may also be a combination of a folder and a file. Alternatively, the single file may be a compressed file, such as a file suffixed with ". Zip", or an uncompressed normal file, such as a file suffixed with ". Txt", "gif".

S220, determining a target processing mode according to the storage space occupied by the first original file and/or the number of the files contained in the first original file.

And S230, processing the first original file according to the target processing mode to obtain a first subfile to be transmitted.

Wherein the processing of the first original file comprises: splitting and merging. That is, the processing manner for the first original file includes a merging processing manner and a splitting processing manner.

Optionally, the number of the first subfiles is one or more than one.

Optionally, the storage space represents a storage space occupied when storing in a static form.

S240, searching the online nodes configured by the second client through the registry, and determining the second nodes from the online nodes.

Wherein, the registry may be the registry 120 provided in the above embodiment.

Optionally, the transmission parameter carries an identifier of the second client. And sending a request carrying the identifier of the second client to the registry, responding to an online list corresponding to the second client fed back by the registry, and determining a corresponding node in the online list, the working state of which is an idle state, as the second node.

And S250, transmitting the first subfile and the target processing mode to a second node.

The target processing mode is transmitted to the second node, which is also convenient for the second node of the second client to perform a sorting operation on the received first subfile according to the target processing mode to obtain the first original file. Optionally, each target processing manner further includes a corresponding identifier, such as: the splitting process is identified by the number "1" and the merging process is identified by the number "2".

The embodiment of the application provides a file transmission method, which determines a target processing mode for a first original file by analyzing the size (such as occupied storage space and file number) of the first original file, so as to process the first original file according to the target processing mode, thereby obtaining a first subfile convenient to transmit; and then, transmitting the first subfile and the target processing mode to the second node. The second node can perform a sorting operation on the received first subfile according to the target processing mode to obtain a first original file. In the method shown in the embodiment of the application, before transmission, a processing mode of a file to be transmitted (such as a first original file) is determined, and corresponding processing is performed, so that a subfile (such as a first subfile) convenient to transmit is obtained, and transmission operation is performed. Even if the files to be transmitted are massive small files or huge files, the files can be efficiently transmitted to the target client through the scheme.

After receiving the transmission instruction, the transmission instruction further includes the information of the uploaded destination client, that is, the information of the second client. However, the second client is an application or a plug-in on the terminal, and cannot perform a specific receiving operation, so the embodiment of the present application further provides a specific process how to determine the second node that performs the receiving operation.

In an optional embodiment, searching for an online node configured by the second client through the registry, and determining the second node from the online node may specifically include:

sending a request carrying an identifier of a second client to a registration center, wherein the transmission parameter carries the identifier of the second client; and responding to the online list corresponding to the second client fed back by the registry, and determining the corresponding node in the online list, the working state of which is an idle state, as the second node.

Optionally, the registry comprises a respective presence list for each client, the presence list being identified by an identification of the respective client. For example, the registry includes online lists corresponding to the first client, the second client, and the third client. The first node is a node in an online list corresponding to the first client; the second node is a node in an online list corresponding to the second client; the third node is a node in the online list corresponding to the third client.

The embodiment of the present application further provides an embodiment, which shows a generation process of an online node.

In an alternative embodiment, the first node enters the initialization phase after startup.

And sending a registration message carrying the relevant configuration information to a registration center so as to become an online node.

Specifically, after being started, any node acquires relevant configuration information from the information entry center, where the configuration information may include configuration information of the started node and/or configuration information of a corresponding client. Since a node is online after it is started, a message is sent to the registry only to inform the registry of the current status. After the node sends the relevant configuration information to the registration center, the registration center caches the relevant configuration information sent by the node and records that the state of the node is an online state.

Optionally, the registration message further carries an identifier of the corresponding client. After receiving the registration message, the registry can also compile the node into the corresponding online list of the corresponding client according to the identifier of the corresponding client.

In the background art, the document to be transmitted may be a document like an invoice generated by banking, which relates to the information security of the user. Therefore, for the safety of the transmission process, the transmission authority between the clients and the access authority of the node are also set so as to avoid the leakage of the file.

In an optional embodiment, before traversing the first shared directory configured by the first client, the method further comprises:

acquiring a first authentication result, wherein the first authentication result represents whether the first node is configured with readable authority to the first shared directory or not; acquiring a second authentication result according to the identifier of the second client; the second authentication result represents whether the first client configures the writable right of the second shared directory configured by the second client.

Optionally, in the configuration information of the first node, information of the transmission right of the first client is recorded. Specifically, whether the first client has the writable right to the second shared directory configured by the second client is determined according to the recorded information of the transmission right.

Optionally, if the first authentication result is configured with a readable permission and the second authentication result is configured with a writable permission, the first shared directory configured by the first client is traversed.

Optionally, if the first authentication result is that no readable permission is configured, the first node finishes executing. And the first client may send a transfer instruction to the other nodes in the corresponding online list in order to restart the file transfer operation.

Optionally, if the second authentication result is that the writable right is not configured, it is determined that the file transfer operation cannot be performed, that is, the first shared directory does not need to be scanned, and the online node configured by the second client is obtained.

Next, how to determine the target processing manner according to the storage space occupied by the first original file and/or the number of files contained in the first original file will be described.

In an optional embodiment, determining the target processing manner according to the storage space occupied by the first original file and/or the number of files included in the first original file specifically includes:

and calculating the total number of files in the first original file and calculating the total storage space occupied by the first original file.

If the total number is larger than a first threshold value, determining that the target processing mode is a merging processing mode; if the total amount is smaller than or equal to a first threshold value and if the total storage space is larger than a second threshold value, determining that the target processing mode is a splitting processing mode;

alternatively, the first and second liquid crystal display panels may be,

Optionally, a storage path of the first original file in the first shared directory is determined according to storage information carried in the transmission parameter. Scanning the storage path in the first shared directory can obtain the number of files corresponding to the storage path and the attribute (including the storage space occupied by the corresponding file) information of each file, thereby determining the storage space occupied by the first original file and the number of the included files.

Optionally, the scanning the storage path in the first shared directory may specifically include: scanning a first-tier path of the storage paths, the first-tier path may include individual files and/or folders therein; counting the number of the single files and/or the occupied storage space; and aiming at the folders, acquiring a second layer path corresponding to the inside of each folder, and counting the file information in the second layer path in a first layer path counting way. And determining that the statistics is finished until the last layer of path is entered. The total amount of the first original files and/or the total storage space occupied by the first original files can be obtained.

When the number of files included in the first original file reaches a first threshold value, merging is required. For example, the first original file is a massive invoice file.

After the target processing mode is determined, specific processing is performed according to the target processing mode.

In an optional embodiment, the processing is performed on the first original file according to a target processing manner to obtain a first subfile to be transmitted, which may specifically include steps Sa1 and Sa2:

sa1, if the target processing mode is a merging processing mode, merging a plurality of files contained in the first original file according to a preset unit file number to obtain at least one first subfile; wherein each first subfile is configured with a file sequence number corresponding to the order of the merging process.

Optionally, when the merging process is performed, starting from a first layer path in the storage paths of the first original file: combining the single files in the first layer path according to a preset unit file number to obtain at least one combined file, and configuring a file sequence number for the at least one combined file, such as 'first layer & 01', 'first layer & 02', and the like; and aiming at the folders in the first layer path, acquiring a second layer path corresponding to the inside of each folder, and executing the processing of the files and/or the folders in the first layer path aiming at the files and/or the folders in the second layer path. For the file sequence number of a single file in the folder, the optional defining mode is as follows: "first layer & first layer folder 01& second layer &01". And finally, obtaining the merged file under each layer of path.

Alternatively, the merging process may be a compression process. The merged file is a compressed packet obtained by compressing a plurality of files.

Optionally, when the merging processing is performed, whether the merging processing is to be performed may also be determined according to the total storage space of the files to be merged. If the total storage space of the files to be merged is not larger than the merging threshold, merging processing is carried out. If the storage space occupied by the single file is larger than the merging threshold, at least one file is removed so that the total storage space of the files to be merged is not larger than the merging threshold, and merging processing is performed again.

Optionally, when merging is performed, the first original file is a file of multiple batches, and when merging is performed, merging may be performed preferentially according to the batches, and then merging may be performed according to the merging threshold.

Sa2, if the target processing mode is a splitting processing mode, splitting a plurality of files contained in the first original file according to a preset unit storage space to obtain at least one first subfile; and each first subfile is configured with a file sequence number corresponding to the splitting processing sequence.

In general, files that are only compressed packages will typically be large.

Optionally, when the splitting process is performed, starting from a first layer path in the storage paths of the first original file: aiming at a single file in a first layer path, checking whether the storage space occupied by each file is not less than the preset unit storage space, if the check result is not less than the preset unit storage space, continuously judging whether the corresponding file is a compressed file, if the corresponding file is the compressed file, splitting the corresponding file according to the preset unit storage space, obtaining at least one split file through splitting, and configuring a file sequence number for the at least one split file, if: "first layer & file name 1&01" \ 8230 "," first layer & file name 1&0n "," first layer & file name 2&01"," first layer & file name 2&02"; and aiming at the folders in the first layer path, acquiring a second layer path corresponding to the inside of each folder, and executing the processing of the files and/or the folders in the first layer path aiming at the files and/or the folders in the second layer path. The "file name 1" and "file name 2" in the file number may be temporary names or original file names.

Optionally, the splitting is decompression, and the split file is a compressed file obtained by decompressing a single file and then compressing the single file according to a preset unit storage space.

Optionally, if the storage space occupied by each file in the first original file is smaller than the preset unit storage space, the splitting process may not be performed.

In a specific implementation, whether to process the first original file may be determined according to a total storage space occupied by the first original file and/or a total number of files included in the first original file.

Optionally, if the total amount is not greater than the first threshold and the total storage space is not greater than the second threshold, the first original file may not be processed and may be directly transmitted.

Either the splitting process or the merging process, at least one file is generated. Next, how to transmit at least one file will be explained.

In an optional embodiment, the transmitting the first subfile to the second node may specifically include:

sequentially adding each first subfile to a cache queue according to the corresponding file sequence number; and sequentially acquiring the first subfiles from the buffer queue and transmitting the first subfiles to the second node.

Specifically, a buffer queue is set. And each time a first subfile is obtained, adding the first subfile to the cache queue according to the file sequence number of the first subfile. And when the transmission interface is idle, acquiring the first subfile from the buffer queue and sending the first subfile to the second node.

Optionally, if the transmission mode of the first node is configured to be one-to-many transmission, a plurality of second nodes may be selected from online nodes of the second client, and the first subfiles in the cache queue are respectively transmitted to the plurality of second nodes.

The above embodiments illustrate how file processing and transmission may be performed when the first client is the object of transmission. When the first client is a receiving object of the original file, how to sort the received file is also an important point. The embodiment of the present application further provides an alternative embodiment to illustrate the receiving and sorting processes.

In an optional embodiment, if the second subfile sent by the third node is received, the method further includes:

and the third node is a node of a third client. Optionally, if the current node is a node that executes the sorting operation, the method includes obtaining the received second subfile from another third node, including obtaining a corresponding target processing manner of the received second subfile from another third node. The target processing mode is a splitting processing mode or a combining processing mode.

And if the target processing mode corresponding to the second subfile is a merging processing mode, determining the path level of each second subfile according to the file sequence number corresponding to each second subfile, and combining according to the path levels to obtain a second original file consisting of files with preset number.

Optionally, the second subfile with "first layer" content in the file sequence number is processed as follows: respectively decompressing each second subfile to obtain a single file under the first layer path; the second subfile with the contents of the first layer and the folder y of the first layer in the file sequence number is processed as follows: the compression processing is performed separately, and the compressed data is combined into a folder, and the file name of the folder is "folder y". And merging the single file under the first-layer path and the folder under the first-layer path to obtain the file content under the first-layer path. The processing method of other path layers can refer to the processing method of files or folders in the path of the first layer.

Optionally, the second subfile with the file sequence number "first layer" and "file name x" is processed as follows: respectively decompressing, merging the files obtained by decompressing, compressing to obtain compressed files, and configuring the name of the compressed file as the 'file name x'. Wherein the "file name x" is the original file name. The above process is performed for each subfile, resulting in all files and/or folders under the first-tier path. The processing mode of other path layers can also be according to the processing mode of files or folders in the path of the first layer. And determining the files in all the paths as second original files.

The embodiment of the application is suitable for any file transmission scene, such as: the method comprises the following steps of transmitting a large amount of small files, transmitting super-large files or transmitting common files. The transmission efficiency of the scheme provided by the embodiment of the application to the massive small files or the huge files is much higher than that of the prior art. In order to more clearly understand the whole transmission process, the embodiment of the present application further provides a specific example for description.

In this example, a specific example is that application a transfers a file to application B. The application A is configured with a node A1 and a node A2 \8230, the node An is configured with a node B1 and a node B2 \8230, and the application B is configured with a node Bm. As shown in fig. 3, the application scenario of the file transmission method includes steps S1001 to S1010.

S1001, the node A1 performs registration.

For example, after node A1 boots up, a registration message is sent to the registry. After receiving the information of the node A1, the registration center records the information. And node A1 is programmed into the online list of application a.

After the nodes B1 and B2 start up, they also need to send registration messages to the registry. After receiving the registration messages sent by node B1 and node B2, the registration center also records and programs node B1 and node B2 into the online list of application B.

S1002, the node A1 receives the transmission instruction.

And the node A1 receives a transmission instruction sent by the application A, and scans the shared directory configured by the application A according to the storage information carried in the transmission instruction to obtain an original file to be transmitted.

S1003, the node A1 processes the original file to obtain a plurality of subfiles.

And calculating the total number of files contained in the original file and the total storage space occupied by the original file. If the total amount is larger than a first threshold value, determining that the original file belongs to the type of the massive files, and if the total storage space is larger than a second threshold value, determining that the original file belongs to the type of the super-large files.

And if the original file belongs to the type of the mass files, merging the original file. Specifically, the batch compression is performed in a preset unit number.

And if the original file belongs to the type of the super large file, splitting the original file. Specifically, the split is performed in batches according to a preset unit storage space, and then the compression is performed.

S1004, the node A1 caches the processed subfiles based on the cache queue.

S1005, the node A1 determines the node B1 and the node B2

And sending a request to a registry to acquire an online list corresponding to the application B, and acquiring the nodes of which the node B1 and the node B2 are online from the online list corresponding to the application B.

Node A1 and node B2 create a network connection.

S1006, node A1 transmits the file to node B1 and node B2.

Node A1 obtains the subfiles from the cache queue and transmits the subfiles to node B1 and node B2.

S1007, the node A1 feeds back the transmission result to the application a.

For example, after the transmission is completed, the node A1 feeds back a message of "transmission success" to the application a.

At this point, the node A1 completes the transmission task.

Fig. 4a shows a schematic structural diagram of a document transmission device according to an embodiment of the present application. As shown in fig. 4a, the apparatus 400 comprises:

the scanning module 410 is configured to, in response to a transmission instruction carrying a transmission parameter, traverse a first shared directory configured by a first client to determine a first original file to be transmitted; the transmission parameter includes an identifier of the second client.

The first determining module 420 is configured to determine a target processing manner according to the storage space occupied by the first original file and/or the number of files included in the first original file.

The processing module 430 is configured to process the first original file according to a target processing manner to obtain a first subfile to be transmitted; wherein the processing of the first original file comprises: splitting and merging.

The second determining module 440 is configured to search for an online node configured by the second client through the registry, and determine the second node from the online node.

The transceiver module 450 is configured to transmit the first subfile and the target processing manner to the second node.

Optionally, the first determining module 420, in determining the target processing manner according to the storage space occupied by the first original file and/or the number of files included in the first original file, is specifically configured to:

calculating the total number of files in the first original file, and/or calculating the total storage space occupied by the first original file;

if the total number is larger than a first threshold value, determining that the target processing mode is a combination processing mode; and if the total amount is smaller than or equal to the first threshold and the total storage space is larger than the second threshold, determining that the target processing mode is the splitting processing mode.

Alternatively, the first and second electrodes may be,

if the total storage space is larger than a second threshold value, determining that the target processing mode is a splitting processing mode; and if the total storage space is smaller than or equal to the second threshold and the total amount is larger than the first threshold, determining that the target processing mode is a merging processing mode.

Optionally, the processing module 430 is configured to, in processing the first original file according to the target processing manner to obtain a first subfile to be transmitted, specifically:

if the target processing mode is a merging processing mode, merging a plurality of files contained in the first original file according to the preset unit file quantity to obtain at least one first subfile; each first subfile is configured with a file sequence number corresponding to the merging processing sequence; if the target processing mode is a splitting processing mode, splitting a plurality of files contained in the first original file according to a preset unit storage space to obtain at least one first subfile; and each first subfile is configured with a file sequence number corresponding to the splitting processing sequence.

Optionally, the second determining module 440 searches for an online node configured by the second client through the registry, and determines the second node from the online node, where the second node is specifically configured to:

sending a request carrying an identifier of a second client to a registration center, wherein the transmission parameter carries the identifier of the second client; and responding to the online list corresponding to the second client fed back by the registry, and determining the corresponding node in the online list, the working state of which is the idle state, as the second node.

Optionally, as shown in fig. 4b, the apparatus 400 further includes a verification module 460, before traversing the first shared directory configured by the first client, the verification module 460 is configured to:

and acquiring a first authentication result, wherein the first authentication result represents whether the first node is configured with readable permission to the first shared directory.

Acquiring a second authentication result according to the identifier of the second client; the second authentication result represents whether the first client configures the writable right of a second shared directory configured by the second client; and traversing the first shared directory configured by the first client if the first authentication result is configured with the readable permission and the second authentication result is configured with the writable permission.

Optionally, the apparatus 400 further includes a registration module 470, where the registration module 470 is specifically configured to, in an initialization stage:

sending a registration message carrying relevant configuration information to a registration center so as to become an online node; the registration message also carries an identification of the first client.

Optionally, if a second subfile sent by a third node is received, the third node is a node configured by a third client; the processing module 430 may also be configured to:

and if the current node is the node for executing the sorting operation, counting all the second subfiles.

If the target processing mode corresponding to the second subfile is a merging processing mode, determining the path level of each second subfile according to the file sequence number corresponding to each second subfile, and recombining according to the path levels to obtain a second original file consisting of a preset number of files; and if the target processing mode corresponding to the second subfile is a split processing mode, determining the path level of each second subfile according to the file sequence number corresponding to each second subfile, and performing merging processing according to the path levels to obtain a second original file.

The apparatus of the embodiment of the present application may execute the method provided by the embodiment of the present application, and the implementation principle is similar, the actions executed by the modules in the apparatus of the embodiments of the present application correspond to the steps in the method of the embodiments of the present application, and for the detailed functional description of the modules of the apparatus, reference may be specifically made to the description in the corresponding method shown in the foregoing, and details are not repeated here.

The embodiment of the application provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to realize the steps of the file transmission method, and compared with the related art, the method can realize the following steps: even if the files to be transmitted are massive small files or huge files, the files can be efficiently transmitted through the scheme.

In an alternative embodiment, an electronic device is provided, as shown in fig. 5, the electronic device 5000 shown in fig. 5 includes: a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are coupled, such as via a bus 5002. Optionally, the electronic device 5000 may further include a transceiver 5004, and the transceiver 5004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data. It should be noted that the transceiver 5004 is not limited to one in practical application, and the structure of the electronic device 5000 does not limit the embodiments of the present application.

The Processor 5001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 5001 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Bus 5002 may include a path that conveys information between the aforementioned components. The bus 5002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 5002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.

The Memory 5003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer, without limitation.

The memory 5003 is used for storing computer programs for executing the embodiments of the present application, and is controlled by the processor 5001 for execution. The processor 5001 is configured to execute computer programs stored in the memory 5003 to implement the steps shown in the foregoing method embodiments.

Wherein, the electronic device includes but is not limited to: and a server.

The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, can implement the steps of the foregoing method embodiments and corresponding content.

Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and claims of this application and in the preceding drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than illustrated or otherwise described herein.

It should be understood that, although each operation step is indicated by an arrow in the flowchart of the embodiment of the present application, the implementation order of the steps is not limited to the order indicated by the arrow. In some implementation scenarios of the embodiments of the present application, the implementation steps in the flowcharts may be performed in other sequences as desired, unless explicitly stated otherwise herein. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on an actual implementation scenario. Some or all of these sub-steps or stages may be performed at the same time, or each of these sub-steps or stages may be performed at different times, respectively. Under the scenario that the execution time is different, the execution sequence of the sub-steps or phases may be flexibly configured according to the requirement, which is not limited in the embodiment of the present application.

The foregoing is only an optional implementation manner of a part of implementation scenarios in this application, and it should be noted that, for those skilled in the art, other similar implementation means based on the technical idea of this application are also within the protection scope of the embodiments of this application without departing from the technical idea of this application.

Claims

1. A method of file transfer, the method comprising:

responding to a transmission instruction carrying transmission parameters, traversing a first shared directory configured by a first client to determine a first original file to be transmitted; the transmission parameters comprise the identification of the second client;

determining a target processing mode according to the storage space occupied by the first original file and/or the number of files contained in the first original file;

processing the first original file according to the target processing mode to obtain a first subfile to be transmitted; wherein the processing of the first original file comprises: splitting and merging;

searching online nodes configured by the second client through a registry, and determining the second nodes from the online nodes;

and transmitting the first subfile and the target processing mode to the second node.

2. The method according to claim 1, wherein the determining the target processing manner according to the storage space occupied by the first original file and/or the number of files included in the first original file comprises:

calculating the total number of files in the first original file and the total storage space occupied by the first original file;

if the total number is larger than a first threshold value, determining that the target processing mode is a combined processing mode;

and if the total number is less than or equal to the first threshold and if the total storage space is greater than a second threshold, determining that the target processing mode is a split processing mode.

3. The method of claim 2, wherein after the step of calculating the total number of files in the first original file and the total storage space occupied by the first original file, the method further comprises:

if the total storage space is larger than a second threshold value, determining that the target processing mode is a splitting processing mode;

and if the total storage space is smaller than or equal to the second threshold and if the total storage space is larger than the first threshold, determining that the target processing mode is a merging processing mode.

4. The method according to claim 3, wherein the processing the first original file according to the target processing manner to obtain a first subfile to be transmitted comprises:

if the target processing mode is a merging processing mode, merging the plurality of files contained in the first original file according to a preset unit file number to obtain the at least one first subfile; each first subfile is configured with a file sequence number corresponding to the merging processing sequence;

if the target processing mode is a splitting processing mode, splitting a plurality of files contained in the first original file according to a preset unit storage space to obtain at least one first subfile; each first subfile is configured with a file sequence number corresponding to the splitting processing sequence.

5. The method of claim 1, wherein the searching for an online node configured by the second client through the registry and determining the second node from the online nodes comprises:

sending a request carrying an identifier of a second client to the registry; the transmission parameter carries an identifier of the second client;

and responding to an online list corresponding to the second client fed back by the registry, and determining a corresponding node in the online list, the working state of which is an idle state, as the second node.

6. The method of claim 1, wherein prior to traversing the first shared directory configured by the first client, the method further comprises:

acquiring a first authentication result; the first authentication result represents whether the first node is configured with readable permission to a first shared directory;

acquiring a second authentication result according to the identifier of the second client; the second authentication result represents whether the first client configures the writable right of a second shared directory configured by the second client;

and traversing a first shared directory configured by the first client if the first authentication result is configured with readable permission and the second authentication result is configured with writable permission.

7. The method of claim 1, wherein if the second subfile sent by a third node is received, the third node is a node configured by a third client; the method further comprises the following steps:

if the corresponding target processing mode of the second subfiles is a merging processing mode, determining the path hierarchy of each second subfile according to the corresponding file sequence number of each second subfile, and recombining according to the path hierarchies to obtain a second original file consisting of files with preset number;

and if the target processing mode corresponding to the second subfile is a split processing mode, determining the path level of each second subfile according to the file sequence number corresponding to each second subfile, and performing merging processing according to the path levels to obtain the second original file.

8. A file transfer apparatus, characterized in that the apparatus comprises:

the scanning module is used for responding to a transmission instruction carrying transmission parameters, traversing a first shared directory configured by a first client to determine a first original file to be transmitted;

the first determining module is used for determining a target processing mode according to the storage space occupied by the first original file and/or the number of the files contained in the first original file;

the processing module is used for processing the first original file according to the target processing mode to obtain a first subfile to be transmitted; wherein the processing of the first original file comprises: splitting and merging;

the second determining module is used for searching online nodes configured by the second client through the registry and determining the second nodes from the online nodes;

a transceiver module, configured to transmit the first subfile and the target processing manner to the second node.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the method of any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1-7 when executed by a processor.