CN113824760B - File concurrency block uploading method and device, electronic equipment and storage medium - Google Patents
File concurrency block uploading method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113824760B CN113824760B CN202110886685.4A CN202110886685A CN113824760B CN 113824760 B CN113824760 B CN 113824760B CN 202110886685 A CN202110886685 A CN 202110886685A CN 113824760 B CN113824760 B CN 113824760B
- Authority
- CN
- China
- Prior art keywords
- uploading
- file
- uploaded
- threads
- concurrent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 121
- 230000005540 biological transmission Effects 0.000 claims description 56
- 238000012545 processing Methods 0.000 claims description 21
- 230000000903 blocking effect Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 125000004122 cyclic group Chemical group 0.000 claims description 8
- 238000000638 solvent extraction Methods 0.000 claims description 5
- 238000005192 partition Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000007906 compression Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001105 regulatory effect Effects 0.000 description 2
- 238000013144 data compression Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a file concurrency block uploading method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; acquiring file block information to be uploaded of a file to be uploaded according to a file identifier in the meta information; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which are not completely transmitted; calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to a client; receiving file block data sent by an uploading thread of a client according to file block information to be uploaded, and updating historical uploading records according to the file block data; and repeatedly executing the process until all the file blocks to be uploaded are uploaded. According to the file concurrency block uploading method, device, electronic equipment and storage medium provided by the embodiment of the invention, the file concurrency high-efficiency uploading is realized through the real-time reasonable distribution of the file uploading threads.
Description
Technical Field
The present invention relates to the field of data transmission technologies, and in particular, to a method and apparatus for uploading concurrent blocks of a file, an electronic device, and a storage medium.
Background
With the development of computer software and hardware technology in recent years, the informatization level of each industry is greatly improved, a large number of data files are created, and each file stores a large amount of data, so how to efficiently transmit the data dispersed on each terminal to a server becomes a worth discussing problem. Due to the large demand for file transmission, a scenario that a plurality of clients transmit large files concurrently under the condition of limited bandwidth is often encountered.
The method generally used in the industry mainly comprises the steps of carrying out block concurrent transmission on large files, however, the method cannot work well under the conditions of small transmission bandwidth and jitter of the transmission bandwidth, the situation that the transmission thread of some files preempts most of server bandwidth resources to cause that the transmission speed of other files is particularly slow or even the transmission of the network is failed due to overtime is frequently caused, and the whole file needs to be retransmitted once the transmission is interrupted due to faults in the transmission process of the conventional transmission method.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a method, a device, electronic equipment and a storage medium for uploading concurrent blocks of files.
The invention provides a file concurrency block uploading method, which is applied to a server and comprises the following steps: the receiving flow of the uploading request comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification;
the file block information to be uploaded is obtained by: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
and (3) uploading a thread allocation flow: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client;
and (3) uploading a data processing flow: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data;
the first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
According to the file concurrency block uploading method provided by the invention, the historical uploading record is updated according to the file block data, and the method comprises the following steps: checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block has completed uploading; if the verification is unsuccessful, discarding the corresponding file block; and/or, when the uploading interrupt of the file block occurs, saving the data information which is partially uploaded, and recording the number of bytes which are uploaded.
According to the file concurrency block uploading method provided by the invention, the calculation of the allowable concurrency uploading thread number comprises the following steps: acquiring the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the allowable concurrency uploading thread number of the client; if the number of the allowed concurrent uploading threads is larger than the maximum number of the allowed concurrent threads of the client, the number of the allowed concurrent uploading threads is adjusted to be the maximum number of the allowed concurrent threads of the client; if the number of the allowed concurrent uploading threads is larger than the current available thread number, the number of the allowed concurrent uploading threads is adjusted to be the current available thread number; and if the number of the allowed concurrent uploading threads is smaller than 1, adjusting the number of the allowed concurrent uploading threads to 1.
According to the file concurrency block uploading method provided by the invention, the method further comprises the steps of calculating a second product of a preset second proportion and a memory for file transmission, and taking the ratio of the second product to the maximum concurrency number of the system as the block size; and partitioning the file to be uploaded according to the partition size.
The invention also provides a file concurrency block uploading method, which is applied to the client and comprises the following steps: the uploading request sending flow comprises the following steps: reading meta information of a file to be uploaded; wherein the meta information includes a file identification; file block to be uploaded and uploading thread information receiving flow: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) data uploading flow: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded; the second loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The invention also provides a file concurrency block uploading device, which is applied to a server and comprises:
an upload request receiving module, configured to: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification;
the file block information obtaining module to be uploaded is used for: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
an uploading thread allocation module, configured to: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client;
the uploading data processing module is used for: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data;
a first loop execution module for: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the file concurrency block uploading method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a file concurrency blocking upload method as described in any one of the above.
According to the file concurrency block uploading method, device, electronic equipment and storage medium, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is realized, the quick and efficient uploading process of the file is realized based on the method, the overall uploading efficiency of the file is improved, and resource conflict is avoided; meanwhile, the breakpoint continuous transmission of the file is realized by uploading based on the existing length of the file block which is not completely transmitted.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for uploading concurrent blocks of a file according to the present invention;
FIG. 2 is a second flowchart of a method for concurrent block upload of files according to the present invention;
FIG. 3 is a schematic flow chart of a method for uploading concurrent blocks of files applied to a server;
FIG. 4 is a schematic flow chart of a concurrent block uploading method of a file applied to a client;
FIG. 5 is a schematic diagram of a file concurrency block uploading device provided by the invention;
fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic flow chart of a method for uploading concurrent blocks of a file, as shown in fig. 1, where the method is applied to a server, and includes:
S110, an uploading request receiving flow: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification;
s120, a file block information acquisition process to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
s130, uploading thread allocation flow: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client;
s140, uploading a data processing flow: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data;
s150, a first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Meta information indicates system data for describing characteristics of one file, such as access rights, file size, file owner, distribution information of file data, etc., which must be obtained when operating one file in order to locate the position of the file and obtain the contents or related attributes of the file.
The client reads the file to be uploaded from the local file system, acquires meta-information of the file, and specifically may include a name of the file (hereinafter referred to as a file name), an identifier set by a user for the file (hereinafter referred to as a file identifier), and a size of the file is sent to the server through an HTTP protocol.
After receiving the HTTP request, the server reads the file name, file identifier, and file size from it. The server uses a file identifier to inquire whether the file is uploaded from a history transmission record file, if the history transmission record does not contain the file, the file is regarded as first transmission, then the file block size set in a configuration file by a user is read, then the file size is divided by the file block size, the number of file blocks needing to be transmitted together and the start-stop offset of each file block are calculated, and the information is written into the history transmission record; if the file is contained in the history transmission record file, the file block size, the initial offset of each file block, the already transmitted length of each file block and whether each file block is transmitted or not are directly read from the history transmission record file.
The file block offset refers to the number of bytes for moving the file to be uploaded forwards or backwards from a designated position, and the file block offset is used for describing that the file block which is completed to be transmitted is positioned at a specific position of the file to be uploaded, namely, describing a blocking mode of the file to be uploaded; meanwhile, the transmission progress of a certain file block which is not transmitted is described by the number of bytes transmitted.
The server calculates the number N of the allowable concurrent uploading threads according to the current server load, and simultaneously sends a calculation result and file block information to be uploaded to the client; the client starts corresponding uploading threads according to the received allowable concurrent uploading thread number N, and performs a parallel uploading process on the file blocks depending on the started threads; the server receives the file block data from the client and updates the history to upload and record according to the actual file block receiving condition.
Taking the historical uploading record as a judgment basis, and repeatedly executing the uploading process of the file when the file to be uploaded is not uploaded; and stopping the file uploading process when the file to be uploaded is uploaded.
According to the file concurrency block uploading method, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is realized, the quick and efficient uploading process of the file is realized based on the method, the overall uploading efficiency of the file is improved, and resource conflict is avoided; uploading is carried out based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the historical uploading record of the file block is updated in real time, so that the reliability of the file concurrent block uploading process is ensured, and the problem of error transmission and missing transmission is avoided.
According to the method for uploading concurrent blocks of a file provided by the invention, in the invention, the method for uploading records according to the file block data update history comprises the following steps: checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block has completed uploading; if the verification is unsuccessful, discarding the corresponding file block; and/or, when the uploading interrupt of the file block occurs, saving the data information which is partially uploaded, and recording the number of bytes which are uploaded.
And compressing the data by using a data compression mode appointed by the server, calculating an MD5 check code of the data, and then using a POST request in an HTTP protocol, taking the MD5 check code and the length of the data as request heads, and taking the part of the file block which is not transmitted as a request body, and transmitting the request body to the server.
After receiving the file block transmission request, the server reads the MD5 check code, the data length and the part of the file block which is not transmitted. If the length reaches the length of the appointed file block, calculating an MD5 check code of the data, if the MD5 check code is consistent with the MD5 check code transmitted by the user, updating the length of the corresponding file block of the appointed file in the history transmission record as the transmission is correct, and identifying the corresponding file block as the transmission is completed, otherwise, discarding the transmission result as the data transmission is incorrect, deleting the uploaded part of the appointed file block, and updating the length of the corresponding file block of the appointed file in the history transmission record as 0. After the server processes the request, the number of threads which are being concurrently transmitted by all clients of the current server is reduced by one.
The server reads the uploaded part of the designated file block, and the data transmitted at this time is spliced, if the total length does not reach the length of the appointed file block, the file block is not transmitted yet, the file block is directly stored, the length of the corresponding file block of the designated file in the history transmission record is updated, and the record is uploaded according to the updated history, so that the file can be continuously uploaded from the uploading interruption position after the system is recovered to be normal.
According to the file concurrency block uploading method provided by the invention, the accuracy of the uploaded file blocks is judged through the cyclic redundancy check code, and the historical uploading record is updated after the judgment is finished, so that the reliability of file uploading is ensured; meanwhile, when the uploading interruption of the file block occurs, the data information which is partially uploaded is stored, and the history uploading record is updated, so that the file uploading can be continued from the uploading interruption position after the system is recovered to be normal, and the effectiveness of the file uploading is ensured.
According to the file concurrency block uploading method provided by the invention, the calculation of the allowable concurrency uploading thread number comprises the following steps: acquiring the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the allowable concurrency uploading thread number of the client; if the number of the allowed concurrent uploading threads is larger than the maximum number of the allowed concurrent threads of the client, the number of the allowed concurrent uploading threads is adjusted to be the maximum number of the allowed concurrent threads of the client; if the number of the allowed concurrent uploading threads is larger than the current available thread number, the number of the allowed concurrent uploading threads is adjusted to be the current available thread number; and if the number of the allowed concurrent uploading threads is smaller than 1, adjusting the number of the allowed concurrent uploading threads to 1.
The server reads a configuration file provided by a user and acquires the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; it should be noted that, the sum of the number of concurrent threads includes both the thread being uploaded and the thread being downloaded; the server multiplies the maximum concurrency number of the system by a preset first proportion to obtain a first product, wherein the first proportion is a preset value, the first product can be preset according to the requirement, the ratio of the first product to the number of files being uploaded and downloaded by the current client is used as the number of allowable concurrent uploading threads of the current client, and the first product can be 0.9 for example.
The number of the allowable concurrent uploading threads of the current client obtained by the calculation is only a theoretical value, after the allowable concurrent uploading threads are obtained by the calculation, a judgment step is needed to be carried out, and the final allowable concurrent uploading threads are obtained based on the judgment result, wherein the specific judgment process is as follows:
(1) if the number of the allowed concurrent uploading threads obtained through calculation is larger than the maximum allowed concurrent threads of the client, the fact that the current file to be uploaded has the requirement for the uploading threads exceeding the maximum capability of the current client is indicated, and the allowed concurrent uploading threads are adjusted to the maximum allowed concurrent threads of the client; (2) if the number of the allowable concurrent uploading threads obtained through calculation is larger than the current available thread number, the fact that the requirement of the current file to be uploaded on the uploading threads exceeds the maximum capability of the current system is indicated, and the allowable concurrent uploading thread number is adjusted to be the current available thread number; (3) if the number of the allowable concurrent uploading threads obtained through calculation is smaller than 1, the fact that the current file to be uploaded has smaller requirements for the uploading threads is indicated, but in order to ensure that the uploading process is normally carried out, the allowable concurrent uploading threads are adjusted to be 1.
According to the file concurrency block uploading method provided by the invention, the current occupied thread number of other clients is subtracted by utilizing the maximum concurrency thread number of the system, the range of the current available thread number is locked, and meanwhile, the number of actually used threads is flexibly adjusted according to actual requirements by presetting a coefficient of a first proportion, so that the balanced utilization of uploading threads is realized; meanwhile, the number of the allowed concurrent uploading threads obtained through calculation is respectively judged with the maximum allowed concurrent threads of the client, the current available threads and the like, the final allowed concurrent uploading threads are obtained based on the judging result, the calculated allowed concurrent uploading threads can be better matched with the current actual uploading capacity of the server or the client based on the judging step, and efficient transmission of the file to be uploaded is ensured on the premise that the server and the client exert the maximum uploading capacity.
According to the file concurrency block uploading method provided by the invention, the method further comprises the following steps: calculating a second product of a preset second proportion and a memory for file transmission, and taking the ratio of the second product to the maximum concurrency number of the system as the block size; and partitioning the file to be uploaded according to the partition size.
It should be noted that, the memory of the server is a fixed value, and when the system is running, the memory of the server is divided into two parts: the occupied memory and the residual memory are too large, so that the running speed of the system is low, and the utilization rate of the server is lower. In the invention, the memory used for file transmission can be the residual memory or can be a set value, the preset second proportion is multiplied by the memory used for file transmission to obtain a second product, meanwhile, the specific partitioning standard of the file block is used as the ratio of the second product to the maximum concurrence number of the system, the file to be uploaded is partitioned, and the value of the preset second proportion can be preset according to actual needs and is not a fixed value. The preset second ratio may take a value of 0.9.
According to the file concurrency block uploading method provided by the invention, the memory uploaded by the final file is regulated and controlled by utilizing the preset second proportion, so that the balanced processing on the two aspects of the running speed of the server and the utilization rate of the server is realized, and the running capacity of the server is exerted to the greatest extent on the premise of meeting the file uploading speed; meanwhile, the adjustability of the second proportion is preset, flexible adjustment of the running capability of the server is achieved, and the difference requirement of priority of file block uploading speed or priority of file block uploading quantity can be met.
Fig. 2 is a second flowchart of a method for uploading concurrent blocks of a file according to the present invention, as shown in fig. 2, where the method is applied to a client, and includes:
s210, an uploading request sending flow: reading meta information of a file to be uploaded; wherein the meta information includes a file identification;
s220, a file block to be uploaded and an uploading thread information receiving process: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
s230, data uploading flow: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded;
s240, a second loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
It should be noted that, the client reads meta information of the file to be uploaded and sends the meta information to the server, the server uses a file identifier in the meta information as an acquisition basis to obtain file block information to be uploaded of the file to be uploaded, the file block information to be uploaded includes a start offset of a file block to be uploaded and a number of bytes transmitted of the file block which is not completely transmitted, and a blocking mode of the file to be uploaded is described by using the file block offset; meanwhile, the transmission progress of a certain file block which is not transmitted is described by the number of bytes transmitted.
Meanwhile, the server calculates the allowable concurrent uploading thread number based on the load of the server, and sends the obtained file block information to be uploaded and the allowable concurrent uploading thread number to the client, the client starts the corresponding number of uploading threads according to the allowable concurrent uploading thread number, and meanwhile, the file block information to be uploaded is used as the basis of file uploading, and the uploading threads are used as transmission paths to send file block data to the server.
Taking the historical uploading record as a judgment basis, and repeatedly executing the uploading process of the file when the file to be uploaded is not uploaded; and stopping the file uploading process when the file to be uploaded is uploaded.
According to the file concurrency block uploading method, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is realized, the quick and efficient uploading process of the file is realized based on the method, the overall uploading efficiency of the file is improved, and resource conflict is avoided; uploading is carried out based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the historical uploading record of the file block is updated in real time, so that the reliability of the file concurrent block uploading process is ensured, and the problem of error transmission and missing transmission is avoided.
Fig. 3 is a flow chart of a method for uploading concurrent blocks of a file applied to a server, which is provided by the invention, as shown in fig. 3, and the method includes:
step1, starting a process, searching a history transmission record of a corresponding file, and searching the history transmission record of the corresponding file by a server based on a file identifier;
step2, calculating a transmission plan and a concurrent thread number, wherein the transmission plan comprises a compression mode of a file block and calculation of a cyclic redundancy check code, and the concurrent thread number represents the allowable concurrent uploading thread number;
step3, receiving file block data, and judging whether the current file block is transmitted completely;
step4, if the current file block is not transmitted, updating the byte number transmitted by the current file block; and if the current file block is transmitted completely, verifying the check code, and updating the file block data and the meta information based on the verification result.
Step5, ending the flow.
According to the file concurrency block uploading method, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is achieved, the rapid and efficient uploading process of the file is achieved based on the method, the overall uploading efficiency of the file is improved, and resource conflict is avoided.
Fig. 4 is a flow chart of a method for file concurrency block upload applied to a client, as shown in fig. 4, provided by the invention, the method includes:
step1, starting a process, and transmitting file meta information to a server, wherein the file meta information comprises file identifications used for distinguishing different files;
step2, acquiring a transmission plan and a concurrent thread number, wherein the transmission plan comprises a compression mode of a file block and calculation of a cyclic redundancy check code, and the concurrent thread number represents the allowable concurrent uploading thread number;
step3, starting a plurality of transmission threads, distributing a file block uploading task for each thread, wherein each uploading task comprises compressing a file block, calculating a check code, and uploading the compressed file block and the obtained check code;
step4, judging whether all files are transmitted completely, and if not, executing Step2 and Step3 again; if so, the process ends.
According to the file concurrency block uploading method, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is achieved, the rapid and efficient uploading process of the file is achieved based on the method, the overall uploading efficiency of the file is improved, and resource conflict is avoided.
Fig. 5 is a schematic structural diagram of a file concurrency block uploading device provided by the present invention, where the device is applied to a server, as shown in fig. 5, and the device includes: an upload request receiving module 510, a file block to be uploaded information obtaining module 520, an upload thread allocating module 530, an upload data processing module 540 and a first loop executing module 550, wherein:
an upload request receiving module 510, configured to: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification;
the file block information to be uploaded acquisition module 520 is configured to: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
an upload thread allocation module 530 for: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client;
an upload data processing module 540 for: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data;
A first loop execution module 550 for: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
According to the file concurrency block uploading device, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is realized, the quick and efficient uploading process of the file is realized based on the concurrent uploading process, the overall uploading efficiency of the file is improved, and resource conflict is avoided; uploading is carried out based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the historical uploading record of the file block is updated in real time, so that the reliability of the file concurrent block uploading process is ensured, and the problem of error transmission and missing transmission is avoided.
According to the file concurrency block uploading device provided by the invention, when the uploading data processing module 540 is used for updating historical uploading records according to the file block data, the uploading data processing module is specifically used for: checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block has completed uploading; if the verification is unsuccessful, discarding the corresponding file block; and/or, when the uploading interrupt of the file block occurs, saving the data information which is partially uploaded, and recording the number of bytes which are uploaded.
The file concurrency block uploading device provided by the invention judges the accuracy of the uploaded file blocks through the cyclic redundancy check code, updates the historical uploading record after the judgment is finished, and ensures the reliability of file uploading based on the judgment; meanwhile, when the uploading of the file blocks is interrupted, the data information which is partially uploaded is stored, and the history uploading record is updated, so that the file uploading can be continued from the uploading interruption position after the system is recovered to be normal, the uploading process has a breakpoint continuous uploading function, and the effectiveness of the file uploading is ensured.
According to the file concurrency block uploading device provided by the invention, the uploading thread allocation module 530 is specifically configured to: acquiring the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the allowable concurrency uploading thread number of the client; if the number of the allowed concurrent uploading threads is larger than the maximum number of the allowed concurrent threads of the client, the number of the allowed concurrent uploading threads is adjusted to be the maximum number of the allowed concurrent threads of the client; if the number of the allowed concurrent uploading threads is larger than the current available thread number, the number of the allowed concurrent uploading threads is adjusted to be the current available thread number; and if the number of the allowed concurrent uploading threads is smaller than 1, adjusting the number of the allowed concurrent uploading threads to 1.
The file concurrency block uploading device provided by the invention locks the range of the current available thread number by subtracting the current occupied thread number of other clients from the maximum concurrency thread number of the system, and flexibly adjusts the actually used thread number according to actual requirements by presetting a coefficient of a first proportion so as to realize balanced utilization of uploading threads; meanwhile, the number of the allowed concurrent uploading threads obtained through calculation is respectively judged with the maximum allowed concurrent threads of the client, the current available threads and the like, the final allowed concurrent uploading threads are obtained based on the judging result, the calculated allowed concurrent uploading threads can be better matched with the current actual uploading capacity of the server or the client based on the judging step, and efficient transmission of the file to be uploaded is ensured on the premise that the server and the client exert the maximum uploading capacity.
According to the file concurrency block uploading device provided by the invention, the device 500 is further used for calculating a second product of a preset second proportion and a memory for file transmission, and taking the ratio of the second product to the maximum concurrency number of the system as the block size; and partitioning the file to be uploaded according to the partition size.
According to the file concurrency block uploading device, the utilization memory of final file uploading is regulated and controlled by utilizing the preset second proportion, so that balanced processing on the two aspects of the running speed of the server and the utilization rate of the server is realized, and the running capacity of the server is exerted to the greatest extent on the premise that the file uploading speed is met; meanwhile, the adjustability of the second proportion is preset, flexible adjustment of the running capability of the server is achieved, and the difference requirements of the priority of the uploading speed of the file blocks or the priority of the uploading quantity of the file blocks are met respectively.
The invention also provides a file concurrency block uploading device, which is applied to the client, and comprises: the system comprises an uploading request sending module, a file block to be uploaded, an uploading thread information receiving module, a data uploading module and a second cycle executing module, wherein:
an upload request sending module, configured to: reading meta information of a file to be uploaded; wherein the meta information includes a file identification;
the file block to be uploaded and the uploading thread information receiving module are used for: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
The data uploading module is used for: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded;
a second loop execution module for: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
According to the file concurrency block uploading device, the server calculates the allowable concurrency uploading thread number, meanwhile, the block processing is carried out on the file to be uploaded, the specific object for concurrency uploading is confirmed, the client finally starts a plurality of uploading threads based on the allowable concurrency uploading thread number, the concurrency uploading process of a plurality of file blocks of the file to be uploaded is realized, the quick and efficient uploading process of the file is realized based on the concurrent uploading process, the overall uploading efficiency of the file is improved, and resource conflict is avoided; uploading is carried out based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the historical uploading record of the file block is updated in real time, so that the reliability of the file concurrent block uploading process is ensured, and the problem of error transmission and missing transmission is avoided.
Fig. 6 illustrates a physical schematic diagram of an electronic device, as shown in fig. 6, which may include: processor 610, communication interface (Communications Interface) 620, memory 630, and communication bus 640, wherein processor 610, communication interface 620, and memory 630 communicate with each other via communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a file concurrent block upload method comprising: the receiving flow of the uploading request comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification; the file block information to be uploaded is obtained by: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) uploading a thread allocation flow: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client; and (3) uploading a data processing flow: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data; the first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Or the file concurrency block uploading method comprises the following steps: the uploading request sending flow comprises the following steps: reading meta information of a file to be uploaded; wherein the meta information includes a file identification; file block to be uploaded and uploading thread information receiving flow: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) data uploading flow: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded; the second loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Further, the logic instructions in the memory 630 may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing a method of file concurrency block upload provided by the above methods, the method of file concurrency block upload comprising: the receiving flow of the uploading request comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification; the file block information to be uploaded is obtained by: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) uploading a thread allocation flow: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client; and (3) uploading a data processing flow: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data; the first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Or the file concurrency block uploading method comprises the following steps: the uploading request sending flow comprises the following steps: reading meta information of a file to be uploaded; wherein the meta information includes a file identification; file block to be uploaded and uploading thread information receiving flow: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) data uploading flow: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded; the second loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
In still another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, is implemented to perform the above-provided file concurrency blocking uploading method, the file concurrency blocking uploading method including: the receiving flow of the uploading request comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification; the file block information to be uploaded is obtained by: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) uploading a thread allocation flow: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client; and (3) uploading a data processing flow: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data; the first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Or the file concurrency block uploading method comprises the following steps: the uploading request sending flow comprises the following steps: reading meta information of a file to be uploaded; wherein the meta information includes a file identification; file block to be uploaded and uploading thread information receiving flow: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted; and (3) data uploading flow: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded; the second loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. A method for file concurrency block uploading, the method being applied to a server, comprising:
the receiving flow of the uploading request comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification;
the file block information to be uploaded is obtained by: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
and (3) uploading a thread allocation flow: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client;
wherein the calculating the number of allowed concurrent uploading threads includes:
acquiring the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client;
subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number;
calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the allowable concurrency uploading thread number of the client;
If the number of the allowed concurrent uploading threads is larger than the maximum number of the allowed concurrent threads of the client, the number of the allowed concurrent uploading threads is adjusted to be the maximum number of the allowed concurrent threads of the client;
if the number of the allowed concurrent uploading threads is larger than the current available thread number, the number of the allowed concurrent uploading threads is adjusted to be the current available thread number;
if the number of the allowed concurrent uploading threads is smaller than 1, the number of the allowed concurrent uploading threads is adjusted to be 1;
and (3) uploading a data processing flow: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data;
the first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
2. The method for uploading concurrent blocks of a file according to claim 1, wherein updating the history upload record according to the file block data comprises:
checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block has completed uploading; if the verification is unsuccessful, discarding the corresponding file block;
And/or, when the uploading interrupt of the file block occurs, saving the data information which is partially uploaded, and recording the number of bytes which are uploaded.
3. The method of file concurrency blocking uploading of claim 1, further comprising:
calculating a second product of a preset second proportion and a memory for file transmission, and taking the ratio of the second product to the maximum concurrency number of the system as the block size; and partitioning the file to be uploaded according to the partition size.
4. A method for file concurrency block uploading, the method being applied to a client, the method comprising:
the uploading request sending flow comprises the following steps: reading meta information of a file to be uploaded; wherein the meta information includes a file identification;
file block to be uploaded and uploading thread information receiving flow: receiving file block information to be uploaded sent by a server and allowing the concurrent uploading thread number; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
and (3) data uploading flow: starting an uploading thread according to the number of the allowed concurrent uploading threads, and sending file block data to the server according to the file block information to be uploaded;
The method for calculating the number of the allowed concurrent uploading threads comprises the following steps:
acquiring the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client;
subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number;
calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the allowable concurrency uploading thread number of the client;
if the number of the allowed concurrent uploading threads is larger than the maximum number of the allowed concurrent threads of the client, the number of the allowed concurrent uploading threads is adjusted to be the maximum number of the allowed concurrent threads of the client;
if the number of the allowed concurrent uploading threads is larger than the current available thread number, the number of the allowed concurrent uploading threads is adjusted to be the current available thread number;
if the number of the allowed concurrent uploading threads is smaller than 1, the number of the allowed concurrent uploading threads is adjusted to be 1;
the second loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
5. A file concurrency blocking uploading device, the device being applied to a server, comprising:
An upload request receiving module, configured to: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information includes a file identification;
the file block information obtaining module to be uploaded is used for: acquiring file block information to be uploaded of the file to be uploaded according to the file identifier; the file block information to be uploaded comprises a starting offset of a file block to be uploaded and the number of transmitted bytes of the file block which is not completely transmitted;
an uploading thread allocation module, configured to: calculating the number of allowed concurrent uploading threads, and sending the file block information to be uploaded and the number of allowed concurrent uploading threads to the client;
wherein the calculating the number of allowed concurrent uploading threads includes:
acquiring the maximum concurrency number of the system and the maximum concurrency thread number allowed by the client;
subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number;
calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the allowable concurrency uploading thread number of the client;
If the number of the allowed concurrent uploading threads is larger than the maximum number of the allowed concurrent threads of the client, the number of the allowed concurrent uploading threads is adjusted to be the maximum number of the allowed concurrent threads of the client;
if the number of the allowed concurrent uploading threads is larger than the current available thread number, the number of the allowed concurrent uploading threads is adjusted to be the current available thread number;
if the number of the allowed concurrent uploading threads is smaller than 1, the number of the allowed concurrent uploading threads is adjusted to be 1;
the uploading data processing module is used for: receiving file block data sent by an uploading thread of the client according to the file block information to be uploaded, and updating historical uploading records according to the file block data;
a first loop execution module for: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor performs the steps of the file concurrency blocking upload method according to any one of claims 1 to 4 when the program is executed.
7. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the file concurrency blocking uploading method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110886685.4A CN113824760B (en) | 2021-08-03 | 2021-08-03 | File concurrency block uploading method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110886685.4A CN113824760B (en) | 2021-08-03 | 2021-08-03 | File concurrency block uploading method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113824760A CN113824760A (en) | 2021-12-21 |
| CN113824760B true CN113824760B (en) | 2024-01-30 |
Family
ID=78924217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110886685.4A Active CN113824760B (en) | 2021-08-03 | 2021-08-03 | File concurrency block uploading method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113824760B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114285838A (en) * | 2021-12-22 | 2022-04-05 | 中国农业银行股份有限公司 | File uploading method, device, system, storage medium and electronic equipment |
| CN114598697B (en) * | 2022-03-29 | 2024-06-07 | 浪潮云信息技术股份公司 | Method and device for transmitting file and computer readable storage medium |
| CN115617760B (en) * | 2022-12-20 | 2023-04-18 | 中化现代农业有限公司 | Method for quickly uploading large-batch files |
| CN117155922B (en) * | 2023-10-31 | 2024-01-30 | 国家超级计算天津中心 | File transmission method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103442037A (en) * | 2013-08-09 | 2013-12-11 | 华南理工大学 | Method for achieving multithreading breakpoint upload of oversized file based on FTP |
| WO2014056428A1 (en) * | 2012-10-12 | 2014-04-17 | 腾讯科技（深圳）有限公司 | File uploading method and system |
| CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
| CN108093063A (en) * | 2017-12-26 | 2018-05-29 | 中国电信股份有限公司新疆分公司 | Big file multithreading FTP method for uploading |
| CN109547566A (en) * | 2018-12-25 | 2019-03-29 | 华南理工大学 | A kind of multithreading upload optimization method distributed based on memory |
| CN109756568A (en) * | 2018-12-29 | 2019-05-14 | 上海掌门科技有限公司 | Processing method, equipment and the computer readable storage medium of file |
| WO2020001488A1 (en) * | 2018-06-28 | 2020-01-02 | 杭州海康威视系统技术有限公司 | File download method and apparatus, client and computer readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9197702B2 (en) * | 2013-12-06 | 2015-11-24 | Cellco Partnership | System for and method for media upload multithreading for large file uploads |
-
2021
- 2021-08-03 CN CN202110886685.4A patent/CN113824760B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014056428A1 (en) * | 2012-10-12 | 2014-04-17 | 腾讯科技（深圳）有限公司 | File uploading method and system |
| CN103442037A (en) * | 2013-08-09 | 2013-12-11 | 华南理工大学 | Method for achieving multithreading breakpoint upload of oversized file based on FTP |
| CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
| CN108093063A (en) * | 2017-12-26 | 2018-05-29 | 中国电信股份有限公司新疆分公司 | Big file multithreading FTP method for uploading |
| WO2020001488A1 (en) * | 2018-06-28 | 2020-01-02 | 杭州海康威视系统技术有限公司 | File download method and apparatus, client and computer readable storage medium |
| CN109547566A (en) * | 2018-12-25 | 2019-03-29 | 华南理工大学 | A kind of multithreading upload optimization method distributed based on memory |
| CN109756568A (en) * | 2018-12-29 | 2019-05-14 | 上海掌门科技有限公司 | Processing method, equipment and the computer readable storage medium of file |
Non-Patent Citations (2)
| Title |
|---|
| 基于Web的大文件高效上传方法;阮晓龙，李朋楠;《 计算机系统应用》(第2020年第3期期);第234-239页 * |
| 断点续传及多线程机制在远程传版中的应用研究;黎苑文;程明智;徐秀花;杨义先;;北京印刷学院学报(06);第53-56页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113824760A (en) | 2021-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113824760B (en) | File concurrency block uploading method and device, electronic equipment and storage medium | |
| CN106170968B (en) | A kind of data compression storage method, device and distributed file system | |
| CN109683930B (en) | Air conditioning equipment program upgrading method, device and system and household electrical appliance | |
| CN112000355A (en) | Method and device for firmware upgrading and household appliance | |
| CN110535776B (en) | Gateway current limiting method, device, gateway, system and storage medium | |
| US20170085340A1 (en) | Method and apparatus for data communication of vehicle | |
| CN111490947A (en) | Data packet transmitting method, data packet receiving method, system, device and medium | |
| CN105450682A (en) | Method, device, and system for data synchronous storage and synchronizing data to client | |
| CN111930406A (en) | Vehicle-mounted terminal micro control unit software upgrading method, electronic equipment and storage medium | |
| CN111752803A (en) | Method, device and medium for collecting and reporting buried point data | |
| CN109614131A (en) | A kind of broadcasting equipment upgrading control system | |
| CN109936609B (en) | Terminal chain type upgrading method and device and upgrading management server | |
| CN107707395B (en) | Data transmission method, device and system | |
| CN111651419B (en) | Data storage method, device and storage medium | |
| CN106790653B (en) | File transmission processing method and device | |
| CN116366629A (en) | File uploading and acquiring method, computer equipment and storage medium | |
| CN112685145A (en) | Task management method and device and computer equipment | |
| CN110430279B (en) | File downloading control method and device | |
| CN112395132B (en) | File backup method, system and computer storage medium | |
| CN111221556A (en) | Method and system for remotely upgrading RSU mirror image | |
| CN113297145B (en) | Migration report generation method and device, electronic equipment and computer storage medium | |
| CN112055849A (en) | Exclusive control system and exclusive control method | |
| CN114710333B (en) | Method, system, computer device and storage medium for data transmission and verification | |
| CN113986606B (en) | Elevator program upgrading method, system, device and storage medium | |
| CN116527539B (en) | Data consistency verification method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |