CN113824760A - File concurrent block uploading method and device, electronic equipment and storage medium - Google Patents
File concurrent block uploading method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113824760A CN113824760A CN202110886685.4A CN202110886685A CN113824760A CN 113824760 A CN113824760 A CN 113824760A CN 202110886685 A CN202110886685 A CN 202110886685A CN 113824760 A CN113824760 A CN 113824760A
- Authority
- CN
- China
- Prior art keywords
- file
- uploading
- uploaded
- block
- concurrent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 140
- 230000005540 biological transmission Effects 0.000 claims description 58
- 238000012545 processing Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 9
- 125000004122 cyclic group Chemical group 0.000 claims description 8
- 230000000903 blocking effect Effects 0.000 claims description 5
- 238000005192 partition Methods 0.000 claims description 4
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000007906 compression Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001105 regulatory effect Effects 0.000 description 2
- 238000013144 data compression Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
Abstract
The invention provides a method and a device for uploading files in a concurrent and blocked manner, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving meta information of a file to be uploaded, which is sent by a client; acquiring file block information to be uploaded of a file to be uploaded according to the file identifier in the meta information; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; calculating the number of uploading threads allowed to be concurrent, and sending the information of the file blocks to be uploaded and the number of uploading threads allowed to be concurrent to the client; receiving file block data sent by an uploading thread of a client according to file block information to be uploaded, and updating a historical uploading record according to the file block data; and repeatedly executing the process until all the file blocks to be uploaded are uploaded. According to the file concurrent block uploading method and device, the electronic equipment and the storage medium, the file concurrent efficient uploading is achieved through real-time reasonable distribution of the file uploading threads.
Description
Technical Field
The invention relates to the technical field of data transmission, in particular to a method and a device for uploading files in a concurrent and blocked manner, electronic equipment and a storage medium.
Background
With the rapid development of computer software and hardware technology in recent years, the informatization level of various industries is greatly improved, a large number of data files are generated, and each file stores a large amount of data, so that the problem of how to efficiently transmit data scattered on each terminal to a server becomes a worth of discussion. Due to the large demand for file transmission at present, a scenario that a plurality of clients concurrently transmit a large file under a limited bandwidth condition is often encountered.
The existing method commonly used in the industry is mainly to transmit large files in a block and concurrently, however, the method cannot work well under the conditions of small transmission bandwidth and transmission bandwidth jitter, a scene that transmission threads of some files occupy most of server bandwidth resources to cause extremely slow transmission speed of other files and even network overtime transmission failure often occurs, and once transmission interruption occurs due to failure in the transmission process of the existing transmission method, the whole file needs to be retransmitted.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a method and a device for uploading files in a concurrent and block-by-block manner, electronic equipment and a storage medium.
The invention provides a method for uploading files in a concurrent and blocked manner, which is applied to a server and comprises the following steps: an upload request receiving process: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier;
obtaining information of file blocks to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
uploading thread distribution flow: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client;
uploading data and processing flow: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data;
the first loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
According to the file concurrent block uploading method provided by the invention, the updating of the historical uploading record according to the file block data comprises the following steps: checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block is uploaded; if the verification is unsuccessful, discarding the corresponding file block; and/or when the uploading of the file blocks is interrupted, storing the partially uploaded data information and recording the number of bytes uploaded.
According to the file concurrent block uploading method provided by the invention, the calculation of the number of uploading threads allowing concurrency comprises the following steps: acquiring the maximum concurrency number of a system and the maximum concurrent thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the number of uploading threads allowing concurrency of the client; if the number of the uploading threads allowed to be transmitted concurrently is larger than the maximum number of the uploading threads allowed by the client, adjusting the number of the uploading threads allowed to be transmitted concurrently to the maximum number of the uploading threads allowed by the client; if the number of the uploading threads allowed to be concurrent is larger than the current available thread number, adjusting the number of the uploading threads allowed to be concurrent to the current available thread number; and if the number of the uploading threads allowing the concurrency is less than 1, adjusting the number of the uploading threads allowing the concurrency to be 1.
According to the file concurrent block uploading method provided by the invention, the method further comprises the steps of calculating a second product of a preset second proportion and a memory used for file transmission, and taking the ratio of the second product to the maximum concurrent number of the system as the block size; and partitioning the file to be uploaded according to the size of the partitions.
The invention also provides a file concurrent block uploading method, which is applied to the client and comprises the following steps: an uploading request sending process: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier; a file block to be uploaded and an uploading thread information receiving process: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; data uploading process: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded; the second loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The invention also provides a file concurrent block uploading device, which is applied to a server and comprises:
an upload request receiving module configured to: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier;
the file block information acquisition module to be uploaded is used for: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
an upload thread allocation module to: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client;
an upload data processing module for: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data;
a first loop execution module to: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein when the processor executes the program, the steps of any one of the file concurrent block uploading methods are realized.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for concurrent block-wise uploading of a file as described in any of the above.
According to the file concurrent block uploading method, the file concurrent block uploading device, the electronic equipment and the storage medium, the server calculates the number of uploading threads allowing concurrence, the files to be uploaded are processed in a block mode at the same time, the specific object for concurrent uploading is confirmed, the client finally starts a plurality of uploading threads based on the number of the uploading threads allowing concurrence, the concurrent uploading process of a plurality of file blocks of the files to be uploaded is achieved, the fast and efficient uploading process of the files is achieved based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided; meanwhile, the file is uploaded based on the existing length of the file block which is not completely transmitted, so that the breakpoint transmission of the file is realized.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a file concurrent block uploading method provided by the present invention;
FIG. 2 is a second schematic flowchart of a file concurrent block uploading method according to the present invention;
FIG. 3 is a schematic flow chart of a file concurrent block uploading method applied to a server according to the present invention;
FIG. 4 is a schematic flow chart of a file concurrent block uploading method applied to a client according to the present invention;
FIG. 5 is a schematic structural diagram of a file concurrent block uploading apparatus provided by the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow diagram of a file concurrent block uploading method provided by the present invention, and as shown in fig. 1, the method is applied to a server and includes:
s110, upload request receiving process: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier;
s120, acquiring information of the file blocks to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
s130, uploading a thread distribution process: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client;
s140, uploading data processing flow: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data;
s150, a first loop execution flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Meta-information represents system data that is used to characterize a file, such as access rights, file size, file owner, file data distribution information, etc., and must be obtained when manipulating a file in order to locate the file and obtain the content or associated attributes of the file.
The client reads a file to be uploaded from a local file system, acquires meta information of the file, specifically may include a name of the file (hereinafter, referred to as a file name), an identifier set by a user for the file (hereinafter, referred to as a file identifier), and a size of the file, and sends the file to the server through an HTTP protocol.
After receiving the HTTP request, the server reads the file name, the file identifier, and the file size therefrom. The server uses the file identifier to inquire whether the file is uploaded from the historical transmission record file, if the historical transmission record does not contain the file, the file is considered to be transmitted for the first time, then the file block size set in the configuration file by the user is read, then the file block size is divided by the file block size, the number of the file blocks which need to be transmitted and the starting and stopping offset of each file block are calculated, and the information is written into the historical transmission record; if the file is contained in the historical transmission record file, the file block size, the initial offset of each file block, the transmission length of each file block and whether each file block is completely transmitted or not are directly read from the historical transmission record file.
The file block offset refers to the number of bytes for moving a file to be uploaded forwards or backwards from a specified position, and the file block offset is used for describing that the file block which is completely transmitted is located at a specific position of the file to be uploaded, namely, the file block offset is used for describing the blocking mode of the file to be uploaded; meanwhile, the transmission progress of a certain file block which is not transmitted is described by using the number of transmitted bytes.
The server calculates the number N of uploading threads allowed to be concurrent according to the current server load, and simultaneously sends the calculation result and the information of the file blocks to be uploaded to the client; the client starts corresponding number of uploading threads according to the received number N of uploading threads allowing concurrency, and performs parallel uploading process on the file blocks depending on the started multiple threads; the server receives a plurality of file block data sent by the client and updates the historical uploading record in real time according to the actual file block receiving condition.
Taking the historical uploading record as a judgment basis, and repeatedly executing the file uploading process when the file to be uploaded is not uploaded; and when the file to be uploaded finishes uploading, stopping the file uploading process.
According to the file concurrent block uploading method provided by the invention, the number of uploading threads allowing concurrency is calculated through the server, the files to be uploaded are subjected to block processing at the same time, a specific object for concurrent uploading is confirmed, and the client finally opens a plurality of uploading threads based on the number of the uploading threads allowing concurrency, so that the concurrent uploading process of a plurality of file blocks of the files to be uploaded is realized, the rapid and efficient uploading process of the files is realized based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided; the file is uploaded based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the reliability of the file concurrent block uploading process is ensured by updating the historical uploading records of the file blocks in real time, and the problems of error transmission and missing transmission are avoided.
According to the file concurrent block uploading method provided by the invention, in the invention, the updating of the historical uploading record according to the file block data comprises the following steps: checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block is uploaded; if the verification is unsuccessful, discarding the corresponding file block; and/or when the uploading of the file blocks is interrupted, storing the partially uploaded data information and recording the number of bytes uploaded.
The data compression method specified by the server is used for compressing the data, the MD5 check code of the data is calculated, and then the POST request in the HTTP protocol is used for transmitting the MD5 check code and the length of the data as a request header and the part of the file block which is not transmitted as a request body to the server.
After receiving the file block transmission request, the server reads the MD5 check code, the data length and the part of the file block which is not transmitted yet. If the length reaches the length of the appointed file block, calculating the MD5 check code of the data, if the length is consistent with the MD5 check code transmitted by the user, regarding the transmission as correct, updating the length of the corresponding file block of the appointed file in the historical transmission record, identifying the corresponding file block as the transmission is finished, otherwise, regarding the data transmission as wrong, discarding the transmission result, deleting the uploaded part of the appointed file block, and updating the length of the corresponding file block of the appointed file in the historical transmission record, which is 0. And after the server processes the request, reducing the number of threads which are transmitted by all the clients of the current server in a concurrent manner by one.
The server reads the uploaded part of the specified file block, and combines the uploaded part with the data transmitted this time, if the total length does not reach the length of the appointed file block, the file block is not transmitted, the file block is directly stored, the transmitted length of the corresponding file block of the specified file in the history transmission record is updated, and the history transmission record is updated according to the transmitted length, so that the file can be continuously uploaded from the uploading interruption position after the system is recovered to be normal.
According to the file concurrent block uploading method provided by the invention, the accuracy of the uploaded file blocks is judged through the cyclic redundancy check codes, and the historical uploading record is updated after the judgment is finished, so that the uploading reliability of the file is ensured; meanwhile, when the uploading of the file blocks is interrupted, the partially uploaded data information is saved, the history uploading record is updated, the file can be continuously uploaded from the uploading interruption position after the system is recovered to be normal, and the file uploading effectiveness is ensured.
According to the file concurrent block uploading method provided by the invention, in the invention, the calculating of the number of uploading threads allowing concurrency comprises the following steps: acquiring the maximum concurrency number of a system and the maximum concurrent thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the number of uploading threads allowing concurrency of the client; if the number of the uploading threads allowed to be transmitted concurrently is larger than the maximum number of the uploading threads allowed by the client, adjusting the number of the uploading threads allowed to be transmitted concurrently to the maximum number of the uploading threads allowed by the client; if the number of the uploading threads allowed to be concurrent is larger than the current available thread number, adjusting the number of the uploading threads allowed to be concurrent to the current available thread number; and if the number of the uploading threads allowing the concurrency is less than 1, adjusting the number of the uploading threads allowing the concurrency to be 1.
The server reads a configuration file provided by a user and acquires the maximum concurrent number of the system and the maximum concurrent thread number allowed by the client; subtracting the sum of concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; it should be noted that the sum of concurrent threads includes both the uploading thread and the downloading thread; the server multiplies the maximum concurrency number of the system by a preset first ratio to obtain a first product, wherein the first ratio is a preset value and can be preset according to needs, the ratio of the first product to the number of files uploaded and downloaded by the current client is used as the number of uploading threads allowing concurrency of the current client, and the value of the first product can be 0.9, for example.
The number of uploading threads allowed to be concurrently sent by the current client side obtained through the calculation is only a theoretical value, after the number of uploading threads allowed to be concurrently sent is obtained through the calculation, a judgment step needs to be carried out, and a final number of uploading threads allowed to be concurrently sent is obtained based on a judgment result, wherein the specific judgment process is as follows:
if the number of the uploading threads allowed to be concurrently acquired through calculation is larger than the maximum number of the concurrent threads allowed by the client, the requirement of the current file to be uploaded on the uploading threads exceeds the maximum capacity of the current client, and the number of the uploading threads allowed to be concurrently acquired is adjusted to the maximum number of the concurrent threads allowed by the client; if the number of the uploading threads allowed to be concurrently obtained through calculation is larger than the current available number of the uploading threads, the requirement of the current file to be uploaded on the uploading threads is beyond the maximum capacity of the current system, and at the moment, the number of the uploading threads allowed to be concurrently is adjusted to the current available number of the uploading threads; and thirdly, if the number of the uploading threads which are allowed to be transmitted in a concurrent mode and are obtained through calculation is smaller than 1, the requirement of the current file to be uploaded on the uploading threads is smaller, but in order to ensure the normal operation of the uploading process, the uploading threads which are allowed to be transmitted in a concurrent mode are adjusted to be 1.
According to the file concurrent block uploading method provided by the invention, the range of the current available thread number is locked by subtracting the thread number currently occupied by other clients from the maximum concurrent thread number of the system, and meanwhile, the actually used thread number is flexibly adjusted according to actual requirements by presetting a first proportion coefficient, so that the balanced utilization of uploading threads is realized; meanwhile, the number of the uploading threads which are allowed to be transmitted in parallel and obtained through calculation is judged with the maximum number of the uploading threads allowed by the client, the number of the currently available uploading threads and the like respectively, the final number of the uploading threads which are allowed to be transmitted in parallel is obtained based on the judgment result, the number of the uploading threads which are allowed to be transmitted in parallel and obtained through calculation can be better matched with the current actual uploading capacity of the server or the client based on the judgment step, and efficient transmission of files to be uploaded is achieved on the premise that the server and the client exert the maximum uploading capacity.
According to the file concurrent block uploading method provided by the invention, in the invention, the method further comprises the following steps: calculating a second product of a preset second proportion and a memory for file transmission, and taking the ratio of the second product to the maximum concurrency number of the system as the block size; and partitioning the file to be uploaded according to the size of the partitions.
It should be noted that the memory of the server is a fixed value, and when the system runs, the memory of the server is divided into two parts: the occupied memory and the residual memory are too large, which causes the low operation speed of the system, and the residual memory is too large, which indicates that the utilization rate of the server is low. In the invention, the memory used for file transmission may be a residual memory or may be a set value, a preset second ratio is multiplied by the memory used for file transmission to obtain a second product, and meanwhile, the ratio of the second product to the maximum concurrent number of the system is used as a specific blocking standard of the file block to perform blocking processing on the file to be uploaded, and the value of the preset second ratio may be preset according to actual needs and is not a fixed and unchangeable value. The preset second ratio may take a value of 0.9.
According to the file concurrent block uploading method provided by the invention, the memory for uploading the final file is regulated and controlled by utilizing the preset second proportion, so that the balance processing of two aspects of the running speed of the server and the utilization rate of the server is realized, and the running capacity of the server is exerted to the maximum extent on the premise of meeting the file uploading speed; meanwhile, the adjustability of the second proportion is preset, so that the flexible adjustment of the operation capacity of the server is realized, and the difference requirement of the priority of the uploading speed of the file blocks or the priority of the uploading quantity of the file blocks can be met.
Fig. 2 is a second flowchart of the file concurrent block uploading method provided by the present invention, and as shown in fig. 2, the method is applied to the client, and includes:
s210, an uploading request sending process: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier;
s220, a file block to be uploaded and uploading thread information receiving process: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
s230, data uploading flow: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded;
s240, a second loop executes the process: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The client reads the meta information of the file to be uploaded and sends the meta information to the server, the server obtains the file block information of the file to be uploaded by taking the file identifier in the meta information as an acquisition basis, the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted, and the file block offset is used for describing the blocking mode of the file to be uploaded; meanwhile, the transmission progress of a certain file block which is not transmitted is described by using the number of transmitted bytes.
Meanwhile, the server calculates the number of uploading threads allowed to be concurrent based on the load of the server, and sends the obtained information of the file blocks to be uploaded and the number of the uploading threads allowed to be concurrent to the client, the client starts the corresponding number of uploading threads by taking the number of the uploading threads allowed to be concurrent as the basis, meanwhile, the information of the file blocks to be uploaded is taken as the basis for uploading the file, and the uploading threads are taken as a transmission path to send the file block data to the server.
Taking the historical uploading record as a judgment basis, and repeatedly executing the file uploading process when the file to be uploaded is not uploaded; and when the file to be uploaded finishes uploading, stopping the file uploading process.
According to the file concurrent block uploading method provided by the invention, the number of uploading threads allowing concurrency is calculated through the server, the files to be uploaded are subjected to block processing at the same time, a specific object for concurrent uploading is confirmed, and the client finally opens a plurality of uploading threads based on the number of the uploading threads allowing concurrency, so that the concurrent uploading process of a plurality of file blocks of the files to be uploaded is realized, the rapid and efficient uploading process of the files is realized based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided; the file is uploaded based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the reliability of the file concurrent block uploading process is ensured by updating the historical uploading records of the file blocks in real time, and the problems of error transmission and missing transmission are avoided.
Fig. 3 is a schematic flowchart of a file concurrent block uploading method applied to a server, where as shown in fig. 3, the method includes:
step1, starting the process, searching the historical transmission record of the corresponding file, and searching the historical transmission record of the corresponding file by the server based on the file identifier;
step2, calculating a transmission plan and the number of concurrent threads, wherein the transmission plan comprises a compression mode of a file block and the calculation of a cyclic redundancy check code, and the number of concurrent threads represents the number of uploading threads allowed to be concurrent;
step3, receiving file block data, and judging whether the transmission of the current file block is finished;
step4, if the current file block is not completely transmitted, updating the number of bytes already transmitted by the current file block; and if the current file block is transmitted, verifying the check code, and updating the file block data and the meta information based on the verification result.
Step5, the process ends.
According to the file concurrent block uploading method provided by the invention, the number of uploading threads allowing concurrency is calculated through the server, the files to be uploaded are subjected to block processing at the same time, a specific object for concurrent uploading is confirmed, and the client finally opens a plurality of uploading threads based on the number of the uploading threads allowing concurrency, so that the concurrent uploading process of a plurality of file blocks of the files to be uploaded is realized, the rapid and efficient uploading process of the files is realized based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided.
Fig. 4 is a schematic flowchart of a file concurrent block uploading method applied to a client according to the present invention, and as shown in fig. 4, the method includes:
step1, starting the process, transmitting file meta information to the server, wherein the file meta information comprises file identifiers for distinguishing different files;
step2, acquiring a transmission plan and a concurrent thread number, wherein the transmission plan comprises a compression mode of a file block and calculation of a cyclic redundancy check code, and the concurrent thread number represents the number of uploading threads allowed to be concurrent;
step3, starting a plurality of transmission threads, allocating a file block uploading task for each thread, wherein each uploading task comprises compressing a file block, calculating a check code, and uploading the compressed file block and the obtained check code;
step4, judging whether all files are completely transmitted, if not, executing Step2 and Step3 again; if it has, the flow ends.
According to the file concurrent block uploading method provided by the invention, the number of uploading threads allowing concurrency is calculated through the server, the files to be uploaded are subjected to block processing at the same time, a specific object for concurrent uploading is confirmed, and the client finally opens a plurality of uploading threads based on the number of the uploading threads allowing concurrency, so that the concurrent uploading process of a plurality of file blocks of the files to be uploaded is realized, the rapid and efficient uploading process of the files is realized based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided.
Fig. 5 is a schematic structural diagram of a file concurrent block uploading apparatus provided by the present invention, which is applied to a server, and as shown in fig. 5, the apparatus includes: an upload request receiving module 510, a to-be-uploaded file block information obtaining module 520, an upload thread allocating module 530, an upload data processing module 540, and a first loop executing module 550, wherein:
an upload request receiving module 510 configured to: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier;
a file block information to be uploaded acquiring module 520, configured to: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
an upload thread assignment module 530 to: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client;
an upload data processing module 540, configured to: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data;
a first loop execution module 550 for: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
According to the file concurrent block uploading device, the number of uploading threads allowing concurrence is calculated through the server, the files to be uploaded are processed in a block mode, a specific object for concurrent uploading is confirmed, the client terminal finally opens the plurality of uploading threads based on the number of the uploading threads allowing concurrence, the concurrent uploading process of the plurality of file blocks of the files to be uploaded is achieved, the fast and efficient uploading process of the files is achieved based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided; the file is uploaded based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the reliability of the file concurrent block uploading process is ensured by updating the historical uploading records of the file blocks in real time, and the problems of error transmission and missing transmission are avoided.
According to the file concurrent block uploading apparatus provided by the present invention, when the upload data processing module 540 is configured to update the historical upload record according to the file block data, it is specifically configured to: checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block is uploaded; if the verification is unsuccessful, discarding the corresponding file block; and/or when the uploading of the file blocks is interrupted, storing the partially uploaded data information and recording the number of bytes uploaded.
The file concurrent block uploading device provided by the invention judges the accuracy of the uploaded file blocks through the cyclic redundancy check codes, updates the historical uploading record after the judgment is finished, and ensures the reliability of file uploading based on the judgment; meanwhile, when the uploading of the file blocks is interrupted, the partially uploaded data information is stored, the historical uploading record is updated, and the file uploading can be continued from the uploading interruption position after the system is recovered to be normal, so that the uploading process has an interruption point continuous transmission function, and the file uploading effectiveness is ensured.
According to the file concurrent block uploading apparatus provided by the present invention, when the uploading thread allocating module 530 is used for calculating the number of uploading threads allowing for concurrency, it is specifically configured to: acquiring the maximum concurrency number of a system and the maximum concurrent thread number allowed by the client; subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number; calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the number of uploading threads allowing concurrency of the client; if the number of the uploading threads allowed to be transmitted concurrently is larger than the maximum number of the uploading threads allowed by the client, adjusting the number of the uploading threads allowed to be transmitted concurrently to the maximum number of the uploading threads allowed by the client; if the number of the uploading threads allowed to be concurrent is larger than the current available thread number, adjusting the number of the uploading threads allowed to be concurrent to the current available thread number; and if the number of the uploading threads allowing the concurrency is less than 1, adjusting the number of the uploading threads allowing the concurrency to be 1.
According to the file concurrent block uploading device, the range of the current available thread number is locked by subtracting the thread number currently occupied by other clients from the maximum concurrent thread number of the system, and meanwhile, the coefficient of the first proportion is preset to flexibly adjust the actually used thread number according to actual requirements, so that the uploading thread is utilized in a balanced manner; meanwhile, the number of the uploading threads which are allowed to be transmitted in parallel and obtained through calculation is judged with the maximum number of the uploading threads allowed by the client, the number of the currently available uploading threads and the like respectively, the final number of the uploading threads which are allowed to be transmitted in parallel is obtained based on the judgment result, the number of the uploading threads which are allowed to be transmitted in parallel and obtained through calculation can be better matched with the current actual uploading capacity of the server or the client based on the judgment step, and efficient transmission of files to be uploaded is achieved on the premise that the server and the client exert the maximum uploading capacity.
According to the file concurrent block uploading device provided by the present invention, the device 500 is further configured to calculate a second product of a preset second ratio and a memory used for file transmission, and a ratio of the second product to the maximum concurrent number of the system is used as a block size; and partitioning the file to be uploaded according to the size of the partitions.
According to the file concurrent block uploading device, the utilization memory for uploading the final file is regulated and controlled by utilizing the preset second proportion, so that the balance processing of the two aspects of the running speed of the server and the utilization rate of the server is realized, and the running capacity of the server is exerted to the greatest extent on the premise of meeting the file uploading speed; meanwhile, the adjustability of the second proportion is preset, so that the flexible adjustment of the operation capacity of the server is realized, and the difference requirements of the uploading speed priority of the file blocks or the uploading quantity priority of the file blocks are met respectively.
The invention also provides a file concurrent block uploading device, which is applied to a client, and comprises: upload request sending module, wait to upload file block and upload thread information receiving module, data upload module and second circulation execution module, wherein:
an upload request sending module, configured to: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier;
a file block to be uploaded and an uploading thread information receiving module, configured to: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
a data upload module to: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded;
a second loop execution module to: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
According to the file concurrent block uploading device, the number of uploading threads allowing concurrence is calculated through the server, the files to be uploaded are processed in a block mode, a specific object for concurrent uploading is confirmed, the client terminal finally opens the plurality of uploading threads based on the number of the uploading threads allowing concurrence, the concurrent uploading process of the plurality of file blocks of the files to be uploaded is achieved, the fast and efficient uploading process of the files is achieved based on the process, the overall uploading efficiency of the files is improved, and resource conflict is avoided; the file is uploaded based on the existing length of the file block which is not completely transmitted, so that breakpoint continuous transmission of the file is realized; meanwhile, the reliability of the file concurrent block uploading process is ensured by updating the historical uploading records of the file blocks in real time, and the problems of error transmission and missing transmission are avoided.
Fig. 6 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 6: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a file concurrent block upload method comprising: an upload request receiving process: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier; obtaining information of file blocks to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; uploading thread distribution flow: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client; uploading data and processing flow: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data; the first loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Or the file concurrent block uploading method comprises the following steps: an uploading request sending process: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier; a file block to be uploaded and an uploading thread information receiving process: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; data uploading process: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded; the second loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the file concurrent block uploading method provided by the above methods, the file concurrent block uploading method including: an upload request receiving process: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier; obtaining information of file blocks to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; uploading thread distribution flow: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client; uploading data and processing flow: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data; the first loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Or the file concurrent block uploading method comprises the following steps: an uploading request sending process: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier; a file block to be uploaded and an uploading thread information receiving process: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; data uploading process: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded; the second loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to execute the file concurrent block uploading method provided in the foregoing, the file concurrent block uploading method including: an upload request receiving process: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier; obtaining information of file blocks to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; uploading thread distribution flow: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client; uploading data and processing flow: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data; the first loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
Or the file concurrent block uploading method comprises the following steps: an uploading request sending process: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier; a file block to be uploaded and an uploading thread information receiving process: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted; data uploading process: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded; the second loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. A file concurrent block uploading method is applied to a server and is characterized by comprising the following steps:
an upload request receiving process: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier;
obtaining information of file blocks to be uploaded: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
uploading thread distribution flow: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client;
uploading data and processing flow: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data;
the first loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
2. The method for uploading the file according to the concurrent blocks as claimed in claim 1, wherein the updating the history uploading record according to the block data of the file comprises:
checking the completely uploaded file blocks through a cyclic redundancy check code; if the verification is successful, recording the information that the corresponding file block is uploaded; if the verification is unsuccessful, discarding the corresponding file block;
and/or when the uploading of the file blocks is interrupted, storing the partially uploaded data information and recording the number of bytes uploaded.
3. The method for uploading the file according to the concurrent block, according to claim 1, wherein the calculating the number of uploading threads allowing the concurrency comprises:
acquiring the maximum concurrency number of a system and the maximum concurrent thread number allowed by the client;
subtracting the sum of the concurrent thread numbers of other clients from the maximum concurrent number of the system to obtain the current available thread number;
calculating a first product of a preset first proportion and the maximum concurrency number of the system, and taking the ratio of the first product to the number of files being uploaded and downloaded as the number of uploading threads allowing concurrency of the client;
if the number of the uploading threads allowed to be transmitted concurrently is larger than the maximum number of the uploading threads allowed by the client, adjusting the number of the uploading threads allowed to be transmitted concurrently to the maximum number of the uploading threads allowed by the client;
if the number of the uploading threads allowed to be concurrent is larger than the current available thread number, adjusting the number of the uploading threads allowed to be concurrent to the current available thread number;
and if the number of the uploading threads allowing the concurrency is less than 1, adjusting the number of the uploading threads allowing the concurrency to be 1.
4. The method for file concurrent block upload according to claim 1, further comprising:
calculating a second product of a preset second proportion and a memory for file transmission, and taking the ratio of the second product to the maximum concurrency number of the system as the block size; and partitioning the file to be uploaded according to the size of the partitions.
5. A file concurrent block uploading method is applied to a client and is characterized by comprising the following steps:
an uploading request sending process: reading meta information of a file to be uploaded; wherein the meta information comprises a file identifier;
a file block to be uploaded and an uploading thread information receiving process: receiving file block information to be uploaded and uploading thread number allowing concurrence sent by a server; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
data uploading process: starting an uploading thread according to the uploading thread number allowing concurrence, and sending file block data to the server according to the file block information to be uploaded;
the second loop executes the flow: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
6. The utility model provides a file concurrent blocking upload device, the device is applied to the server, its characterized in that includes:
an upload request receiving module configured to: receiving meta information of a file to be uploaded, which is sent by a client; wherein the meta information comprises a file identifier;
the file block information acquisition module to be uploaded is used for: acquiring file block information to be uploaded of the file to be uploaded according to the file identification; the file block information to be uploaded comprises the initial offset of the file block to be uploaded and the number of bytes of the file block which is not completely transmitted;
an upload thread allocation module to: calculating the uploading thread number allowing concurrency, and sending the file block information to be uploaded and the uploading thread number allowing concurrency to the client;
an upload data processing module for: receiving file block data sent by the uploading thread of the client according to the file block information to be uploaded, and updating a historical uploading record according to the file block data;
a first loop execution module to: and repeatedly executing the process until all the file blocks to be uploaded are uploaded.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the file concurrent block upload method according to any of claims 1 to 5 when executing the program.
8. A non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the file concurrent block upload method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110886685.4A CN113824760B (en) | 2021-08-03 | 2021-08-03 | File concurrency block uploading method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110886685.4A CN113824760B (en) | 2021-08-03 | 2021-08-03 | File concurrency block uploading method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113824760A true CN113824760A (en) | 2021-12-21 |
| CN113824760B CN113824760B (en) | 2024-01-30 |
Family
ID=78924217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110886685.4A Active CN113824760B (en) | 2021-08-03 | 2021-08-03 | File concurrency block uploading method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113824760B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114285838A (en) * | 2021-12-22 | 2022-04-05 | 中国农业银行股份有限公司 | File uploading method, device, system, storage medium and electronic equipment |
| CN114598697A (en) * | 2022-03-29 | 2022-06-07 | 浪潮云信息技术股份公司 | Method and device for transmitting file and computer readable storage medium |
| CN115617760A (en) * | 2022-12-20 | 2023-01-17 | 中化现代农业有限公司 | Method for rapidly uploading large-batch files |
| CN117155922A (en) * | 2023-10-31 | 2023-12-01 | 国家超级计算天津中心 | File transmission method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103442037A (en) * | 2013-08-09 | 2013-12-11 | 华南理工大学 | Method for achieving multithreading breakpoint upload of oversized file based on FTP |
| WO2014056428A1 (en) * | 2012-10-12 | 2014-04-17 | 腾讯科技（深圳）有限公司 | File uploading method and system |
| US20150163301A1 (en) * | 2013-12-06 | 2015-06-11 | Cellco Partnership D/B/A Verizon Wireless | System for and method for media upload multithreading for large file uploads |
| CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
| CN108093063A (en) * | 2017-12-26 | 2018-05-29 | 中国电信股份有限公司新疆分公司 | Big file multithreading FTP method for uploading |
| CN109547566A (en) * | 2018-12-25 | 2019-03-29 | 华南理工大学 | A kind of multithreading upload optimization method distributed based on memory |
| CN109756568A (en) * | 2018-12-29 | 2019-05-14 | 上海掌门科技有限公司 | Processing method, equipment and the computer readable storage medium of file |
| WO2020001488A1 (en) * | 2018-06-28 | 2020-01-02 | 杭州海康威视系统技术有限公司 | File download method and apparatus, client and computer readable storage medium |
-
2021
- 2021-08-03 CN CN202110886685.4A patent/CN113824760B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014056428A1 (en) * | 2012-10-12 | 2014-04-17 | 腾讯科技（深圳）有限公司 | File uploading method and system |
| CN103442037A (en) * | 2013-08-09 | 2013-12-11 | 华南理工大学 | Method for achieving multithreading breakpoint upload of oversized file based on FTP |
| US20150163301A1 (en) * | 2013-12-06 | 2015-06-11 | Cellco Partnership D/B/A Verizon Wireless | System for and method for media upload multithreading for large file uploads |
| CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
| CN108093063A (en) * | 2017-12-26 | 2018-05-29 | 中国电信股份有限公司新疆分公司 | Big file multithreading FTP method for uploading |
| WO2020001488A1 (en) * | 2018-06-28 | 2020-01-02 | 杭州海康威视系统技术有限公司 | File download method and apparatus, client and computer readable storage medium |
| CN109547566A (en) * | 2018-12-25 | 2019-03-29 | 华南理工大学 | A kind of multithreading upload optimization method distributed based on memory |
| CN109756568A (en) * | 2018-12-29 | 2019-05-14 | 上海掌门科技有限公司 | Processing method, equipment and the computer readable storage medium of file |
Non-Patent Citations (2)
| Title |
|---|
| 阮晓龙，李朋楠: "基于Web的大文件高效上传方法", 《 计算机系统应用》, no. 2020, pages 234 - 239 * |
| 黎苑文;程明智;徐秀花;杨义先;: "断点续传及多线程机制在远程传版中的应用研究", 北京印刷学院学报, no. 06, pages 53 - 56 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114285838A (en) * | 2021-12-22 | 2022-04-05 | 中国农业银行股份有限公司 | File uploading method, device, system, storage medium and electronic equipment |
| CN114598697A (en) * | 2022-03-29 | 2022-06-07 | 浪潮云信息技术股份公司 | Method and device for transmitting file and computer readable storage medium |
| CN115617760A (en) * | 2022-12-20 | 2023-01-17 | 中化现代农业有限公司 | Method for rapidly uploading large-batch files |
| CN115617760B (en) * | 2022-12-20 | 2023-04-18 | 中化现代农业有限公司 | Method for quickly uploading large-batch files |
| CN117155922A (en) * | 2023-10-31 | 2023-12-01 | 国家超级计算天津中心 | File transmission method and device |
| CN117155922B (en) * | 2023-10-31 | 2024-01-30 | 国家超级计算天津中心 | File transmission method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113824760B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113824760A (en) | File concurrent block uploading method and device, electronic equipment and storage medium | |
| CN109683930B (en) | Air conditioning equipment program upgrading method, device and system and household electrical appliance | |
| CN107689976B (en) | File transmission method and device | |
| CN106170968B (en) | A kind of data compression storage method, device and distributed file system | |
| US10020916B2 (en) | Method and apparatus for data communication of vehicle | |
| CN110661829B (en) | File downloading method and device, client and computer readable storage medium | |
| CN111641563B (en) | Flow self-adaption method and system based on distributed scene | |
| CN112948174A (en) | Message processing method and device, computer equipment and storage medium | |
| CN110995617B (en) | MQTT-based data transmission method and device, computer equipment and storage medium | |
| CN111490963A (en) | Data processing method, system, equipment and storage medium based on QUIC protocol stack | |
| CN110287007A (en) | Data call response method, server and computer readable storage medium | |
| CN103259797A (en) | Data file transmission method and platform | |
| CN112422497B (en) | Message transmission method and device and computer equipment | |
| CN111490947A (en) | Data packet transmitting method, data packet receiving method, system, device and medium | |
| CN112153132A (en) | File uploading method, device and equipment based on virtualization management platform | |
| CN110532237A (en) | Concurrent processing method, the apparatus and system of formatted data file | |
| CN106790653B (en) | File transmission processing method and device | |
| CN107707395B (en) | Data transmission method, device and system | |
| CN113672284A (en) | Instruction execution method, system, electronic equipment and storage medium | |
| CN110413398B (en) | Task scheduling method and device, computer equipment and storage medium | |
| CN109936609B (en) | Terminal chain type upgrading method and device and upgrading management server | |
| CN110474954B (en) | Block chain-based file sharing method and device, computer equipment and readable storage medium | |
| CN111245949A (en) | File filing and transmission method, device and equipment | |
| KR20220053386A (en) | Data communication method and apparatus for efficient file transmission | |
| CN113132324B (en) | Sample identification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |