CN116405491B - File batch uploading method and system, electronic equipment and computer readable storage medium - Google Patents
File batch uploading method and system, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN116405491B CN116405491B CN202310678904.9A CN202310678904A CN116405491B CN 116405491 B CN116405491 B CN 116405491B CN 202310678904 A CN202310678904 A CN 202310678904A CN 116405491 B CN116405491 B CN 116405491B
- Authority
- CN
- China
- Prior art keywords
- batch
- uploading
- files
- uploaded
- nth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 96
- 101100134058 Caenorhabditis elegans nth-1 gene Proteins 0.000 claims abstract description 29
- 238000012795 verification Methods 0.000 claims description 28
- 238000004458 analytical method Methods 0.000 claims description 7
- 230000001934 delay Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 15
- 230000005540 biological transmission Effects 0.000 description 18
- 238000012546 transfer Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 13
- 230000005856 abnormality Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of computer networks, in particular to a method, a system, electronic equipment and a computer readable storage medium for uploading files in batches, and aims to solve the problem that uploading efficiency is low because an uploading queue cannot be dynamically adjusted according to network conditions in the prior art. For this purpose, the file batch uploading method of the invention comprises the following steps: the 1 st batch uploading is completed according to the preset file quantity; determining a reference number corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch; determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch; and uploading files of the nth batch according to the determined number of files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading. According to the method and the device for uploading the files, the number of the files to be transmitted next time is dynamically adjusted and determined according to the number of the files uploaded last time, so that the file uploading process is more flexible, and the uploading efficiency is improved.
Description
Technical Field
The invention relates to the field of computer networks, and particularly provides a method and a system for uploading files in batches, electronic equipment and a computer readable storage medium.
Background
Along with the continuous development of science and technology, people can download data files through a network to acquire required information, and on the other hand, people can upload the data files in the terminal through the network to realize information sharing. Due to the increasing trend of data, single file uploading cannot meet the uploading demands of people, and therefore many network systems and applications start to upload files in a batch uploading mode.
CN115114248A discloses a method for uploading files in batches, which adopts an asynchronous processing method for uploading and obtaining results, can obtain uploading results in real time and does not obstruct uploading of the next file, and has the advantages of high uploading efficiency and capability of obtaining uploading results for a single file in real time, thereby avoiding the defects of overlong queuing time and the fact that the results can be obtained at last. However, in the scheme, when a fault, network abnormality and the like occur, the uploading process cannot be interrupted in time, and the user experience is affected; the uploading queue cannot be dynamically adjusted according to the network condition of the client and the corresponding condition of the server, so that the overall process consumes longer time; and the corresponding crash compensation mechanism is not provided, and after the crash abnormality of the client occurs, the file can only be reprocessed, so that the user experience is affected.
Accordingly, there is a need in the art for a new file batch upload method, system, electronic device, and computer-readable storage medium to solve the above-mentioned problems.
Disclosure of Invention
In order to overcome at least one of the above drawbacks, the present invention is provided to provide a method, a system, an electronic device, and a computer readable storage medium for batch uploading of files, which solve or at least partially solve the technical problem that the uploading queue cannot be dynamically adjusted in the prior art, resulting in long overall process time, low uploading efficiency, and affecting user experience.
In a first aspect, the present invention provides a method for batch uploading files, including:
the 1 st batch uploading is completed according to the preset file quantity;
determining a reference number corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch;
determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch;
and uploading the files of the nth batch according to the determined number of the files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading.
In one technical scheme of the file batch uploading method, the determining the reference number corresponding to the n-th batch uploading based on the number of files uploaded by the n-1-th batch includes:
When n=2, the preset file number of the 1 st batch is used as the corresponding reference number of the 2 nd batch uploading.
In one technical scheme of the file batch uploading method, the determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch includes:
acquiring network conditions respectively corresponding to the 1 st batch uploading and the 2 nd batch uploading;
and determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch uploading.
In one technical scheme of the file batch uploading method, the determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch uploading includes:
and if the network bandwidth uploaded by the 2 nd batch is smaller than the network bandwidth uploaded by the 1 st batch, changing the number of files to be uploaded by the 2 nd batch into a number smaller than the reference number corresponding to the 2 nd batch.
In one technical scheme of the file batch uploading method, the determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch uploading includes:
If the network bandwidth of the batch 2 uploading is not less than the network bandwidth of the batch 1 uploading, comparing the network delays respectively corresponding to the batch 1 uploading and the batch 2 uploading;
if the absolute value of the difference between the network delay corresponding to the 2 nd batch uploading and the network delay corresponding to the 1 st batch uploading is smaller than a first preset threshold, changing the number of files to be uploaded in the 2 nd batch to be larger than the reference number corresponding to the 2 nd batch uploading;
and if the absolute value of the difference between the network delay corresponding to the 2 nd batch uploading and the network delay corresponding to the 1 st batch uploading is not smaller than the first preset threshold, keeping the number of the files to be uploaded in the 2 nd batch as the reference number corresponding to the 2 nd batch uploading.
In one technical scheme of the file batch uploading method, the determining the reference number corresponding to the n-th batch uploading based on the number of files uploaded by the n-1-th batch includes:
when N is 3 to N, obtaining time consumption corresponding to the batch with the shortest time consumption in the finished batch uploading and time consumption corresponding to the N-1 batch uploading, and obtaining the size relation between the number of files uploaded by the N-1 batch and the reference number corresponding to the N-1 batch uploading, wherein the finished batch uploading does not comprise the N-1 batch uploading;
And determining the reference number corresponding to the uploading of the nth batch according to the relationship between the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch and the time consumption corresponding to the uploading of the nth-1 batch and the relationship between the number of files uploaded by the nth-1 batch and the reference number corresponding to the uploading of the nth-1 batch.
In one technical scheme of the method for uploading files in batches, the determining the reference number corresponding to the n-th batch upload according to the time-consuming relationship between the time-consuming corresponding to the batch with the shortest time-consuming in the completed batch upload and the time-consuming corresponding to the n-1-th batch upload and the size relationship between the number of files uploaded by the n-1-th batch and the reference number corresponding to the n-1-th batch upload includes:
if the number of files uploaded by the n-1 th batch is smaller than the number of the reference files corresponding to the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption corresponding to the completed batch uploading is smaller than a second preset threshold, determining that the number of the reference files corresponding to the n-1 th batch is kept equal to the number of the files uploaded by the n-1 th batch;
and if the number of the files uploaded by the n-1 th batch is smaller than the number of the files uploaded by the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption corresponding to the completed batch uploading is larger than the second preset threshold value, determining that the number of the files uploaded by the n-1 th batch is smaller than the number of the files uploaded by the n-1 th batch.
In one technical scheme of the method for uploading files in batches, the determining the reference number corresponding to the n-th batch upload according to the time consumption corresponding to the batch with the shortest time consumption in the completed batch upload and the time consumption corresponding to the n-1-th batch upload and the size relationship between the number of files uploaded by the n-1-th batch and the reference number corresponding to the n-1-th batch upload includes:
if the number of the files uploaded by the n-1 th batch is equal to the number of the reference files corresponding to the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the completed batch uploading is smaller than a third preset threshold, determining that the number of the reference files corresponding to the n-1 th batch is kept the same as the number of the files uploaded by the n-1 th batch;
and if the number of the files uploaded by the n-1 th batch is equal to the number of the files uploaded by the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the completed batch uploading is not less than the third preset threshold value, determining that the number of the files uploaded by the n-1 th batch is less than the number of the files uploaded by the n-1 th batch.
In one technical scheme of the method for uploading files in batches, the determining the reference number corresponding to the n-th batch upload according to the time consumption corresponding to the batch with the shortest time consumption in the completed batch upload and the time consumption corresponding to the n-1-th batch upload and the size relationship between the number of files uploaded by the n-1-th batch and the reference number corresponding to the n-1-th batch upload includes:
if the number of the files uploaded by the n-1 th batch is larger than the number of the files uploaded by the n-1 th batch and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the completed batch uploading is smaller than a fourth preset threshold, determining that the number of the files uploaded by the n-1 th batch is kept equal to the number of the files uploaded by the n-1 th batch;
if the number of the files uploaded by the n-1 th batch is larger than the reference number corresponding to the uploading of the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the uploading of the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch is not smaller than the fourth preset threshold, and meanwhile the time consumption corresponding to the uploading of the n-1 th batch is smaller than the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch, determining that the reference number corresponding to the uploading of the n-1 th batch is larger than the number of the files uploaded by the n-1 th batch;
And if the number of the files uploaded by the n-1 th batch is larger than the reference number corresponding to the uploading of the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the uploading of the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch is not smaller than the fourth preset threshold, and meanwhile, the time consumption corresponding to the uploading of the n-1 th batch is larger than the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch, determining that the reference number corresponding to the uploading of the n-1 th batch is smaller than the number of the files uploaded by the n-1 th batch.
In one technical scheme of the file batch uploading method, the determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch includes:
acquiring network conditions respectively corresponding to the n-1 th batch uploading and the n-th batch uploading;
and determining the number of files to be uploaded in the nth batch according to the network condition and the reference number corresponding to the uploading of the nth batch.
In one technical scheme of the file batch uploading method, the determining the number of files to be uploaded in the nth batch according to the network condition and the reference number corresponding to the uploading of the nth batch includes:
If the network bandwidth uploaded by the nth batch is smaller than the network bandwidth uploaded by the nth-1 batch, changing the number of files to be uploaded by the nth batch to be smaller than the reference number corresponding to the uploading of the nth batch.
In one technical scheme of the file batch uploading method, the determining the number of files to be uploaded in the nth batch according to the network condition and the reference number corresponding to the uploading of the nth batch includes:
if the network bandwidth uploaded by the nth batch is not less than the network bandwidth uploaded by the nth batch, comparing the network delays respectively corresponding to the nth batch uploading and the nth batch uploading;
if the absolute value of the difference between the network delay corresponding to the uploading of the nth batch and the network delay corresponding to the uploading of the nth-1 batch is smaller than a fifth preset threshold, changing the number of files to be uploaded of the nth batch to be larger than the reference number corresponding to the uploading of the nth batch;
and if the absolute value of the difference between the network delay corresponding to the uploading of the nth batch and the network delay corresponding to the uploading of the nth-1 batch is not smaller than the fifth preset threshold, keeping the number of files to be uploaded of the nth batch as the reference number corresponding to the uploading of the nth batch.
In one technical scheme of the file batch uploading method, the "uploading files of the nth batch according to the determined number of files to be uploaded of the nth batch" includes:
sorting the remaining files which are not uploaded according to the size;
and selecting k largest files from the ordered non-uploaded files to upload the files of the nth batch, wherein k is equal to the determined number of files to be uploaded of the nth batch.
In one technical scheme of the file batch uploading method, before the "complete the 1 st batch uploading with the preset number of files", the method further includes:
performing first verification on the file to be uploaded, wherein the first verification comprises at least one of quantity verification, type verification and size verification;
and placing all files passing the first verification into a first array.
In one technical scheme of the file batch uploading method, after the "complete 1 st batch uploading with preset number of files", the method further includes:
placing the uploading result of each batch into a second group;
judging whether the lengths of the first array and the second array are equal, if so, determining that all files passing the first verification are uploaded.
In a second aspect, the present invention provides a file batch upload system, comprising:
the analysis module is configured to determine the corresponding reference number of the uploading of the nth batch based on the number of the files uploaded by the nth-1 batch; determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch;
the uploading module is configured to finish batch 1 uploading according to the number of preset files; and uploading the files of the nth batch according to the determined number of the files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading.
In a third aspect, a control device is provided, where the control device includes a processor and a storage device, where the storage device is adapted to store a plurality of program codes, where the program codes are adapted to be loaded and executed by the processor to perform the file batch upload method according to any one of the above-mentioned file batch upload methods.
In a fourth aspect, a computer readable storage medium is provided, where a plurality of program codes are stored, the program codes are adapted to be loaded and executed by a processor to perform the file batch upload method according to any one of the above-mentioned file batch upload methods.
Scheme 1. A file batch uploading method is characterized by comprising the following steps:
the 1 st batch uploading is completed according to the preset file quantity;
determining a reference number corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch;
determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch;
and uploading the files of the nth batch according to the determined number of the files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading.
The method according to claim 1, wherein the determining the reference number corresponding to the n-th batch upload based on the number of files uploaded by the n-1-th batch includes:
when n=2, the preset file number of the 1 st batch is used as the corresponding reference number of the 2 nd batch uploading.
The method according to the scheme 2, wherein the determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading in the nth batch includes:
acquiring network conditions respectively corresponding to the 1 st batch uploading and the 2 nd batch uploading;
and determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch uploading.
Scheme 4. The method according to scheme 3, wherein the "determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch upload" includes:
and if the network bandwidth uploaded by the 2 nd batch is smaller than the network bandwidth uploaded by the 1 st batch, changing the number of files to be uploaded by the 2 nd batch into a number smaller than the reference number corresponding to the 2 nd batch.
Scheme 5. The method according to scheme 3, wherein the "determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch upload" includes:
if the network bandwidth of the batch 2 uploading is not less than the network bandwidth of the batch 1 uploading, comparing the network delays respectively corresponding to the batch 1 uploading and the batch 2 uploading;
if the absolute value of the difference between the network delay corresponding to the 2 nd batch uploading and the network delay corresponding to the 1 st batch uploading is smaller than a first preset threshold, changing the number of files to be uploaded in the 2 nd batch to be larger than the reference number corresponding to the 2 nd batch uploading;
and if the absolute value of the difference between the network delay corresponding to the 2 nd batch uploading and the network delay corresponding to the 1 st batch uploading is not smaller than the first preset threshold, keeping the number of the files to be uploaded in the 2 nd batch as the reference number corresponding to the 2 nd batch uploading.
The method according to claim 1, wherein the "determining the number of references corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch" includes:
when N is 3 to N, obtaining time consumption corresponding to the batch with the shortest time consumption in the finished batch uploading and time consumption corresponding to the N-1 batch uploading, and obtaining the size relation between the number of files uploaded by the N-1 batch and the reference number corresponding to the N-1 batch uploading, wherein the finished batch uploading does not comprise the N-1 batch uploading;
and determining the reference number corresponding to the uploading of the nth batch according to the relationship between the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch and the time consumption corresponding to the uploading of the nth-1 batch and the relationship between the number of files uploaded by the nth-1 batch and the reference number corresponding to the uploading of the nth-1 batch.
The method according to claim 6, wherein the determining the reference number corresponding to the n-th batch upload according to the size relationship between the time consumption corresponding to the batch with the shortest time consumption in the completed batch upload and the time consumption corresponding to the n-1-th batch upload and the size relationship between the number of files uploaded by the n-1-th batch and the reference number corresponding to the n-1-th batch upload includes:
If the number of files uploaded by the n-1 th batch is smaller than the number of the corresponding reference files uploaded by the n-1 th batch and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption corresponding to the completed batch uploading is smaller than a second preset threshold value, determining that the number of the corresponding reference files uploaded by the n-1 th batch is kept equal to the number of the files uploaded by the n-1 th batch;
and if the number of the files uploaded by the n-1 th batch is smaller than the number of the files uploaded by the n-1 th batch and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption corresponding to the completed batch uploading is not smaller than the second preset threshold value, determining that the number of the files uploaded by the n-1 th batch is smaller than the number of the files uploaded by the n-1 th batch.
The method according to claim 6, wherein the determining the reference number corresponding to the n-th batch upload according to the size relationship between the time consumption corresponding to the batch with the shortest time consumption in the completed batch upload and the time consumption corresponding to the n-1-th batch upload and the size relationship between the number of files uploaded by the n-1-th batch and the reference number corresponding to the n-1-th batch upload includes:
If the number of the files uploaded by the n-1 th batch is equal to the number of the corresponding reference files uploaded by the n-1 th batch and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the completed batch uploading is smaller than a third preset threshold value, determining that the number of the corresponding reference files uploaded by the n-1 th batch is kept the same as the number of the files uploaded by the n-1 th batch;
and if the number of the files uploaded by the n-1 th batch is equal to the number of the files uploaded by the n-1 th batch and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the completed batch uploading is not less than the third preset threshold value, determining that the number of the files uploaded by the n-1 th batch is less than the number of the files uploaded by the n-1 th batch.
The method according to claim 6, wherein the determining the reference number corresponding to the n-th batch upload according to the size relationship between the time consumption corresponding to the batch with the shortest time consumption in the completed batch upload and the time consumption corresponding to the n-1-th batch upload and the size relationship between the number of files uploaded by the n-1-th batch and the reference number corresponding to the n-1-th batch upload includes:
If the number of the files uploaded by the n-1 th batch is larger than the number of the files uploaded by the n-1 th batch and the absolute value of the difference between the time consumption corresponding to the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the completed batch uploading is smaller than a fourth preset threshold, determining that the number of the files uploaded by the n-1 th batch is kept equal to the number of the files uploaded by the n-1 th batch;
if the number of the files uploaded by the n-1 th batch is larger than the reference number corresponding to the uploading of the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the uploading of the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch is not smaller than the fourth preset threshold, and meanwhile the time consumption corresponding to the uploading of the n-1 th batch is smaller than the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch, determining that the reference number corresponding to the uploading of the n-1 th batch is larger than the number of the files uploaded by the n-1 th batch;
and if the number of the files uploaded by the n-1 th batch is larger than the reference number corresponding to the uploading of the n-1 th batch, and the absolute value of the difference between the time consumption corresponding to the uploading of the n-1 th batch and the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch is not smaller than the fourth preset threshold, and meanwhile, the time consumption corresponding to the uploading of the n-1 th batch is larger than the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch, determining that the reference number corresponding to the uploading of the n-1 th batch is smaller than the number of the files uploaded by the n-1 th batch.
The method according to any one of claims 6 to 9, wherein the determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading in the nth batch includes:
acquiring network conditions respectively corresponding to the n-1 th batch uploading and the n-th batch uploading;
and determining the number of files to be uploaded in the nth batch according to the network condition and the reference number corresponding to the uploading of the nth batch.
The method according to claim 10, wherein the "determining the number of files to be uploaded in the nth batch according to the network condition and the reference number corresponding to the uploading in the nth batch" includes:
if the network bandwidth uploaded by the nth batch is smaller than the network bandwidth uploaded by the nth-1 batch, changing the number of files to be uploaded by the nth batch to be smaller than the reference number corresponding to the uploading of the nth batch.
The method according to claim 10, wherein the "determining the number of files to be uploaded in the nth batch according to the network condition and the reference number corresponding to the uploading in the nth batch" includes:
if the network bandwidth uploaded by the nth batch is not less than the network bandwidth uploaded by the nth batch, comparing the network delays respectively corresponding to the nth batch uploading and the nth batch uploading;
If the absolute value of the difference between the network delay corresponding to the uploading of the nth batch and the network delay corresponding to the uploading of the nth-1 batch is smaller than a fifth preset threshold, changing the number of files to be uploaded of the nth batch to be larger than the reference number corresponding to the uploading of the nth batch;
and if the absolute value of the difference between the network delay corresponding to the uploading of the nth batch and the network delay corresponding to the uploading of the nth-1 batch is not smaller than the fifth preset threshold, keeping the number of files to be uploaded of the nth batch as the reference number corresponding to the uploading of the nth batch.
Scheme 13. The method according to scheme 1, wherein "the uploading of files of the nth batch by the determined number of files to be uploaded" comprises:
sorting the remaining files which are not uploaded according to the size;
and selecting k largest files from the ordered non-uploaded files to upload the files of the nth batch, wherein k is equal to the determined number of files to be uploaded of the nth batch.
Scheme 14. The method according to scheme 1, characterized in that before the "complete batch 1 upload with preset number of files", the method further comprises:
Performing first verification on the file to be uploaded, wherein the first verification comprises at least one of quantity verification, type verification and size verification;
and placing all files passing the first verification into a first array.
The method according to claim 14, wherein after the "complete batch 1 upload with preset number of files", the method further comprises:
placing the uploading result of each batch into a second group;
judging whether the lengths of the first array and the second array are equal, if so, determining that all files passing the first verification are uploaded.
Scheme 16. A file batch upload system, characterized in that it comprises:
the analysis module is configured to determine the corresponding reference number of the uploading of the nth batch based on the number of the files uploaded by the nth-1 batch; determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch;
the uploading module is configured to finish batch 1 uploading according to the number of preset files; and uploading the files of the nth batch according to the determined number of the files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading.
An electronic device comprising a processor and storage means, the storage means being adapted to store a plurality of program code, characterised in that the program code is adapted to be loaded and executed by the processor to perform the method of any one of claims 1 to 15.
A computer readable storage medium having stored therein a plurality of program codes, wherein the program codes are adapted to be loaded and executed by a processor to perform the method of any one of the schemes 1 to 15.
The technical scheme provided by the invention has at least one or more of the following beneficial effects:
in the technical scheme of implementing the invention, all files to be uploaded are firstly transmitted in batches, then the reference number is determined according to the number of files uploaded in the previous batch, and finally the number of files to be transmitted in the next batch is determined according to the reference number.
Further, in the technical scheme of implementing the invention, the reference number is determined according to the size relation between the reference number corresponding to the n-1 th batch uploading and the number of files corresponding to the n-1 th batch uploading (finally uploaded) and the size relation between the time spent for the n-1 th batch uploading and the shortest time spent in the rest completed batches. Comparing the reference number of the n-1 batch with the number of the finally uploaded files to determine the transmission condition of the n-1 batch, comparing the time consumption corresponding to the n-1 batch transmission with the shortest time consumption under the corresponding transmission condition to judge whether the number of the files transmitted by the n-1 batch is reasonable or not, and adjusting the new reference number, namely the reference number corresponding to the n batch transmission. Through the two comparisons, the adjustment of the new reference number is more accurate and reasonable.
Further, under the condition that the network bandwidth is relatively good in preliminary judgment, the network condition is judged more accurately by utilizing the network delay corresponding to the n-1 batch uploading and the network delay corresponding to the n batch uploading, and the condition that the network bandwidth is relatively good but the delay is increased is considered, so that the number of files uploaded next time is determined more accurately.
Further, in the technical scheme of implementing the invention, before uploading the batch files, all the files to be uploaded are arranged according to the file size, and then the largest file in the rest files to be uploaded is selected for each batch of transmission, so that the success of uploading the larger file can be ensured, the stability of the subsequent uploading network is ensured, and more files are transmitted simultaneously.
Further, in the technical scheme of implementing the invention, after all batches are transmitted, whether the lengths of the first array for storing the files to be uploaded and the second array for storing all the transmitted files are equal is judged, so that file missing transmission is avoided.
Drawings
The present disclosure will become more readily understood with reference to the accompanying drawings. As will be readily appreciated by those skilled in the art: the drawings are for illustrative purposes only and are not intended to limit the scope of the present invention. Moreover, like numerals in the figures are used to designate like parts, wherein:
FIG. 1 is a program flow diagram of a file batch upload method according to one embodiment of the present invention;
FIG. 2 is a flow chart illustrating the main steps of a file batch upload method according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of a pre-file upload parameter configuration interface in accordance with one embodiment of the present invention;
FIG. 4 is a schematic diagram of an interface for selecting files to upload according to one embodiment of the present invention;
FIG. 5 is a schematic diagram of a first verification failure popup interface in accordance with an embodiment of the present invention;
FIG. 6 is a diagram of a file upload progress interface in accordance with one embodiment of the present invention;
FIG. 7 is a block diagram illustrating a file batch upload system in accordance with one embodiment of the present invention.
List of reference numerals：
71: an analysis module; 72: and uploading the module.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module," "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, or software components, such as program code, or a combination of software and hardware. The processor may be a central processor, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware, or a combination of both. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "a and/or B" means all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and/or B" and may include A alone, B alone or A and B. The singular forms "a", "an" and "the" include plural referents.
Referring to fig. 1, fig. 1 is a program flow diagram of a file batch upload method according to an embodiment of the present invention. As shown in FIG. 1, the file batch uploading method provided by the invention comprises the following steps:
firstly, parameters required for batch uploading are configured, for example, a file uploading parameter configuration interface schematic diagram shown in fig. 3, and specific parameters can include: the number of files selected at a time, the total number of uploads, the default number of columns per time, the file type, and the file size limit.
After the parameters are configured, the user can click an upload button in the interface schematic diagram of the file to be uploaded, as shown in fig. 4, and then pop up a pop-up window for selecting the file to be uploaded, so that the user can select the file to be uploaded.
When the user confirms the file to be uploaded, the first verification is started, and the verification can comprise at least one of the number, the total number, the file type and the file size of the current selection. Specifically, the verification process may include: firstly, checking whether the selection quantity exceeds the limit, and if so, prompting a user to select the file to exceed the limit by a popup window; secondly, checking whether the total uploading quantity exceeds the limit, namely checking whether the quantity of the selected files and the quantity of the uploaded files is larger than the total uploading quantity; and thirdly, checking whether the size and the type of each file accord with the configuration parameters. Any step of verification failure is prompted by a popup window, for example, a first verification failure prompt popup window interface schematic diagram shown in fig. 5. All the file information passing the first verification is put into an array named realFileArray, and specifically, the put information can be the exclusive naming of the file, the information of the corresponding file can be represented by the corresponding number and the like. And finally, opening a new popup window, displaying the uploading progress in real time in a progress bar mode by using a file uploading progress interface schematic diagram shown in fig. 6, and entering an uploading flow.
For the uploading process, the main step flow diagram of the file batch uploading method shown in fig. 2 is combined, and specifically includes:
before step S10, the files to be uploaded are arranged in descending order according to the file size, so that the largest file is transmitted first for each batch of uploading, and thus more files are transmitted when the subsequent network is stable and concurrent.
Step S10: and finishing the uploading of the 1 st batch according to the preset number of files. For example, the number of files to be transmitted is 100, 10 files are firstly taken out according to a preset default value (for example, the default value is set to 10) of the number of single pairing columns, and the files are transferred into an uploading function constructed based on Promise to call a back-end interface for uploading the files of the batch.
In this embodiment, the time stamp time1 is recorded at the beginning of the first uploading, if a network abnormality or other abnormality occurs, the interface returns the related information of popup window prompt and ends the uploading process, for example, after the processing of the back end without abnormality is completed, the uploading result is returned, the time stamp time2 of the ending of uploading is recorded at this time, and the difference between time1 and time2 is stored in the field uploading time to indicate that the time spent for uploading is long.
In addition, in the uploading process (time 1-time 2) or before uploading (before time 1), the current request estimated delay1 (i.e. the current delay condition of the network in ms) is obtained through the navigator/connection/rtt (an API interface of the browser) attribute of the browser, and stored in the delay mainkey field, and the current network environment bandwidth width1 is obtained through the navigator/connection/downlink (an API interface of the browser) attribute and stored in the downlink key field.
Step S20: and determining the corresponding reference number of the uploading of the nth batch based on the number of the files uploaded by the nth-1 batch.
Step S30: and determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch.
Step S40: and uploading the files of the nth batch according to the determined number of the files to be uploaded of the nth batch.
In this embodiment, when n is taken to be 2, for the 2 nd batch upload, the number of files 10 uploaded by the 1 st batch is directly used as the reference number of the 2 nd batch upload. Record batch 2 start upload timestamp time3 and end timestamp time4. And comparing the delay2 estimated by the current request obtained from the uploading of the 2 nd batch with the delay MainKey field, and comparing the delay2 estimated by the current request with the downlink Key field after obtaining the network environment bandwidth width 2. If width2 is smaller than width1, i.e. the bandwidth is smaller, the network stability is reduced no matter how delay is, and the number of files uploaded in batch 2 is reduced on the basis of the corresponding reference number 10, for example, 5 files are uploaded in batch 2; if width2 is larger than or equal to width1, i.e. the bandwidth is not smaller, further judging the network delay fluctuation condition, if ABS (delay 2-delay 1) is smaller than 200ms, i.e. the network delay fluctuation is smaller than or equal to 200ms (delay 2 is preferably smaller than or equal to delay 1), stabilizing the network environment, expanding the number of files uploaded by the 2 nd batch on the basis of the corresponding reference number 10, for example, uploading 15 files by the 2 nd batch, otherwise, if ABS (delay 2-delay 1) is larger than or equal to 200ms, i.e. the network delay fluctuation is larger, keeping the number of files uploaded by the 2 nd batch on the basis of the corresponding reference number 10, i.e. uploading 10 files by the 2 nd batch. The decision mode of determining the number of file transmissions by adjusting the network bandwidth and delay based on the reference number is also applicable to the uploading process of the 3 rd and subsequent batches.
In this embodiment, when n is taken to be 3, for the reference number determining process of the 3 rd lot, for example, (a) the bandwidth is smaller with 2 < with 1, and the number of the 2 nd lot uploading files is reduced to 5 relative to the reference number 10, comparing the time T2 for uploading the 2 nd lot with the time T1 for the shortest lot in the completed lot (the completed lot does not include the n-1, i.e. the 2 nd lot, so the completed lot includes only the 1 st lot, and the shortest time is the time T1 of the 1 st lot): if ABS (T2-T1) < 2.5s, it indicates that at the time of batch 2 transfer, although the network environment is poor, the transfer time is still long, so that the number of files 5 transferred from batch 2 can be used as the reference number of batch 3 transfer; if ABS (T2-T1) > 2.5s, it is indicated that at the time of batch 2 transfer, although the transfer amount has been reduced due to poor network environment, the transfer time is still too long, and therefore it is required to reduce to 3 on the basis of the number 5 of files transferred at batch 2 as the reference number of batch 3 transfer.
In this embodiment, when n is taken to be 3, for the reference number determining process of the 3 rd batch, for example, (b 1) width2 is greater than or equal to width1 and ABS (delay 2-delay 1) is greater than or equal to 200ms, that is, the bandwidth is not reduced but the network delay fluctuation is slightly larger, and the number of files uploaded by the 2 nd batch is equal to the reference number, comparing the size relationship between T2 and T1: if ABS (T2-T1) < 5s, the time consumption of the transfer in the 2 nd batch is enough, but the quantity 10 of the 2 nd batch transfer files is continuously used as the reference quantity of the 3 rd batch transfer in view of the slightly large delay fluctuation of the 2 nd batch uploading; if ABS (T2-T1) is 5s or more, it is indicated that the transfer takes too much time in lot 2, so it is necessary to reduce the number of files to 5 based on the number of files 10 transferred in lot 2 as the reference number of transfer in lot 3.
In this embodiment, when n is taken to be 3, for the reference number determining process of the 3 rd batch, for example, (b 1) width2 is greater than or equal to width1 and ABS (delay 2-delay 1) < 200ms, i.e. the bandwidth is not reduced, the network delay fluctuation is small, and the number of the 2 nd batch of uploaded files is enlarged to 15 relative to the reference number 10 thereof, the size relationship between T2 and T1 is compared: if ABS (T2-T1) < 3s, the time consumption of the 2 nd batch transmission is enough, and the number 15 of the 2 nd batch transmission files is continued to be used as the reference number of the 3 rd batch transmission; if ABS (T2-T1) > 3s, it is indicated that the transfer time of lot 2 fluctuates greatly with respect to the shortest time, and since the network condition at the time of transfer of lot 2 is good, it is necessary to further determine the size relationship between T2 and T1: if T2 is less than T1, it means that the time consumption of the 2 nd batch transmission is far less than the shortest time consumption, so that the number of files 15 of the 2 nd batch transmission is enlarged to 20 to be used as the reference number of the 3 rd batch transmission, otherwise, if T2 is greater than T1, it means that the time consumption of the 2 nd batch transmission is far greater than the shortest time consumption, so that the number of files 15 of the 2 nd batch transmission is reduced to 10 to be used as the reference number of the 3 rd batch transmission.
In this embodiment, after determining the value of ABS (T2-T1), if the value of T2 is less than T1, then the value of T2 is assigned to T1, i.e., T1 always represents the shortest time. The method of comparing the time-consuming absolute value of the difference with the magnitude relation of the corresponding preset threshold value so as to set the reference number of the next transmission is also applicable to the uploading process of the 4 th and subsequent batches. Different thresholds are set according to different conditions, so that different network conditions can be guaranteed to correspond to a reasonable adjustment range.
In this embodiment, the reference number corresponding to the determined 3 rd batch uploading is replaced by the "2 nd reference number 10" in fig. 1 to perform 3 rd batch uploading, all batches are transmitted completely, then the result is traversed, whether abnormal information exists in the result is judged, if so, the related information is prompted by popup window, wherein the judgment basis of whether all the transmission batches are completed completely is as follows: and placing all the uploaded file information into an upscaleresult array, and judging whether the lengths of the upscalearray array and the upscaleresult array are equal. If yes, the selected file is completely uploaded, and if not, the file uploading operation is returned to be continuously executed.
Preferably, in this embodiment, the uploading result of each batch, such as a file name, a file icon, a file dedicated number, etc., may be stored in the cache array, so that when the uploading is failed and needs to be uploaded again due to the conditions of refreshing a page, network abnormality, etc., it is able to show to the user which files have been uploaded, and the cache mechanism may keep the uploaded file information after the occurrence of a crash abnormality or misoperation of the user at the client, so as to optimize the user experience.
Based on the steps S10-S40, the file batch uploading method provided by the invention firstly transmits all files to be uploaded in batches, then determines a reference number according to the number of the files uploaded in the previous batch, and finally determines the number of the files to be transmitted in the next batch according to the reference number; by the method for dynamically adjusting the number of the single queues, the number of requests is reduced, the problem of performance overhead caused by concurrency of the client interfaces is solved, the number of requests can be effectively reduced, the time required by the whole uploading process is shortened, and the efficiency of batch uploading of files is improved.
It should be noted that, although the foregoing embodiments describe the steps in a specific order, it will be understood by those skilled in the art that, in order to achieve the effects of the present invention, the steps are not necessarily performed in such an order, and may be performed simultaneously (in parallel) or in other orders, and these variations are within the scope of the present invention.
The invention further provides a file batch uploading system.
Referring to fig. 7, fig. 7 is a main block diagram of a file batch upload system according to one embodiment of the present invention. As shown in fig. 7, the file batch uploading system in the embodiment of the invention mainly includes an analysis module 71 and an uploading module 72. In some embodiments, one or more of analysis module 71 and upload module 72 may be combined together into one module. In some embodiments analysis module 71 may be configured to determine a number of references corresponding to the n-th batch upload based on the number of files uploaded by the n-1-th batch; and determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch. The upload module 72 may be configured to determine the number of files to upload next based on the first bandwidth and the second bandwidth. The upload module 13 may be configured to complete the 1 st batch upload in a preset number of files; and uploading files of the nth batch according to the determined number of files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading. In one embodiment, the description of the specific implementation functions may refer to step S10-step S40.
The foregoing batch file uploading system is used for executing the batch file uploading method embodiment shown in fig. 1 to 6, and the technical principles of the two, the technical problems to be solved and the technical effects to be produced are similar, and those skilled in the art can clearly understand that, for convenience and brevity of description, the specific working process and related description of the batch file uploading system can refer to the description of the batch file uploading method embodiment, and will not be repeated here.
It will be appreciated by those skilled in the art that the present invention may implement all or part of the above-described methods according to the above-described embodiments, or may be implemented by means of a computer program for instructing relevant hardware, where the computer program may be stored in a computer readable storage medium, and where the computer program may implement the steps of the above-described embodiments of the method when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include: any entity or device, medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like capable of carrying the computer program code. It should be noted that the computer readable storage medium may include content that is subject to appropriate increases and decreases as required by jurisdictions and by jurisdictions in which such computer readable storage medium does not include electrical carrier signals and telecommunications signals.
Further, the invention also provides a control device. In one control device embodiment according to the present invention, the control device includes a processor and a storage device, the storage device may be configured to store a program for executing the file batch upload method of the above-described method embodiment, and the processor may be configured to execute the program in the storage device, including, but not limited to, the program for executing the file batch upload method of the above-described method embodiment. For convenience of explanation, only those portions of the embodiments of the present invention that are relevant to the embodiments of the present invention are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present invention. The control device may be a control device formed of various electronic devices.
Further, the invention also provides a computer readable storage medium. In one embodiment of a computer readable storage medium according to the present invention, the computer readable storage medium may be configured to store a program for executing the file batch upload method of the above-described method embodiment, which may be loaded and executed by a processor to implement the above-described file batch upload method. For convenience of explanation, only those portions of the embodiments of the present invention that are relevant to the embodiments of the present invention are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present invention. The computer readable storage medium may be a storage device including various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present invention is a non-transitory computer readable storage medium.
Further, it should be understood that, since the respective modules are merely set to illustrate the functional units of the apparatus of the present invention, the physical devices corresponding to the modules may be the processor itself, or a part of software in the processor, a part of hardware, or a part of a combination of software and hardware. Accordingly, the number of individual modules in the figures is merely illustrative.
Those skilled in the art will appreciate that the various modules in the system may be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principle of the present invention, and therefore, the technical solution after splitting or combining falls within the protection scope of the present invention.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will fall within the scope of the present invention.
Claims (10)
1. The file batch uploading method is characterized by comprising the following steps of:
the 1 st batch uploading is completed according to the preset file quantity;
determining a reference number corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch;
determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch;
carrying out file uploading of the nth batch according to the determined number of files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading;
the "determining the number of references corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch" includes:
when N is 3 to N, obtaining time consumption corresponding to the batch with the shortest time consumption in the finished batch uploading and time consumption corresponding to the N-1 batch uploading, and obtaining the size relation between the number of files uploaded by the N-1 batch and the reference number corresponding to the N-1 batch uploading, wherein the finished batch uploading does not comprise the N-1 batch uploading;
and determining the reference number corresponding to the uploading of the nth batch according to the relationship between the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch and the time consumption corresponding to the uploading of the nth-1 batch and the relationship between the number of files uploaded by the nth-1 batch and the reference number corresponding to the uploading of the nth-1 batch.
2. The method of claim 1, wherein determining the number of references corresponding to the n-th batch upload based on the number of files uploaded by the n-1-th batch comprises:
when n=2, the preset file number of the 1 st batch is used as the corresponding reference number of the 2 nd batch uploading.
3. The method of claim 2, wherein the determining the number of files to be uploaded for the nth lot based on the number of references corresponding to the uploading of the nth lot comprises:
acquiring network conditions respectively corresponding to the 1 st batch uploading and the 2 nd batch uploading;
and determining the number of files to be uploaded in the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch uploading.
4. A method according to claim 3, wherein said determining the number of files to be uploaded for the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch upload comprises:
and if the network bandwidth uploaded by the 2 nd batch is smaller than the network bandwidth uploaded by the 1 st batch, changing the number of files to be uploaded by the 2 nd batch into a number smaller than the reference number corresponding to the 2 nd batch.
5. A method according to claim 3, wherein said determining the number of files to be uploaded for the 2 nd batch according to the network condition and the reference number corresponding to the 2 nd batch upload comprises:
If the network bandwidth of the batch 2 uploading is not less than the network bandwidth of the batch 1 uploading, comparing the network delays respectively corresponding to the batch 1 uploading and the batch 2 uploading;
if the absolute value of the difference between the network delay corresponding to the 2 nd batch uploading and the network delay corresponding to the 1 st batch uploading is smaller than a first preset threshold, changing the number of files to be uploaded in the 2 nd batch to be larger than the reference number corresponding to the 2 nd batch uploading;
and if the absolute value of the difference between the network delay corresponding to the 2 nd batch uploading and the network delay corresponding to the 1 st batch uploading is not smaller than the first preset threshold, keeping the number of the files to be uploaded in the 2 nd batch as the reference number corresponding to the 2 nd batch uploading.
6. The method of claim 1, wherein prior to the "complete batch 1 upload with preset number of files", the method further comprises:
performing first verification on the file to be uploaded, wherein the first verification comprises at least one of quantity verification, type verification and size verification;
and placing all files passing the first verification into a first array.
7. The method of claim 6, wherein after the "complete batch 1 upload with preset number of files", the method further comprises:
Placing the uploading result of each batch into a second group;
judging whether the lengths of the first array and the second array are equal, if so, determining that all files passing the first verification are uploaded.
8. A file batch upload system, the system comprising:
the analysis module is configured to determine the corresponding reference number of the uploading of the nth batch based on the number of the files uploaded by the nth-1 batch; determining the number of files to be uploaded in the nth batch based on the reference number corresponding to the uploading of the nth batch; the "determining the number of references corresponding to the uploading of the nth batch based on the number of files uploaded by the nth-1 batch" includes: when N is 3 to N, obtaining time consumption corresponding to the batch with the shortest time consumption in the finished batch uploading and time consumption corresponding to the N-1 batch uploading, and obtaining the size relation between the number of files uploaded by the N-1 batch and the reference number corresponding to the N-1 batch uploading, wherein the finished batch uploading does not comprise the N-1 batch uploading; determining a reference number corresponding to the uploading of the nth batch according to the relationship between the time consumption corresponding to the batch with the shortest time consumption in the uploading of the completed batch and the time consumption corresponding to the uploading of the nth-1 batch and the relationship between the number of files uploaded by the nth-1 batch and the reference number corresponding to the uploading of the nth-1 batch;
The uploading module is configured to finish batch 1 uploading according to the number of preset files; and uploading the files of the nth batch according to the determined number of the files to be uploaded of the nth batch, wherein N traverses from 2 to N, and N is the total batch number of batch uploading.
9. An electronic device comprising a processor and a storage means, the storage means being adapted to store a plurality of program code, characterized in that the program code is adapted to be loaded and executed by the processor to perform the method of any one of claims 1 to 7.
10. A computer readable storage medium, in which a plurality of program codes are stored, characterized in that the program codes are adapted to be loaded and run by a processor to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310678904.9A CN116405491B (en) | 2023-06-09 | 2023-06-09 | File batch uploading method and system, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310678904.9A CN116405491B (en) | 2023-06-09 | 2023-06-09 | File batch uploading method and system, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116405491A CN116405491A (en) | 2023-07-07 |
| CN116405491B true CN116405491B (en) | 2023-09-15 |
Family
ID=87008054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310678904.9A Active CN116405491B (en) | 2023-06-09 | 2023-06-09 | File batch uploading method and system, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116405491B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401552A (en) * | 2020-03-11 | 2020-07-10 | 浙江大学 | Federal learning method and system based on batch size adjustment and gradient compression rate adjustment |
| CN114172998A (en) * | 2021-12-07 | 2022-03-11 | 北京度友信息技术有限公司 | File transmission method, file transmission device, electronic equipment, medium and product |
| CN115037700A (en) * | 2022-04-29 | 2022-09-09 | 北京龙腾佳讯科技股份公司 | Complex network data packet transmission method, system, terminal and storage medium |
| CN115114248A (en) * | 2022-06-22 | 2022-09-27 | 广州鲁邦通智能科技有限公司 | Method and system for uploading files in batches |
| CN115297104A (en) * | 2022-06-30 | 2022-11-04 | 中银金融科技有限公司 | File uploading method and device, electronic equipment and storage medium |
| CN115617760A (en) * | 2022-12-20 | 2023-01-17 | 中化现代农业有限公司 | Method for rapidly uploading large-batch files |
| CN116192759A (en) * | 2022-12-13 | 2023-05-30 | 网易（杭州）网络有限公司 | Data transmission amount adjusting method and device, electronic equipment and storage medium |
| CN116233083A (en) * | 2023-03-16 | 2023-06-06 | 河北新冀网络传媒有限公司 | Efficient transmission method for digital media information |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130254771A1 (en) * | 2012-03-20 | 2013-09-26 | Google Inc. | Systems and methods for continual, self-adjusting batch processing of a data stream |
-
2023
- 2023-06-09 CN CN202310678904.9A patent/CN116405491B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401552A (en) * | 2020-03-11 | 2020-07-10 | 浙江大学 | Federal learning method and system based on batch size adjustment and gradient compression rate adjustment |
| CN114172998A (en) * | 2021-12-07 | 2022-03-11 | 北京度友信息技术有限公司 | File transmission method, file transmission device, electronic equipment, medium and product |
| CN115037700A (en) * | 2022-04-29 | 2022-09-09 | 北京龙腾佳讯科技股份公司 | Complex network data packet transmission method, system, terminal and storage medium |
| CN115114248A (en) * | 2022-06-22 | 2022-09-27 | 广州鲁邦通智能科技有限公司 | Method and system for uploading files in batches |
| CN115297104A (en) * | 2022-06-30 | 2022-11-04 | 中银金融科技有限公司 | File uploading method and device, electronic equipment and storage medium |
| CN116192759A (en) * | 2022-12-13 | 2023-05-30 | 网易（杭州）网络有限公司 | Data transmission amount adjusting method and device, electronic equipment and storage medium |
| CN115617760A (en) * | 2022-12-20 | 2023-01-17 | 中化现代农业有限公司 | Method for rapidly uploading large-batch files |
| CN116233083A (en) * | 2023-03-16 | 2023-06-06 | 河北新冀网络传媒有限公司 | Efficient transmission method for digital media information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116405491A (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108089814B (en) | Data storage method and device | |
| US9513835B2 (en) | Impact-based migration scheduling from a first tier at a source to a second tier at a destination | |
| CN111737230B (en) | Data verification method, device, electronic equipment and readable storage medium | |
| CN107193813B (en) | Data table connection mode processing method and device | |
| CN111324303B (en) | SSD garbage recycling method, SSD garbage recycling device, computer equipment and storage medium | |
| US10771358B2 (en) | Data acquisition device, data acquisition method and storage medium | |
| WO2019109352A1 (en) | Method and device for sampling performance data of apparatus | |
| US9460399B1 (en) | Dynamic event driven storage system optimization | |
| EP3598247A1 (en) | Network timing method and device | |
| CN112466378A (en) | Solid state disk operation error correction method and device and related components | |
| CN116405491B (en) | File batch uploading method and system, electronic equipment and computer readable storage medium | |
| CN105929851B (en) | It is a kind of that holder method and apparatus are controlled using rocking bar equipment | |
| CN115576502B (en) | Data storage method and device, electronic equipment and storage medium | |
| CN113329078B (en) | Data storage method and device | |
| JP2006079488A (en) | Data collection apparatus, data collection system and data collection method | |
| CN110096311B (en) | Method, device and equipment for evaluating aggregation time in streaming calculation and storage medium | |
| CN110046082B (en) | IO characteristic information processing method, IO characteristic information acquisition method and related device | |
| CN114756698A (en) | Play behavior detection method, equipment and storage medium | |
| US11301442B2 (en) | Method and system for using array level time buckets to efficiently calculate top contributors using relevant performance metric | |
| CN109510774B (en) | Method for realizing flow control in data synchronization process | |
| US20210319058A1 (en) | Location-based alarm notification application | |
| CN116391177A (en) | Prioritized inactive memory device updates | |
| US20200410359A1 (en) | Computing device and parameter synchronization method implemented by computing device | |
| CN112905484A (en) | Self-adaptive closed loop performance test method, system and medium | |
| CN112583667A (en) | Content delivery network link evaluation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |