CN113726842B - File uploading method and device, electronic equipment and storage medium - Google Patents
File uploading method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113726842B CN113726842B CN202110778008.0A CN202110778008A CN113726842B CN 113726842 B CN113726842 B CN 113726842B CN 202110778008 A CN202110778008 A CN 202110778008A CN 113726842 B CN113726842 B CN 113726842B
- Authority
- CN
- China
- Prior art keywords
- file
- uploaded
- uploading
- fragment data
- linked list
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9015—Buffering arrangements for supporting a linked list
Abstract
The invention provides a file uploading method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving a file uploading instruction; traversing the file bidirectional linked list, and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files; and sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded. The invention uses the bidirectional linked list structure to carry out centralized and unified scheduling and management on all the files to be processed, realizes simple and convenient parallel processing of multiple files, ensures that the fragment uploading process does not need to depend on a publishing and subscribing mode any more, can monitor the uploading state in real time, ensures that the file processing flow is not lost, prepares for retrying and breakpoint continuous transmission, realizes high efficiency and stability of file uploading, and simultaneously improves the user experience.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a file uploading method and apparatus, an electronic device, and a storage medium.
Background
At present, videos and pictures uploaded from front pages are usually small files of no more than 20M, and are uploaded directly without fragmentation. And for the file with large data volume, a fragment uploading mode is adopted. The uploading interruption situation often occurs in the uploading of large files, and in order to realize breakpoint continuous transmission, an event model is generally adopted for subscription and publishing, so that the file fragmentation processing condition is monitored in real time, and the file processing flow is not lost.
The method is suitable for uploading files with small data volume without fragmentation, but the uploading time is prolonged along with the increase of file data, and the network environment has instability, so that the probability of file uploading failure is increased, and therefore, a fragmentation uploading mode is usually adopted for uploading large files. Under the mode of realizing file breakpoint transmission by using a subscription and publication mode, the used event model is divided into event monitoring, event triggering and processing call-back and event destruction, the three parts are always independently present in different files, if the management is improper, the problem of memory leakage of a browser is caused, the processing process is complicated, and stable and efficient multi-file parallel processing cannot be realized.
Disclosure of Invention
The invention provides a file uploading method and device, which are used for solving the problems that the process of monitoring a file processing flow is complicated and the memory is leaked due to improper management in the prior art, realizing high efficiency and stability of file uploading and improving user experience.
The invention provides a file uploading method, which comprises the following steps:
receiving a file uploading instruction;
traversing the file bidirectional linked list, and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files;
and sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
According to the file uploading method provided by the invention, the method further comprises the following steps: under the condition of receiving a file uploading instruction, if a newly added file is received, generating a proxy object of the newly added file;
judging whether an abort file exists in the file doubly-linked list or not according to the state data;
if yes, adding the proxy object of the newly added file to the front of the first suspension file;
and if not, adding the proxy object of the newly added file to the tail part of the file doubly linked list.
According to the file uploading method provided by the invention, the state data comprises: the method comprises the steps of a to-be-processed fragment queue, a success queue, a failure queue, a fragment total number, a termination identifier and a file unique identifier;
according to the state data of the file to be uploaded, sending the fragment data corresponding to the file to be uploaded to a server, and the method comprises the following steps:
determining the state of the file to be uploaded according to the state data of the file to be uploaded;
if the state of the file to be uploaded is a suspension state, stopping accessing the subsequent proxy object;
if the state of the file to be uploaded is a non-suspension state and the total number of fragments of the file to be uploaded is 1, directly uploading fragment data corresponding to the file to be uploaded to a server, if the file to be uploaded is successfully uploaded, storing the fragment data to a success queue of the agent object, and if the file to be uploaded is failed, storing the fragment data to a failure queue of the agent object;
if the state of the file to be uploaded is a non-suspension state, the total number of fragments of the file to be uploaded is greater than 1, and the unique file identifier is null, setting the state of the file to be uploaded to be a locking state, and continuing to access the proxy object of the next file to be uploaded;
and if the state of the file to be uploaded is a non-suspension state, the total number of fragments of the file to be uploaded is greater than 1, and a unique file identifier exists, sending fragment data corresponding to the file to be uploaded to a server according to the resource pool capacity.
According to the file uploading method provided by the invention, the step of sending the fragment data corresponding to the file to be uploaded to a server according to the resource pool capacity comprises the following steps:
judging whether the number of the fragments to be processed corresponding to the file to be uploaded is less than the capacity of an available resource pool or not;
if yes, uploading the fragment data corresponding to the file to be uploaded to a server one by one, if the fragment data are uploaded successfully, storing the fragment data to a success queue of the proxy object, and if the fragment data are uploaded unsuccessfully, storing the fragment data to a failure queue of the proxy object;
if not, the fragment data corresponding to the file to be uploaded is uploaded to a server through an available resource pool according to the current available resource pool capacity, if the uploading is successful, the fragment data is stored to a successful queue of the proxy object, and if the uploading is failed, the fragment data is stored to a failed queue of the proxy object.
According to the file uploading method provided by the invention, the method further comprises the following steps:
and after the fragment data corresponding to the file to be uploaded is uploaded, deleting the file to be uploaded under the condition that the fragment data corresponding to the file to be uploaded is judged to be successfully uploaded according to the success queue of the file to be uploaded.
According to the file uploading method provided by the invention, the method further comprises the following steps:
receiving a suspension instruction of a target file;
searching a proxy object corresponding to the target file from the head of the file doubly linked list backwards;
and modifying the abort identifier of the proxy object into true, and removing the proxy object from the current position and inserting the proxy object into the tail part of the file doubly-linked list.
According to the file uploading method provided by the invention, the method further comprises the following steps:
receiving a retry instruction of a target file;
finding a proxy object corresponding to the target file in the file doubly linked list;
and copying the fragment data in the agent object failure queue to a to-be-processed fragment queue one by one, and moving out the agent object from the current position and inserting the agent object into the head of the file doubly linked list.
The invention also provides a file uploading device, comprising:
the external instruction receiving module is used for receiving a file uploading instruction;
the file traversing linked list module is used for traversing the file bidirectional linked list and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files;
and the fragment data sending module is used for sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
According to the file uploading device provided by the invention, the traversal file linked list module is further configured to:
under the condition of receiving a file uploading instruction, if a newly added file is received, generating a proxy object of the newly added file;
judging whether an abort file exists in the file doubly-linked list or not according to the state data;
if yes, adding the proxy object of the newly added file to the front of the first suspension file;
and if not, adding the proxy object of the newly added file to the tail part of the file doubly linked list.
According to a file uploading device provided by the present invention, the status data includes: the method comprises the steps of a to-be-processed fragment queue, a success queue, a failure queue, a fragment total number, a termination identifier and a file unique identifier;
the fragment data sending module is further configured to:
determining the state of the file to be uploaded according to the state data of the file to be uploaded;
if the state of the file to be uploaded is a suspension state, stopping accessing subsequent proxy objects;
if the state of the file to be uploaded is a non-suspension state and the total number of fragments of the file to be uploaded is 1, fragment data corresponding to the file to be uploaded is directly uploaded to a server, if the file to be uploaded is successfully uploaded, the fragment data is stored to a success queue of the proxy object, and if the file to be uploaded is failed, the fragment data is stored to a failure queue of the proxy object;
if the state of the file to be uploaded is a non-suspension state, the total number of fragments of the file to be uploaded is greater than 1, and the unique file identifier is null, setting the state of the file to be uploaded to be a locking state, and continuing to access the proxy object of the next file to be uploaded;
and if the state of the file to be uploaded is a non-suspension state, the total number of fragments of the file to be uploaded is greater than 1, and a unique file identifier exists, sending fragment data corresponding to the file to be uploaded to a server according to the resource pool capacity.
According to the file uploading device provided by the present invention, the fragment data sending module is further configured to:
judging whether the number of the fragments to be processed corresponding to the file to be uploaded is less than the capacity of an available resource pool or not;
if yes, uploading the fragment data corresponding to the file to be uploaded to a server one by one, if the uploading is successful, storing the fragment data to a success queue of the proxy object, and if the uploading is failed, storing the fragment data to a failure queue of the proxy object;
if not, the fragment data corresponding to the file to be uploaded is uploaded to a server through an available resource pool according to the current available resource pool capacity, if the uploading is successful, the fragment data is stored to a successful queue of the proxy object, and if the uploading is failed, the fragment data is stored to a failed queue of the proxy object.
According to the file uploading device provided by the invention, the traversal file linked list module is further configured to:
and after the fragment data corresponding to the file to be uploaded is uploaded, deleting the file to be uploaded under the condition that the fragment data corresponding to the file to be uploaded is judged to be successfully uploaded according to the success queue of the file to be uploaded.
According to the file uploading device provided by the invention, the traversal file linked list module is further configured to:
receiving a suspension instruction of a target file;
searching a proxy object corresponding to the target file from the head of the file doubly linked list backwards;
and modifying the abort identifier of the proxy object into true, and removing the proxy object from the current position and inserting the proxy object into the tail part of the file doubly-linked list.
According to the file uploading device provided by the invention, the traversal file linked list module is further configured to:
receiving a retry instruction of a target file;
finding a proxy object corresponding to the target file in the file doubly linked list;
and copying the fragment data in the agent object failure queue to a to-be-processed fragment queue one by one, and moving out the agent object from the current position and inserting the agent object into the head of the file doubly linked list.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the file upload method as any one of the above.
The present invention also provides a computer program product comprising computer program/instructions which, when executed by a processor, implement the steps of the file upload method as described in any of the above.
According to the file uploading method and device, the file uploading instruction is received, the file bidirectional linked list is traversed, the proxy object corresponding to the file to be uploaded is determined, and then the fragment data corresponding to the file to be uploaded is sent to the server according to the state data of the file to be uploaded. The method uses a bidirectional linked list structure to carry out centralized and unified scheduling and management on all files to be processed, realizes simple and convenient parallel processing of multiple files, ensures that the fragment uploading process does not need to depend on a publishing and subscribing mode any more, can monitor the uploading state in real time, ensures that the file processing flow is not lost, prepares for retry and breakpoint continuous transmission, realizes high efficiency and stability of file uploading, and simultaneously improves user experience.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a file uploading method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a file storage structure according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of sending fragmented data to a server according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of sending fragmented data to a server according to resource pool capacity according to an embodiment of the present invention;
fig. 5 is a schematic view of a processing flow for receiving a new file according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating a process for receiving an abort instruction according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating a process for receiving a call-out instruction according to an embodiment of the present invention;
FIG. 8 is a flowchart illustrating a process for receiving a delete command according to an embodiment of the present invention;
FIG. 9 is a flowchart illustrating a process for receiving a retry order according to an embodiment of the invention;
fig. 10 is a schematic structural diagram of a file uploading device provided in an embodiment of the present invention;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
Fig. 1 is a schematic flowchart of a file uploading method according to an embodiment of the present invention, and as shown in fig. 1, the method includes:
The invention generates a corresponding proxy object for each file to be uploaded, and the proxy object is used for recording information such as the processing result, the file processing progress, the file interruption identification and the like of each file fragment. The method has the advantages that all files to be processed are intensively and uniformly scheduled and managed by using the bidirectional linked list structure, so that the simple and convenient parallel processing of multiple files is realized, the fragment uploading process does not need to depend on a publishing subscription mode any more, the uploading state can be monitored in real time, the file processing flow is not lost, and the preparation is made for retry and breakpoint continuous transmission.
FIG. 2 is a diagram illustrating a file storage structure according to an embodiment of the present invention, as shown in FIG. 2, in which all the file proxy objects are concatenated using a doubly linked list. Each proxy object stores state data and fragment data of a file, wherein the state data specifically comprises: the total number of fragments, a pending fragment queue, a success queue, a failure queue, an abort identifier, a file unique identifier, and the like. The total number of the fragments records the number of the fragments of a file, the to-be-processed fragment queue stores all fragment data to be uploaded of the file, the successful queue stores the successfully uploaded fragment data in the file, the failure queue stores the unsuccessfully uploaded fragment data in the file, the suspension identifier records whether the current fragment belongs to the suspended uploading state, the unique file identifier is used for distinguishing different files, and the identifier is carried by each file fragment for uploading.
Fig. 3 is a schematic flow diagram illustrating a process of sending fragmented data to a server according to an embodiment of the present invention, and as shown in fig. 3, a process of sending fragmented data corresponding to a file to be uploaded to the server according to status data of the file to be uploaded includes:
Specifically, if the abort identifier of the file to be uploaded is true, it indicates that the current file has stopped being uploaded. Because all the suspension files are placed at the tail part of the file linked list, the subsequent files of the suspension files are in the suspension state and do not need to be processed continuously.
Specifically, if the current file to be uploaded is not suspended, the file can be directly uploaded without fragmentation due to small data volume of the file. It should be noted that the processing object of the present invention is mainly a large file that needs to be uploaded in fragments, but the present invention is also applicable to small files, and a proxy object needs to be generated and linked to a file linked list containing all files to be uploaded, except that the total number of fragments of all small files is 1 and all types of queues are 1 when the length is longest.
And step 360, judging whether the unique file identifier is empty, if so, executing step 370, and if not, executing step 380.
The file unique identifier is used for identifying the file to which the fragment belongs, and each fragment is required to be uploaded with the identifier, so that any fragment owned by the file which does not obtain the unique identifier cannot be sent.
And 380, sending the fragment data corresponding to the file to be uploaded to a server according to the resource pool capacity.
For a large file with a file unique identifier, the method can be suitable for uploading partial fragment data of the file by the current resource pool capacity. Where the resource pool capability is the number of channels that send an XHR request (XMLHttpRequest) to the server. Generally, browsers have an upper limit on XHR requests processed simultaneously for the same domain name, and the request upper limits of different browsers are different.
According to one embodiment of the invention, the resource pool capability can be flexibly configured.
Generally, when a large file is uploaded in a fragmented way or in a large scale, the page flow is blocked. The browser has an upper limit on XHR requests which can be processed at the same time in the same domain name, and if all upload requests are sent at one time, the request processing capability of the browser in the same domain name is occupied by the request of uploading files, so that a page cannot respond to user operation requiring asynchronous data in time, and great inconvenience is brought to the use of users. The invention can realize the request flow limitation by configuring the resource pool capacity, namely when the request quantity is more in the same time period, the request processing capacity of the same domain name of the browser is not fully occupied, one sixth of the request processing capacity is reserved to deal with the user operation needing asynchronous data, when the user operation is not available, the reserved request processing capacity can be cancelled, and all resources are used for processing the file uploading task. Further, the file fragment being uploaded occupies a part of channels in the resource pool capacity, so that other files to be uploaded cannot use all transmission channels, and only can upload a part of file fragments thereof by using the available resource pool capacity, where the available resource pool capacity refers to that the initial resource pool capacity is reduced to the current remaining available channel number due to the occupied request transmission channel in the fragment uploading process.
Fig. 4 is a schematic flowchart illustrating a process of sending fragmented data to a server according to a resource pool capability according to an embodiment of the present invention, as shown in fig. 4, including:
Specifically, when the fragment upload request of the file to be uploaded does not occupy the current available resource pool capacity, all the fragment data of the file to be uploaded can be uploaded, and the current available resource pool capacity is updated at the same time. And after the XHR request uploaded by the fragments is sent, receiving a response of the server, if the fragments are successfully uploaded, storing the fragment data to a successful queue of the corresponding proxy object, and if the fragments are unsuccessfully uploaded, storing the fragment data to a failed queue of the corresponding proxy object. And under the condition that the quantity of fragment uploading requests of the file to be uploaded exceeds the current available resource pool capacity, only fragment data equivalent to the current available resource capacity can be uploaded, if the uploading is successful, the fragment data is stored to a success queue of the proxy object, and if the uploading is failed, the fragment data is stored to a failure queue of the proxy object. In summary, the file uploading method provided by the invention traverses the file bidirectional linked list by receiving the file uploading instruction, determines the proxy object corresponding to the file to be uploaded, and then sends the fragment data corresponding to the file to be uploaded to the server according to the state data of the file to be uploaded. The method uses the agent to replace a publish-subscribe mode so as to monitor the processing of the file fragmentation, thereby realizing high efficiency and stability of file uploading and simultaneously improving user experience.
According to an embodiment of the present invention, in the case of receiving the instruction to upload the file, if a new file is received, the steps shown in fig. 5 are executed:
Specifically, a newly added file is received, information such as file type and file size is checked, and the newly added file is fragmented according to a certain rule, wherein the number of fragments is usually within 10000, and the size of each fragment is within 10M. Generating proxy objects of the newly added files according to the file fragment information, wherein each proxy object stores state data and fragment data of the files, and the state data specifically comprises the following steps: the total number of fragments, a pending fragment queue, a success queue, a failure queue, an abort identifier, a file unique identifier, and the like. The total number of the fragments records the number of the fragments of a file, the to-be-processed fragment queue stores all fragment data to be uploaded of the file, the successful queue stores the successfully uploaded fragment data in the file, the failure queue stores the unsuccessfully uploaded fragment data in the file, the suspension identifier records whether the current fragment belongs to the suspended uploading state, the unique file identifier is used for distinguishing different files, and the identifier is carried by each file fragment for uploading.
The total number of the fragments of the newly added file is the number of the fragments of the file, the queue to be processed stores all fragment data, the success queue and the failure queue are empty by default, the suspension identifier is false by default, and the unique identifier of the file is an automatically generated serial number.
And 540, linking the proxy object of the newly added file to the tail part of the existing file linked list through the bidirectional pointer. And subsequently, the uploading fragmentation request can be sent to the server according to the uploading sequence, and the expectation of the user is met.
According to an embodiment of the present invention, the process of sending the fragment data corresponding to the file to be uploaded to the server according to the status data of the file to be uploaded is the same as the above steps 310 to 350.
According to an embodiment of the present invention, the process of sending the fragmented data to the server according to the resource pool capability is the same as the above steps 410-430.
According to an embodiment of the present invention, the file uploading method further includes:
and after the fragment data corresponding to the file to be uploaded is uploaded, deleting the file to be uploaded under the condition that the fragment data corresponding to the file to be uploaded is judged to be successfully uploaded according to the success queue of the file to be uploaded.
Specifically, the situation that the to-be-processed fragment queue and the failure queue in the proxy object are both empty, and the length of the success queue is equal to the total number of fragments indicates that all fragments of the to-be-uploaded file are uploaded successfully, all fragment data can be taken out of the success queue, the fragment data is synthesized into a complete file by means of other services, and the file is deleted from the file linked list.
Fig. 6 is a flowchart illustrating a process of receiving an abort instruction according to an embodiment of the present invention, as shown in fig. 6, including:
And step 620, searching the proxy object corresponding to the target file from the head of the file doubly linked list backwards.
The target file can be found in a mode of searching the target file from front to back, the suspended file is placed at the tail of the linked list, the non-suspended module is processed preferentially, unnecessary judgment can be reduced, and the main thread capability of javascript is not wasted.
Fig. 7 is a flowchart illustrating a process of receiving a call-out command according to an embodiment of the present invention, as shown in fig. 7, including:
And 720, searching the proxy object corresponding to the target file from the tail part of the file doubly linked list forward.
The method comprises the steps of setting the called files at the head of a linked list, and preferentially processing all the files to be uploaded to give timely feedback to a user. For the file which is continuously transmitted at the breakpoint, the priority processing authority cannot be obtained usually, mature PC side applications are only put at the tail of a queue after the file is interrupted, if priority processing is needed, the previous files need to be interrupted one by one, the breakpoint transmission of pure webpages is carried out without any processing on the request priority, the files which are just waken by a user cannot be preferentially processed after interruption, and the processing mode does not meet the expectation of the user. The invention uses the proxy object to manage a single file, and realizes the dynamic adjustment of the uploading priority of the file by adjusting the position of the proxy object in the linked list.
Fig. 8 is a schematic flow chart of a process for receiving a delete instruction according to an embodiment of the present invention, as shown in fig. 8, including:
And step 820, searching the proxy object corresponding to the target file from the head of the file doubly linked list backwards.
Fig. 9 is a flowchart illustrating a process of receiving a retry instruction according to an embodiment of the invention, as shown in fig. 9, including:
at step 910, a retry instruction of the target file is received.
And 920, finding a proxy object corresponding to the target file in the file doubly linked list.
The files which are continuously transmitted at the break point are arranged at the head of the linked list, and are preferentially processed in all the files to be uploaded, so that the files are fed back to the user in time, and the psychological expectation of the user is met.
The following describes the file uploading device provided by the present invention, and the file uploading device described below and the file uploading method described above may be referred to correspondingly.
Fig. 10 is a schematic structural diagram of a file uploading apparatus according to an embodiment of the present invention, and as shown in fig. 10, the apparatus includes:
and an external instruction receiving module 1010, configured to receive an instruction to upload a file.
A traverse file linked list module 1020, configured to traverse a file bidirectional linked list, and determine a proxy object corresponding to a file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files.
A fragment data sending module 1030, configured to send the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
The file uploading device provided by the invention is used for traversing the file bidirectional linked list by receiving the file uploading instruction, determining the proxy object corresponding to the file to be uploaded, and then sending the fragment data corresponding to the file to be uploaded to the server according to the state data of the file to be uploaded. The device uses a bidirectional linked list structure to carry out centralized unified scheduling and management on all files to be processed, realizes simple and convenient parallel processing of multiple files, ensures that the fragment uploading process does not need to rely on a publishing subscription mode any more and can monitor the uploading state in real time, ensures that the file processing flow is not lost, prepares for retry and breakpoint continuous transmission, realizes high efficiency and stability of file uploading, and simultaneously improves user experience.
According to an embodiment of the present invention, the traverse file linked list module 1020 is further configured to:
under the condition of receiving a file uploading instruction, if a newly added file is received, generating a proxy object of the newly added file;
judging whether an abort file exists in the file doubly-linked list or not according to the state data;
if yes, adding the proxy object of the newly added file to the front of the first suspension file;
and if not, adding the proxy object of the newly added file to the tail part of the file doubly linked list.
According to one embodiment of the invention, the status data comprises: the method comprises the steps of a to-be-processed fragment queue, a success queue, a failure queue, a fragment total number, a suspension identifier and a file unique identifier;
the transmit sliced data module 1030 is further configured to:
determining the state of the file to be uploaded according to the state data of the file to be uploaded;
if the state of the file to be uploaded is a suspension state, stopping accessing the subsequent proxy object;
if the state of the file to be uploaded is a non-suspension state and the total number of fragments of the file to be uploaded is 1, directly uploading fragment data corresponding to the file to be uploaded to a server, if the file to be uploaded is successfully uploaded, storing the fragment data to a success queue of the agent object, and if the file to be uploaded is failed, storing the fragment data to a failure queue of the agent object;
if the file to be uploaded is in a non-suspension state, the total number of the fragments of the file to be uploaded is greater than 1, and a unique file identifier exists, transmitting fragment data corresponding to the file to be uploaded to a server according to the resource pool capacity;
and if the state of the file to be uploaded is a non-suspension state, the total number of the fragments of the file to be uploaded is greater than 1, and the unique identifier of the file is null, setting the state of the file to be uploaded to be a locking state, and continuously accessing the proxy object of the next file to be uploaded.
According to an embodiment of the present invention, the transmitting fragmented data module 1030 is further configured to:
judging whether the number of the fragments to be processed corresponding to the file to be uploaded is less than the capacity of an available resource pool or not;
if yes, uploading the fragment data corresponding to the file to be uploaded to a server one by one, if the uploading is successful, storing the fragment data to a success queue of the proxy object, and if the uploading is failed, storing the fragment data to a failure queue of the proxy object;
if not, the fragment data corresponding to the file to be uploaded is uploaded to a server through an available resource pool according to the current available resource pool capacity, if the uploading is successful, the fragment data is stored to a successful queue of the proxy object, and if the uploading is failed, the fragment data is stored to a failed queue of the proxy object.
According to an embodiment of the present invention, the traverse file linked list module 1020 is further configured to:
and after the fragment data corresponding to the file to be uploaded is uploaded, deleting the file to be uploaded under the condition that the fragment data corresponding to the file to be uploaded is judged to be successfully uploaded according to the success queue of the file to be uploaded.
According to an embodiment of the present invention, the traverse file linked list module 1020 is further configured to:
receiving a suspension instruction of a target file;
searching a proxy object corresponding to the target file from the head of the file doubly linked list backwards;
and modifying the abort identifier of the proxy object into true, and removing the proxy object from the current position and inserting the proxy object into the tail part of the file doubly-linked list.
According to an embodiment of the present invention, the traverse file linked list module 1020 is further configured to:
receiving a call-up instruction of a target file;
searching a proxy object corresponding to the target file from the tail part of the file bidirectional linked list forward;
and modifying the abort identifier of the proxy object into false, and removing the proxy object from the current position and inserting the proxy object into the head of the file doubly linked list.
According to an embodiment of the present invention, the traverse file linked list module 1020 is further configured to:
receiving a deleting instruction of a target file;
searching a proxy object corresponding to the target file from the head of the file doubly linked list backwards;
and removing the proxy object from the file doubly linked list, and releasing the memory space occupied by the proxy object.
According to an embodiment of the present invention, the traverse file linked list module 1020 is further configured to:
receiving a retry instruction of a target file;
finding a proxy object corresponding to the target file in the file doubly linked list;
and copying the fragment data in the agent object failure queue to a to-be-processed fragment queue one by one, and moving out the agent object from the current position and inserting the agent object into the head of the file doubly linked list.
Fig. 11 illustrates a physical structure diagram of an electronic device, and as shown in fig. 11, the electronic device may include: a processor (processor)1110, a communication Interface (Communications Interface)1120, a memory (memory)1130, and a communication bus 1140, wherein the processor 1110, the communication Interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. Processor 1110 may invoke logic instructions in memory 1130 to perform a file upload method comprising: receiving a file uploading instruction; traversing the file bidirectional linked list, and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files; and sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
In addition, the logic instructions in the memory 1130 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the file uploading method provided by the above methods, the method including: receiving a file uploading instruction; traversing the file bidirectional linked list, and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files; and sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the file uploading methods provided above, the method including: receiving a file uploading instruction; traversing the file bidirectional linked list, and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files; and sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. A file uploading method is characterized by comprising the following steps:
receiving a file uploading instruction;
traversing the file bidirectional linked list, and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files;
and sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
2. The file uploading method according to claim 1, further comprising: under the condition of receiving a file uploading instruction, if a newly added file is received, generating a proxy object of the newly added file;
judging whether an abort file exists in the file bidirectional linked list according to the state data;
if yes, adding the proxy object of the newly added file to the front of the first suspension file;
and if not, adding the proxy object of the newly added file to the tail part of the file doubly linked list.
3. The file uploading method according to claim 1, wherein the status data comprises: the method comprises the steps of a to-be-processed fragment queue, a success queue, a failure queue, a fragment total number, a termination identifier and a file unique identifier;
according to the state data of the file to be uploaded, sending the fragment data corresponding to the file to be uploaded to a server, and the method comprises the following steps:
determining the state of the file to be uploaded according to the state data of the file to be uploaded;
if the state of the file to be uploaded is a suspension state, stopping accessing the subsequent proxy object;
if the state of the file to be uploaded is a non-suspension state and the total number of fragments of the file to be uploaded is 1, directly uploading fragment data corresponding to the file to be uploaded to a server, if the file to be uploaded is successfully uploaded, storing the fragment data to a success queue of the agent object, and if the file to be uploaded is failed, storing the fragment data to a failure queue of the agent object;
if the file to be uploaded is in a non-suspension state, the total number of the fragments of the file to be uploaded is greater than 1, and the unique file identifier is null, setting the state of the file to be uploaded to be a locking state, and continuing to access the proxy object of the next file to be uploaded;
and if the state of the file to be uploaded is a non-suspension state, the total number of fragments of the file to be uploaded is greater than 1, and a unique file identifier exists, sending fragment data corresponding to the file to be uploaded to a server according to the resource pool capacity.
4. The file uploading method according to claim 3, wherein the sending the fragmented data corresponding to the file to be uploaded to a server according to the resource pool capacity comprises:
judging whether the number of the fragments to be processed corresponding to the file to be uploaded is less than the capacity of an available resource pool or not;
if yes, uploading the fragment data corresponding to the file to be uploaded to a server one by one, if the uploading is successful, storing the fragment data to a success queue of the proxy object, and if the uploading is failed, storing the fragment data to a failure queue of the proxy object;
if not, the fragment data corresponding to the file to be uploaded is uploaded to a server through an available resource pool according to the current available resource pool capacity, if the uploading is successful, the fragment data is stored to a successful queue of the proxy object, and if the uploading is failed, the fragment data is stored to a failed queue of the proxy object.
5. The file uploading method according to claim 4, further comprising:
and after the fragment data corresponding to the file to be uploaded is uploaded, deleting the file to be uploaded under the condition that the fragment data corresponding to the file to be uploaded is judged to be successfully uploaded according to the success queue of the file to be uploaded.
6. The file uploading method according to claim 3, further comprising:
receiving a suspension instruction of a target file;
searching a proxy object corresponding to the target file from the head of the file doubly linked list backwards;
and modifying the abort identifier of the proxy object into true, and removing the proxy object from the current position and inserting the proxy object into the tail part of the file doubly-linked list.
7. The file uploading method according to claim 3, further comprising:
receiving a retry instruction of a target file;
finding a proxy object corresponding to the target file in the file doubly linked list;
and copying the fragment data in the agent object failure queue to a fragment queue to be processed one by one, and moving out the agent object from the current position and inserting the agent object into the head of the file doubly linked list.
8. A file uploading apparatus, comprising:
the external instruction receiving module is used for receiving a file uploading instruction;
the file traversing linked list module is used for traversing the file bidirectional linked list and determining a proxy object corresponding to the file to be uploaded; the file bidirectional linked list comprises a plurality of proxy objects which are sequentially linked through bidirectional pointers, and the proxy objects comprise state data and fragment data of corresponding files;
and the fragment data sending module is used for sending the fragment data corresponding to the file to be uploaded to a server according to the state data of the file to be uploaded.
9. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the file uploading method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110778008.0A CN113726842B (en) | 2021-07-09 | 2021-07-09 | File uploading method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110778008.0A CN113726842B (en) | 2021-07-09 | 2021-07-09 | File uploading method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113726842A CN113726842A (en) | 2021-11-30 |
| CN113726842B true CN113726842B (en) | 2022-07-05 |
Family
ID=78673097
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110778008.0A Active CN113726842B (en) | 2021-07-09 | 2021-07-09 | File uploading method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113726842B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114650287B (en) * | 2022-04-06 | 2024-04-30 | 中国工商银行股份有限公司 | Breakpoint continuous method, device and system for data file |
| CN114780003A (en) * | 2022-05-23 | 2022-07-22 | 北京达佳互联信息技术有限公司 | Material uploading method and related equipment |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9413825B2 (en) * | 2007-10-31 | 2016-08-09 | Emc Corporation | Managing file objects in a data storage system |
| US10547664B2 (en) * | 2013-03-21 | 2020-01-28 | Oracle International Corporation | Enable uploading and submitting multiple files |
| CN105141696A (en) * | 2015-09-17 | 2015-12-09 | 成都华为技术有限公司 | File uploading method and device |
| CN107872489B (en) * | 2016-09-28 | 2020-06-05 | 杭州海康威视数字技术股份有限公司 | File slice uploading method and device and cloud storage system |
| CN106453668A (en) * | 2016-12-26 | 2017-02-22 | 网宿科技股份有限公司 | Reverse file uploading method and system |
| CN107172131B (en) * | 2017-04-26 | 2020-12-04 | 腾讯科技（深圳）有限公司 | File uploading method and device |
| CN110650164B (en) * | 2018-06-26 | 2021-03-05 | 马上消费金融股份有限公司 | File uploading method and device, terminal and computer storage medium |
| CN110300151B (en) * | 2019-05-22 | 2022-02-11 | 深圳壹账通智能科技有限公司 | Data file uploading method and system |
| CN111897668B (en) * | 2020-09-30 | 2021-01-08 | 南京拓界信息技术有限公司 | File parallel copying method and device based on linked list and computer equipment |
-
2021
- 2021-07-09 CN CN202110778008.0A patent/CN113726842B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113726842A (en) | 2021-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113726842B (en) | File uploading method and device, electronic equipment and storage medium | |
| US10552449B2 (en) | Storage constrained synchronization of shared content items | |
| US9442997B2 (en) | Idle state triggered constrained synchronization of shared content items | |
| US10908950B1 (en) | Robotic process automation system with queue orchestration and task prioritization | |
| US8275787B2 (en) | System for managing data collection processes | |
| JP5585062B2 (en) | Information processing apparatus, information processing method, data management server, and data synchronization system | |
| EP3495981B1 (en) | Directory deletion method and device, and storage server | |
| EP3273664B1 (en) | Data processing method and device, server, and controller | |
| CN113238861A (en) | Task execution method and device | |
| CN111400041A (en) | Server configuration file management method and device and computer readable storage medium | |
| CN113687781A (en) | Method, device, equipment and medium for pulling up thermal data | |
| US8482767B2 (en) | Print job submission with sleep mechanism | |
| US11252121B2 (en) | Message sending method and terminal device | |
| US20240129251A1 (en) | Data processing method and apparatus, computer device, and readable storage medium | |
| CN114296953A (en) | Multi-cloud heterogeneous system and task processing method | |
| KR101845195B1 (en) | Multiple Resource Subscriptions Association Method in an M2M system | |
| CN112540966A (en) | File synchronization method and device | |
| CN115951845A (en) | Disk management method, device, equipment and storage medium | |
| CN105740065A (en) | Method and device for controlling execution of asynchronous tasks | |
| CN107357808B (en) | Data management method, device and equipment | |
| CN110837428A (en) | Storage device management method and device | |
| CN114374657A (en) | Data processing method and device | |
| CN110941591A (en) | File deletion method, device and equipment and readable storage medium | |
| CN112688980A (en) | Resource distribution method and device, and computer equipment | |
| CN114465958B (en) | Input and output control method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220114 Address after: 100085 Floor 101 102-1, No. 35 Building, No. 2 Hospital, Xierqi West Road, Haidian District, Beijing Applicant after: Seashell Housing (Beijing) Technology Co.,Ltd. Address before: 101309 room 24, 62 Farm Road, Erjie village, Yangzhen, Shunyi District, Beijing Applicant before: Beijing fangjianghu Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |