CN105450769A - File transmission method and device - Google Patents
File transmission method and device Download PDFInfo
- Publication number
- CN105450769A CN105450769A CN201510925190.2A CN201510925190A CN105450769A CN 105450769 A CN105450769 A CN 105450769A CN 201510925190 A CN201510925190 A CN 201510925190A CN 105450769 A CN105450769 A CN 105450769A
- Authority
- CN
- China
- Prior art keywords
- subfile
- thread
- list
- sub
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a file transmission method and device. The method comprises the steps of pre-acquiring a list of split sub-files of a to-be-transmitted file, booting a plurality of sub-threads, and setting a file receiving path, retransmission number of times and retransmission time interval; locking the list of sub-files by the plurality of sub-threads, and acquiring a to-be-transmitted sub-file by each of the plurality of sub-threads, so as to allow the plurality of sub-threads to execute file transmission simultaneously; if transmission succeeds, continuing to transmit other to-be-transmitted sub-files by a corresponding sub-thread; and if transmission fails, waiting for a duration corresponding to retransmission time interval, retransmitting till the execution number of times of the transmission task reaches the retransmission number of times, but transmission still fails, returning the sub-file failing to transmit to the list of sub-files and terminating corresponding sub-thread. The method adopts multi-thread load balance and retransmission policies, thereby further improving success rate of file transmission when in breakpoint transmission of multiple threads.
Description
Technical field
The present invention relates to technical field of data transmission, particularly a kind of method of file transfer and device.
Background technology
Along with the development of science and technology, the informationalized degree of every profession and trade generally improves, and data file transmission between networks also gets more and more.
In order to ensure that data file can be transmitted by quick, stable carrying out, current file transfer in conjunction with file declustering combination, multiple threads technology, carries out transfer files in multithreading breakpoint transmission mode mostly.
Because the configuration of the server of execute file transformation task is not quite similar, and the network environment of each transmission node also different, fashion time bad, especially server configuration lower, network speed is slow or unstable time, easily there is the situation of bust this in file, thus can reduce the success rate of file transfer to a certain extent in transmitting procedure.
Summary of the invention
The invention provides a kind of method and device of file transfer, on the basis of multithreading breakpoint transmission, the success rate of file transfer can be improved further.
In order to achieve the above object, the present invention is achieved through the following technical solutions:
On the one hand, the invention provides a kind of method of file transfer, obtain the subfile list after file declustering to be transmitted in advance, start multiple sub-thread, bust this number of retries, bust this retry time interval, file reception path are set, also comprise:
S1: when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and perform S2;
S2: utilize the sub-thread of described target, transfers to the position corresponding with described file reception path by described target subfile;
S3: judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, performs S1, otherwise, perform S4;
S4: judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and perform S2, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlocks described subfile list, the sub-thread of described target terminates to perform.
Further, after S4, also comprise:
After described multiple sub-thread all terminates to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
Further, described multiple sub-thread, comprising:
The kernel that the Thread Count number of described multiple sub-thread equals the server of execute file transformation task keeps count of and subtracts 1 again.
Further,
Before subfile list after described acquisition file declustering to be transmitted, comprise further: pre-set local file path, subfile splits size;
Subfile list after described acquisition file declustering to be transmitted, comprise: split size according to described subfile, file declustering to be transmitted is become multiple subfile to be transmitted, obtain the subfile list comprising described multiple subfile to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
Further, described bust this number of retries, bust this retry time interval are set, comprise:
Arranging bust this number of retries is 2 ~ 5 times, bust this retry time interval 0 ~ 10s.
On the other hand, the invention provides a kind of device of file transfer, comprising:
Acquiring unit, for obtaining the subfile list after file declustering to be transmitted, and exports the information of described subfile list to the first processing unit;
Start unit, for starting multiple sub-thread, and exports the information of described multiple sub-thread to described first processing unit;
First setting unit, for arranging bust this number of retries, bust this retry time interval, file reception path, and the information in described file reception path is exported to transmission unit, the information of described bust this number of retries and described bust this retry time interval is exported to the 3rd processing unit;
Described first processing unit, for receiving the information of the described subfile list exported by described acquiring unit, receive the information of the described multiple sub-threads exported by described start unit, receive the signal of the transmission success exported by described second processing unit, receive the information comprising the described subfile list of the described target subfile of bust this exported by described 3rd processing unit, when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and trigger described transmission unit, and the information of described target subfile and the sub-thread of described target is exported to described transmission unit, the information of described subfile list is exported to described 3rd processing unit,
Described transmission unit, for receiving the information in the described file reception path exported by described first setting unit, receive the information of described target subfile and the sub-thread of described target exported by described first processing unit, receive the signal terminated by the wait of described 3rd processing unit output, utilize the sub-thread of described target, described target subfile is transferred to the position corresponding with described file reception path, and exports to the second processing unit the information that the sub-thread of described target transmits described target subfile;
Described second processing unit, for transmitting the information of described target subfile according to the sub-thread of described target exported by described transmission unit received, judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, trigger described first processing unit, and the signal of transmission success is exported to described first processing unit, otherwise, trigger described 3rd processing unit, and export to described 3rd processing unit the information that the signal of bust this and the sub-thread of described target transmit described target subfile;
Described 3rd processing unit, for receiving the information of described bust this number of retries and the described bust this retry time interval exported by described first setting unit, receive the information of the described subfile list exported by described first processing unit, after the signal and the sub-thread of described target that receive the bust this exported by described second processing unit transmit the information of described target subfile, judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and trigger described transmission unit, and the signal waited for and terminating is exported to described transmission unit, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlock described subfile list, the sub-thread of described target terminates to perform, and to the information of described subfile list of described first processing unit output packet containing the described target subfile of bust this.
Further, this device also comprises:
Fourth processing unit, after all terminating at described multiple sub-thread to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
Further, described start unit, specifically for starting multiple sub-thread, and the kernel that the Thread Count number of described multiple sub-thread equals the server of execute file transformation task keeps count of and subtracts 1 again.
Further, this device also comprises: the second setting unit, and for arranging local file path, subfile splits size;
Described acquiring unit, specifically for splitting size according to described subfile, becomes to comprise the subfile list of multiple subfile to be transmitted by file declustering to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
Further, described first setting unit is 2 ~ 5 times, bust this retry time interval 0 ~ 10s specifically for arranging bust this number of retries.
The invention provides a kind of method and device of file transfer, obtain the subfile list after file declustering to be transmitted in advance, start multiple sub-thread, file reception path, bust this number of retries, bust this retry time interval are set; Multiple sub-thread locks subfile list respectively, and obtains a subfile to be transmitted respectively, thus makes the execute file transmission simultaneously of the plurality of sub-thread; If transmission success, then corresponding sub-thread continues other subfiles to be transmitted of transmission; If bust this, then need to wait for the duration corresponding to bust this retry time interval, and again perform transmission, until the execution number of times of this transformation task reach bust this number of retries but still bust this time, the subfile of bust this is retracted into described subfile list and terminates corresponding sub-thread; Therefore, the present invention, by multithreading load balancing and bust this retry strategy, on the basis of multithreading breakpoint transmission, can improve the success rate of file transfer further.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the flow chart of the method for a kind of file transfer that one embodiment of the invention provides;
Fig. 2 is the flow chart of the method for the another kind of file transfer that one embodiment of the invention provides;
Fig. 3 is the schematic diagram of the device of a kind of file transfer that one embodiment of the invention provides;
Fig. 4 is the schematic diagram of the device of the another kind of file transfer that one embodiment of the invention provides.
Embodiment
For making the object of the embodiment of the present invention, technical scheme and advantage clearly; below in conjunction with the accompanying drawing in the embodiment of the present invention; technical scheme in the embodiment of the present invention is clearly and completely described; obviously; described embodiment is the present invention's part embodiment, instead of whole embodiments, based on the embodiment in the present invention; the every other embodiment that those of ordinary skill in the art obtain under the prerequisite not making creative work, all belongs to the scope of protection of the invention.
As shown in Figure 1, embodiments provide a kind of method of file transfer, can comprise the following steps:
Step 101: obtain the subfile list after file declustering to be transmitted in advance, starts multiple sub-thread, arranges bust this number of retries, bust this retry time interval, file reception path.
Step 102: when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and perform step 103.
Step 103: utilize the sub-thread of described target, transfers to the position corresponding with described file reception path by described target subfile.
Step 104: judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, performs step 102, otherwise, perform step 105.
Step 105: judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and perform step 103, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlocks described subfile list, the sub-thread of described target terminates to perform.
Embodiments provide a kind of method of file transfer, obtain the subfile list after file declustering to be transmitted in advance, start multiple sub-thread, file reception path, bust this number of retries, bust this retry time interval are set; Multiple sub-thread locks subfile list respectively, and obtains a subfile to be transmitted respectively, thus makes the execute file transmission simultaneously of the plurality of sub-thread; If transmission success, then corresponding sub-thread continues other subfiles to be transmitted of transmission; If bust this, then need to wait for the duration corresponding to bust this retry time interval, and again perform transmission, until the execution number of times of this transformation task reach bust this number of retries but still bust this time, the subfile of bust this is retracted into described subfile list and terminates corresponding sub-thread; Therefore, the present invention, by multithreading load balancing and bust this retry strategy, on the basis of multithreading breakpoint transmission, can improve the success rate of file transfer further.
In a kind of possible implementation, after completing in described file transfer to be transmitted, obtain the file transferring to corresponding position, described file reception path, so, after step 105, comprise further:
After described multiple sub-thread all terminates to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
In a kind of possible implementation, in order to make full use of the physical configuration resource of the server of execute file transformation task, ensure that the transmission speed of sub-thread reaches optimal effectiveness, so described multiple sub-thread, comprising simultaneously:
The kernel that the Thread Count number of described multiple sub-thread equals the server of execute file transformation task keeps count of and subtracts 1 again.
In a kind of possible implementation, in order to can according to the quality of the configuration condition of the server of execute file transformation task and network environment, adjustment subfile splits size flexibly, and convenience file is transmitted the merging treatment of rear subfile simultaneously, so,
Before subfile list after described acquisition file declustering to be transmitted, comprise further: pre-set local file path, subfile splits size;
Subfile list after described acquisition file declustering to be transmitted, comprise: split size according to described subfile, file declustering to be transmitted is become multiple subfile to be transmitted, obtain the subfile list comprising described multiple subfile to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
In a kind of possible implementation, in order to can be different according to the good and bad situation of network environment, improve the success rate of file transfer as much as possible, so, described bust this number of retries, bust this retry time interval are set, comprise:
Arranging bust this number of retries is 2 ~ 5 times, bust this retry time interval 0 ~ 10s.
For making the object, technical solutions and advantages of the present invention clearly, below in conjunction with drawings and the specific embodiments, the present invention is described in further detail.
As shown in Figure 2, embodiments provide the method for another kind of file transfer, the method can comprise the following steps:
Step 201: local file path is set, subfile splits size, Thread Count, file reception path, bust this number of retries, bust this retry time interval.
Particularly, arrange kernel that Thread Count number equals the server of execute file transformation task to keep count of and subtract 1 again.
Particularly, arranging bust this number of retries is 2 ~ 5 times, bust this retry time interval 0 ~ 10s.
For example, suppose that an existing server A needs to transfer in server B by network by a system compresses file X, wherein, can be server A to be uploaded onto the server file X B by network, also can be that server B is by network download file X from server A.Such as, in the present embodiment, server A needs to be uploaded onto the server by file X B by network.
First, the local file path of server A and the file reception path of server B can be set.Wherein, the reception URL (UniformResourceLocator, URL(uniform resource locator)) of server and file reception catalogue are normally fixed, and arrange by appointment application server.
Step 202: split size according to described subfile, file declustering to be transmitted is become multiple subfile to be transmitted, obtain the subfile list comprising described multiple subfile to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
Usually, network speed slow and unstable time, suitably can reduce subfile and split size, to reduce the stability of the time needed for transmission, the probability reducing bust this, the number of times reducing data re-transmission, raising transmission, improve the effective rate of utilization of network simultaneously; Fast and when stablize, suitably can increasing subfile and split size, time loss caused during to reduce file declustering and to combine at network speed.Obviously, increase the degree that subfile splits size, should ensure that the quantity of the subfile after file declustering is not less than Thread Count, in order to avoid cause the wasting of resources of unnecessary thread.
In addition, according to the different application demand of user, it can be fixing that subfile splits size, also can be unfixed.Wherein, if subfile splits size and fixes, can split with fixed size in order file, obtain equal-sized multiple subfile, without the need to recording the information such as respective identification and relative ranks of each subfile, and after subfile transmission success, only need to combine in order, it is comparatively convenient to operate.If subfile splits size and is controlled by professional operator, the multiple subfiles differed in size can be obtained, need the information such as respective identification and relative ranks recording each subfile, the combination of subfile bothers relatively, but professional operator can according to the information of each subfile, carry out relevant speciality operation and process, therefore practicality is relatively stronger.Such as, in the present embodiment, because the common files passe that is transmitted as of file X operates, therefore chooser file declustering size is fixed.
By this implementation, can according to the different application demand of user, and in conjunction with network environment situation, selectively determine that subfile splits size, with reach reduce file declustering and combination time caused time loss, reduce time needed for transmission, reduce the probability of bust this, reduce the number of times of data re-transmission, improve the stability of transmission, improve the effect of the effective rate of utilization of network.
Such as, in the present embodiment, because network speed is comparatively slow and unstable, therefore suitably reduce subfile fractionation size, file X is split into 20 equal-sized subfiles, namely subfile 1, subfile 2 ..., subfile 20, and will the subfile list storage of these 20 subfiles be comprised to the position corresponding with local file path in server A.
Step 203: the subfile obtaining the Successful transmissions of the position corresponding with described file reception path, and the target subfile corresponding with the subfile of described Successful transmissions in described subfile list is removed.
In the present embodiment, multiple subfile can be become to facilitate transmission file declustering by file declustering technology, if this transmission resumes on the unsuccessful transformation task basis completed before, so can before the transmission of execution subfile, first the subfile of Successful transmissions is removed from subfile list, without the need to repeating transmission, thus effectively can reduce and repeat to transmit waste to Internet resources, to facilitate user to stop transformation task according to self-demand and network environment situation, and execute file resumes operation at any time.
Step 204: according to described Thread Count, creates multiple sub-thread, and starts described multiple sub-thread.
Usually, low, the network speed of server configuration slow and unstable time, suitably can reduce Thread Count, with the stable operation of Deterministic service device, and improve speed and the stability of file transfer; Fast and when stablize, suitably can increase Thread Count at high, the network speed of server configuration, to improve server resource utilance, and raising network resource utilization and file transfer speed.
In addition, even if high, the network speed of server configuration is fast and when stablizing, Thread Count is also not The more the better, thread is more, coordinates physical resource needed for thread also more, server to bear pressure also larger, so, Thread Count fewer than kernel server number 1 can be set.Such as, in the present embodiment, suppose that the interior check figure of server A and server B is four cores, therefore can to arrange Thread Count be 3.
For example, arranging Thread Count is 3, and creates 3 sub-threads, is respectively sub-thread 1, sub-thread 2, sub-thread 3, starts this 3 sub-threads.
In the present embodiment, user can be made according to the server configuration degree performing transformation task, and in conjunction with network environment situation, selectively determine Thread Count number, with while making full use of server resource allocation, the stable operation of Deterministic service device, while making full use of Internet resources, ensures speed and the stability of file transfer.
Step 205: when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and perform step 206.
Wherein, the acquisition of target subfile can be the arbitrary subfile of random acquisition, also can be obtain subfile by the clooating sequence of subfile in subfile list.Such as, in an embodiment, selecting sequence obtains subfile.
For example, after startup 3 sub-threads, subfile list is in released state, and 3 sub-threads all do not perform transformation task, can make arbitrary sub-thread, as sub-thread 1, the subfile that locks list, and obtain the subfile 1 in subfile list, subfile 1 is removed from subfile list, unlock subfile list again, and perform subsequent file transmission task.After sub-thread 1 unlocks subfile list, sub-thread 2 can continue the sub-thread list that locks, and obtain subfile 2, same, sub-thread 3 obtains sub-thread 3, therefore can make these three sub-threads execute file transformation task simultaneously.
In the present embodiment, to have fixing file transmission task amount different from sub-thread, after arbitrary sub-thread completes a file transmission task, can continue lock subfile list and obtain subfile, unlock subfile list again, continue to obtain subfile to make other sub-threads, thus make multiple sub-thread execute file transformation task simultaneously, until all subfiles are transmitted, therefore this way of realization effectively prevent when the file transmission task amount of stator thread, the parton thread fulfiling file transmission task amount ahead of schedule is in wait state all the time, until whole sub-thread all completes the task amount of each self-retaining, the situation of the sub-thread resources waste caused, therefore, the present embodiment passes through load balancing, the load balancing between multiple sub-thread can be ensured to greatest extent, improve sub-thread resources utilance, contribute to the success rate improving file transfer.
In the present embodiment, if the equal Successful transmissions of all subfiles, then subfile list is empty, and sub-thread locks after subfile list, through judging that subfile list is as empty, therefore the list of sub-thread UNLOCKS subfile, and this sub-thread terminates to perform.After treating that all sub-threads all terminate to perform, the whole subfiles be successfully transmitted in server B can be combined, or perform other subsequent operations.
Step 206: utilize the sub-thread of described target, transfers to the position corresponding with described file reception path by described target subfile.
Step 207: judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, performs step 205, otherwise, perform step 208.
In the present embodiment, if subfile transmission success, such as, utilizing after subfile 1 is successfully transmitted in server B by sub-thread 1 from server A, sub-thread 1 can when subfile list be in released state, continue lock subfile list and obtain other subfiles, execute file transformation task again.
Step 208: judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and perform step 206, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlocks described subfile list, the sub-thread of described target terminates to perform.
Usually, user usually can be made according to self-demand, server configuration degree, and in conjunction with network environment situation, bust this number of retries and bust this retry time interval are selectively set.Such as, when user does not want there is a too much manual intervention to document transmission process, relatively can increase bust this number of retries and bust this retry time interval, perform to make transformation task automatic stabilisation; When user thinks to find mistake, eliminating abnormal conditions as early as possible, bust this number of retries and bust this retry time interval relatively can be reduced.Poor in network stabilization, and when time when interruption situation appears in network is at every turn longer, can relatively increase bust this retry time interval, to avoid too much invalid retry; More stable at network, but when network speed is changeable, relatively can increases bust this number of retries, reduce bust this retry time interval.
In the present embodiment, by file transfer failure retry strategy, user can be made according to self-demand, server configuration degree, and in conjunction with network environment situation, bust this number of retries and bust this retry time interval are selectively set, to ensure the stability of document transmission process, promote the efficiency of file transfer, improve the success rate of file transfer.
Therefore, in the present embodiment, can arrange bust this number of retries is 2 ~ 5 times, bust this retry time interval 0 ~ 10s.Such as, arranging bust this number of retries is 3 times, bust this retry time interval 5s, and wherein, bust this number of retries can be defaulted as, if sub-thread transmits the transformation task failure of a subfile, then the tasks carrying number of times that this sub-thread transmits this subfile should be not more than 3 times.
In detail, the execution number of times of file transmission task can be realized by the mode of retryCounter Auto-counting.Such as, before sub-thread 2 transmits subfile 2, retryCounter initial value is 0, if sub-thread 2 Successful transmissions subfile 2, then retryCounter numerical value is constant, and sub-thread 2 continues other subfiles of transmission; If bust this, then retryCounter numerical value is from adding 1, after waiting for 5s, again performs transformation task, and when transformation task execution number of times is no more than 3 times, be successfully completed transformation task, then retryCounter numerical value resets, and sub-thread 2 continues other subfiles of transmission; If transformation task perform number of times reached 3 times but still bust this time, retryCounter numerical value resets, and sub-thread 2 locks subfile list, and subfile 2 is retracted into subfile list, and sub-thread 2 terminates to perform.
In the present embodiment, if transformation task performs the bust this number of retries that number of times reaches setting, but still during bust this, need the sub-thread terminating this execution transformation task, otherwise, run into file that Network Abnormal interrupts causing repeatedly the situation such as bust this time, if this sub-thread continues to perform other transformation tasks, there will be the situation of file bust this repeatedly equally, thus file transmission task may be caused to occur unlimited vicious circle situation, do not utilize relevant staff's Timeliness coverage and deal with problems.
Step 209: after described multiple sub-thread all terminates to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
In the present embodiment, after all sub-threads all terminate to perform, if the subfile list in server A is empty, the equal transmission success of all subfiles is then described, therefore the subfile in server B can be merged, and the subfile list of deleting in server A, thus complete the transmission of file X from server A to server B.In addition, if the subfile list in server A is not empty, then the non-transmission success of subfile stored in subfile list is described, then can on the basis of current file transformation task, carry out document breakpoint transmission task again, to complete the transmission of file X from server A to server B.
In the present embodiment, the document transmission method proposed effectively combines file transfer failure retry strategy, multithreading load balancing, file declustering combination technique and multithreading breakpoint transmission technology, user can be made according to self-demand, server configuration degree, and in conjunction with network environment situation, flexible adjustment document transmission method, the different actual demands of different user can be adapted to, server resource allocation and Network Environment Resource can be made full use of again, thus it is high to reach file transfer success rate, resource utilization is high, file transmission efficiency is high, file transfer good stability, applied ranges etc. are practical function effectively.
As shown in Figure 3, embodiments provide a kind of device of file transfer, comprising:
Acquiring unit 301, for obtaining the subfile list after file declustering to be transmitted, and exports the information of described subfile list to the first processing unit 304.
Start unit 302, for starting multiple sub-thread, and exports the information of described multiple sub-thread to described first processing unit 304.
First setting unit 303, for arranging bust this number of retries, bust this retry time interval, file reception path, and the information in described file reception path is exported to transmission unit 305, the information of described bust this number of retries and described bust this retry time interval is exported to the 3rd processing unit 307.
Described first processing unit 304, for receiving the information of the described subfile list exported by described acquiring unit 301, receive the information of the described multiple sub-threads exported by described start unit 302, receive the signal of the transmission success exported by described second processing unit 306, receive the information comprising the described subfile list of the described target subfile of bust this exported by described 3rd processing unit 307, when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and trigger described transmission unit 305, and the information of described target subfile and the sub-thread of described target is exported to described transmission unit 305, the information of described subfile list is exported to described 3rd processing unit 307.
Described transmission unit 305, for receiving the information in the described file reception path exported by described first setting unit 303, receive the information of described target subfile and the sub-thread of described target exported by described first processing unit 304, the signal that the wait that reception is exported by described 3rd processing unit 307 terminates, utilize the sub-thread of described target, described target subfile is transferred to the position corresponding with described file reception path, and exports to the second processing unit 306 information that the sub-thread of described target transmits described target subfile.
Described second processing unit 306, for transmitting the information of described target subfile according to the sub-thread of described target exported by described transmission unit 305 received, judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, trigger described first processing unit 304, and the signal of transmission success is exported to described first processing unit 304, otherwise, trigger described 3rd processing unit 307, and export to described 3rd processing unit 307 information that the signal of bust this and the sub-thread of described target transmit described target subfile.
Described 3rd processing unit 307, for receiving the information of described bust this number of retries and the described bust this retry time interval exported by described first setting unit 303, receive the information of the described subfile list exported by described first processing unit 304, after the signal and the sub-thread of described target that receive the bust this exported by described second processing unit 306 transmit the information of described target subfile, judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and trigger described transmission unit 305, and the signal waited for and terminating is exported to described transmission unit 305, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlock described subfile list, the sub-thread of described target terminates to perform, and to the information of described subfile list of described first processing unit 304 output packet containing the described target subfile of bust this.
In a kind of possible implementation, please refer to Fig. 4, this device can also comprise:
Fourth processing unit 401, after all terminating at described multiple sub-thread to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
In a kind of possible implementation, described start unit 302, specifically for starting multiple sub-thread, and the kernel that the Thread Count number of described multiple sub-thread equals the server of execute file transformation task keeps count of and subtracts 1 again.
In a kind of possible implementation, please refer to Fig. 4, this device can also comprise:
Second setting unit 402, for arranging local file path, subfile splits size;
Described acquiring unit 301, specifically for splitting size according to described subfile, becomes to comprise the subfile list of multiple subfile to be transmitted by file declustering to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
In a kind of possible implementation, described first setting unit 303 is 2 ~ 5 times, bust this retry time interval 0 ~ 10s specifically for arranging bust this number of retries.
The content such as information interaction, implementation between each unit in said apparatus, due to the inventive method embodiment based on same design, particular content can see in the inventive method embodiment describe, repeat no more herein.
In sum, embodiments provide a kind of method and device of file transfer, the embodiment of the present invention can have following beneficial effect:
1, embodiments provide a kind of method and device of file transfer, obtain the subfile list after file declustering to be transmitted in advance, start multiple sub-thread, file reception path, bust this number of retries, bust this retry time interval are set; Multiple sub-thread locks subfile list respectively, and obtains a subfile to be transmitted respectively, thus makes the execute file transmission simultaneously of the plurality of sub-thread; If transmission success, then corresponding sub-thread continues other subfiles to be transmitted of transmission; If bust this, then need to wait for the duration corresponding to bust this retry time interval, and again perform transmission, until the execution number of times of this transformation task reach bust this number of retries but still bust this time, the subfile of bust this is retracted into described subfile list and terminates corresponding sub-thread; Therefore, the present embodiment, by multithreading load balancing and bust this retry strategy, on the basis of multithreading breakpoint transmission, can improve the success rate of file transfer further.
2, a kind of method and device of file transfer is embodiments provided, by file transfer failure retry strategy, user can be made according to self-demand, server configuration degree, and in conjunction with network environment situation, bust this number of retries and bust this retry time interval are selectively set, to ensure the stability of document transmission process, promote the efficiency of file transfer, improve the success rate of file transfer.
3, embodiments provide a kind of method and device of file transfer, to have fixing file transmission task amount different from sub-thread, after arbitrary sub-thread completes a file transmission task, can continue lock subfile list and obtain subfile, unlock subfile list again, continue to obtain subfile to make other sub-threads, thus make multiple sub-thread execute file transformation task simultaneously, until all subfiles are transmitted, therefore this way of realization effectively prevent when the file transmission task amount of stator thread, the parton thread fulfiling file transmission task amount ahead of schedule is in wait state all the time, until whole sub-thread all completes the task amount of each self-retaining, the situation of the sub-thread resources waste caused, therefore, the present embodiment passes through load balancing, the load balancing between multiple sub-thread can be ensured to greatest extent, improve sub-thread resources utilance, contribute to the success rate improving file transfer.
4, a kind of method and device of file transfer is embodiments provided, user can be made according to the different application demand of self, and in conjunction with network environment situation, selectively determine that subfile splits size, with reach reduce file declustering and combination time caused time loss, reduce time needed for transmission, reduce the probability of bust this, reduce the number of times of data re-transmission, improve the stability of transmission, improve the effect of the effective rate of utilization of network.
5, embodiments provide a kind of method and device of file transfer, multiple subfile can be become to facilitate transmission file declustering by file declustering technology, if this transmission resumes on the unsuccessful transformation task basis completed before, so can before the transmission of execution subfile, first the subfile of Successful transmissions is removed from subfile list, without the need to repeating transmission, thus effectively can reduce and repeat to transmit waste to Internet resources, transformation task is stopped according to self-demand and network environment situation to facilitate user, and execute file resumes operation at any time.
6, a kind of method and device of file transfer is embodiments provided, user can be made according to the server configuration degree performing transformation task, and in conjunction with network environment situation, selectively determine Thread Count number, with while making full use of server resource allocation, the stable operation of Deterministic service device, while making full use of Internet resources, ensures speed and the stability of file transfer.
7, embodiments provide a kind of method and device of file transfer, the document transmission method proposed effectively combines file transfer failure retry strategy, multithreading load balancing, file declustering combination technique and multithreading breakpoint transmission technology, user can be made according to self-demand, server configuration degree, and in conjunction with network environment situation, flexible adjustment document transmission method, the different actual demands of different user can be adapted to, server resource allocation and Network Environment Resource can be made full use of again, thus it is high to reach file transfer success rate, resource utilization is high, file transmission efficiency is high, file transfer good stability, applied ranges etc. are practical function effectively.
Finally it should be noted that: the foregoing is only preferred embodiment of the present invention, only for illustration of technical scheme of the present invention, be not intended to limit protection scope of the present invention.All any amendments done within the spirit and principles in the present invention, equivalent replacement, improvement etc., be all included in protection scope of the present invention.
Claims (10)
1. a method for file transfer, is characterized in that, obtains the subfile list after file declustering to be transmitted in advance, starts multiple sub-thread, and arrange bust this number of retries, bust this retry time interval, file reception path, the method also comprises:
S1: when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and perform S2;
S2: utilize the sub-thread of described target, transfers to the position corresponding with described file reception path by described target subfile;
S3: judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, performs S1, otherwise, perform S4;
S4: judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and perform S2, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlocks described subfile list, the sub-thread of described target terminates to perform.
2. method according to claim 1, is characterized in that, after S4, comprises further:
After described multiple sub-thread all terminates to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
3. method according to claim 1, is characterized in that, described multiple sub-thread, comprising:
The kernel that the Thread Count number of described multiple sub-thread equals the server of execute file transformation task keeps count of and subtracts 1 again.
4. method according to claim 1, is characterized in that,
Before subfile list after described acquisition file declustering to be transmitted, comprise further: pre-set local file path, subfile splits size;
Subfile list after described acquisition file declustering to be transmitted, comprise: split size according to described subfile, file declustering to be transmitted is become multiple subfile to be transmitted, obtain the subfile list comprising described multiple subfile to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
5. according to described method arbitrary in Claims 1-4, it is characterized in that, described bust this number of retries, bust this retry time interval be set, comprising:
Arranging bust this number of retries is 2 ~ 5 times, bust this retry time interval 0 ~ 10s.
6. a device for file transfer, is characterized in that, comprising:
Acquiring unit, for obtaining the subfile list after file declustering to be transmitted, and exports the information of described subfile list to the first processing unit;
Start unit, for starting multiple sub-thread, and exports the information of described multiple sub-thread to described first processing unit;
First setting unit, for arranging bust this number of retries, bust this retry time interval, file reception path, and the information in described file reception path is exported to transmission unit, the information of described bust this number of retries and described bust this retry time interval is exported to the 3rd processing unit;
Described first processing unit, for receiving the information of the described subfile list exported by described acquiring unit, receive the information of the described multiple sub-threads exported by described start unit, receive the signal of the transmission success exported by described second processing unit, receive the information comprising the described subfile list of the described target subfile of bust this exported by described 3rd processing unit, when described subfile list is in released state, any one current sub-thread of target not performing transformation task in described multiple sub-thread locks described subfile list, and judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, unlock described subfile list, the sub-thread of described target terminates to perform, otherwise, obtain any one the target subfile in described subfile list, described target subfile is removed from described subfile list, unlock described subfile list, and trigger described transmission unit, and the information of described target subfile and the sub-thread of described target is exported to described transmission unit, the information of described subfile list is exported to described 3rd processing unit,
Described transmission unit, for receiving the information in the described file reception path exported by described first setting unit, receive the information of described target subfile and the sub-thread of described target exported by described first processing unit, receive the signal terminated by the wait of described 3rd processing unit output, utilize the sub-thread of described target, described target subfile is transferred to the position corresponding with described file reception path, and exports to the second processing unit the information that the sub-thread of described target transmits described target subfile;
Described second processing unit, for transmitting the information of described target subfile according to the sub-thread of described target exported by described transmission unit received, judge described target subfile whether transmission success, when judged result comprises described target subfile transmission success, trigger described first processing unit, and the signal of transmission success is exported to described first processing unit, otherwise, trigger described 3rd processing unit, and export to described 3rd processing unit the information that the signal of bust this and the sub-thread of described target transmit described target subfile;
Described 3rd processing unit, for receiving the information of described bust this number of retries and the described bust this retry time interval exported by described first setting unit, receive the information of the described subfile list exported by described first processing unit, after the signal and the sub-thread of described target that receive the bust this exported by described second processing unit transmit the information of described target subfile, judge whether the number of times that the sub-thread of described target transmits described target subfile is less than described bust this number of retries, if, wait for the duration corresponding with described bust this retry time interval, and trigger described transmission unit, and the signal waited for and terminating is exported to described transmission unit, otherwise, when described subfile list is in released state, the sub-thread of described target locks described subfile list, described target subfile is retracted into described subfile list, unlock described subfile list, the sub-thread of described target terminates to perform, and to the information of described subfile list of described first processing unit output packet containing the described target subfile of bust this.
7. the device of file transfer according to claim 6, is characterized in that, comprises further:
Fourth processing unit, after all terminating at described multiple sub-thread to perform, judge whether described subfile list is empty, described subfile list is comprised for time empty in judged result, the described target subfile transferring to the position corresponding with described file reception path is merged, and deletes described subfile list.
8. the device of file transfer according to claim 6, is characterized in that, described start unit, and specifically for starting multiple sub-thread, and the kernel that the Thread Count number of described multiple sub-thread equals the server of execute file transformation task keeps count of and subtracts 1 again.
9. the device of file transfer according to claim 6, is characterized in that,
Comprise further: the second setting unit, for arranging local file path, subfile splits size;
Described acquiring unit, specifically for splitting size according to described subfile, becomes to comprise the subfile list of multiple subfile to be transmitted by file declustering to be transmitted, and by described subfile list storage to the position corresponding with described local file path.
10., according to the device of described file transfer arbitrary in claim 6 to 9, it is characterized in that,
Described first setting unit is 2 ~ 5 times, bust this retry time interval 0 ~ 10s specifically for arranging bust this number of retries.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510925190.2A CN105450769A (en) | 2015-12-10 | 2015-12-10 | File transmission method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510925190.2A CN105450769A (en) | 2015-12-10 | 2015-12-10 | File transmission method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105450769A true CN105450769A (en) | 2016-03-30 |
Family
ID=55560535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510925190.2A Pending CN105450769A (en) | 2015-12-10 | 2015-12-10 | File transmission method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105450769A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107682584A (en) * | 2017-10-24 | 2018-02-09 | 上海携程商务有限公司 | Software facsimile transmission method and its system, storage medium, electronic equipment |
| CN108022028A (en) * | 2016-11-01 | 2018-05-11 | 南京途牛科技有限公司 | A kind of method for processing resource and device |
| CN108429799A (en) * | 2018-03-01 | 2018-08-21 | 北京天融信网络安全技术有限公司 | Parallel transmission method, device and the storage medium of batch documents |
| CN110008187A (en) * | 2018-12-18 | 2019-07-12 | 阿里巴巴集团控股有限公司 | File transmission dispatching method, device, equipment and computer readable storage medium |
| CN110661829A (en) * | 2018-06-28 | 2020-01-07 | 杭州海康威视系统技术有限公司 | File downloading method and device, client and computer readable storage medium |
| CN112000292A (en) * | 2020-08-21 | 2020-11-27 | 北京金山云网络技术有限公司 | Data migration method and device, electronic equipment and computer storage medium |
| CN112492033A (en) * | 2020-11-30 | 2021-03-12 | 深圳市移卡科技有限公司 | File transmission method, system and computer readable storage medium |
| WO2022206759A1 (en) * | 2021-03-31 | 2022-10-06 | 华为技术有限公司 | File sending method, device and computer readable storage medium |
| WO2022252802A1 (en) * | 2021-05-31 | 2022-12-08 | 华为技术有限公司 | File transmission method and apparatus, and electronic device and storage medium |
| WO2023072083A1 (en) * | 2021-10-29 | 2023-05-04 | 维沃移动通信有限公司 | File processing method, and electronic device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101577736A (en) * | 2009-06-04 | 2009-11-11 | 中兴通讯股份有限公司 | Method and device for uploading files |
| CN102790771A (en) * | 2012-07-25 | 2012-11-21 | 山东中创软件商用中间件股份有限公司 | File transmission method and system |
| CN103186418A (en) * | 2011-12-30 | 2013-07-03 | 北大方正集团有限公司 | Method and system for distributing tasks |
-
2015
- 2015-12-10 CN CN201510925190.2A patent/CN105450769A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101577736A (en) * | 2009-06-04 | 2009-11-11 | 中兴通讯股份有限公司 | Method and device for uploading files |
| CN103186418A (en) * | 2011-12-30 | 2013-07-03 | 北大方正集团有限公司 | Method and system for distributing tasks |
| CN102790771A (en) * | 2012-07-25 | 2012-11-21 | 山东中创软件商用中间件股份有限公司 | File transmission method and system |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108022028B (en) * | 2016-11-01 | 2021-02-26 | 南京途牛科技有限公司 | Resource processing method and device |
| CN108022028A (en) * | 2016-11-01 | 2018-05-11 | 南京途牛科技有限公司 | A kind of method for processing resource and device |
| CN107682584A (en) * | 2017-10-24 | 2018-02-09 | 上海携程商务有限公司 | Software facsimile transmission method and its system, storage medium, electronic equipment |
| CN108429799A (en) * | 2018-03-01 | 2018-08-21 | 北京天融信网络安全技术有限公司 | Parallel transmission method, device and the storage medium of batch documents |
| CN108429799B (en) * | 2018-03-01 | 2021-11-16 | 北京天融信网络安全技术有限公司 | Parallel transmission method and device for batch files and storage medium |
| CN110661829A (en) * | 2018-06-28 | 2020-01-07 | 杭州海康威视系统技术有限公司 | File downloading method and device, client and computer readable storage medium |
| CN110008187A (en) * | 2018-12-18 | 2019-07-12 | 阿里巴巴集团控股有限公司 | File transmission dispatching method, device, equipment and computer readable storage medium |
| CN110008187B (en) * | 2018-12-18 | 2023-05-12 | 创新先进技术有限公司 | File transmission scheduling method, device, equipment and computer readable storage medium |
| CN112000292A (en) * | 2020-08-21 | 2020-11-27 | 北京金山云网络技术有限公司 | Data migration method and device, electronic equipment and computer storage medium |
| CN112492033A (en) * | 2020-11-30 | 2021-03-12 | 深圳市移卡科技有限公司 | File transmission method, system and computer readable storage medium |
| WO2022206759A1 (en) * | 2021-03-31 | 2022-10-06 | 华为技术有限公司 | File sending method, device and computer readable storage medium |
| WO2022252802A1 (en) * | 2021-05-31 | 2022-12-08 | 华为技术有限公司 | File transmission method and apparatus, and electronic device and storage medium |
| CN115484250A (en) * | 2021-05-31 | 2022-12-16 | 华为技术有限公司 | File transmission method, system, electronic device and storage medium |
| EP4333315A4 (en) * | 2021-05-31 | 2024-08-14 | Huawei Tech Co Ltd | File transmission method and apparatus, and electronic device and storage medium |
| WO2023072083A1 (en) * | 2021-10-29 | 2023-05-04 | 维沃移动通信有限公司 | File processing method, and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105450769A (en) | File transmission method and device | |
| CN109496418B (en) | File downloading method and device, and equipment/terminal/server | |
| CN106843170A (en) | Method for scheduling task based on token | |
| CN105516086B (en) | Method for processing business and device | |
| CN110659151B (en) | Data verification method and device and storage medium | |
| CN103237049A (en) | File transfer method and device | |
| CN103139157B (en) | A kind of based on the network communication method of socket, Apparatus and system | |
| CN107786583A (en) | A kind of document down loading method and device | |
| CN110392381B (en) | Multi-channel wireless communication method and device for nuclear power station and terminal equipment | |
| US20180034894A1 (en) | Data Transmission Method, and Terminal | |
| US20240160474A1 (en) | Multi-core processor task scheduling method, and device and storage medium | |
| CN115426361A (en) | Distributed client packaging method and device, main server and storage medium | |
| WO2024001411A1 (en) | Multi-thread scheduling method and device | |
| CN111147585B (en) | Equipment upgrading method, device, storage medium and system | |
| CN106776032A (en) | The treating method and apparatus of the I/O Request of distributed block storage | |
| RU2522995C2 (en) | Method and apparatus for creating peer-to-peer group in peer-to-peer application and method of using peer-to-peer group | |
| CN102377478A (en) | Information processing device, communication system, and information processing method | |
| CN113630808A (en) | Data transmission method, first device and second device | |
| CN107506491B (en) | OSD data distribution method and device of distributed file system | |
| CN102984160A (en) | Distributed network scan task processing method and system | |
| CN103701830B (en) | A kind of data that realize process and mutual method and system | |
| CN111309467A (en) | Task distribution method and device, electronic equipment and storage medium | |
| CN105992360B (en) | Resource allocation method and device | |
| CN112015601B (en) | Method and device for processing data of multiple data centers | |
| CN116996450B (en) | Management data processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160330 |
|
| WD01 | Invention patent application deemed withdrawn after publication |