CN101651676A - Online download method for large data volume files - Google Patents
Online download method for large data volume files Download PDFInfo
- Publication number
- CN101651676A CN101651676A CN200910091939A CN200910091939A CN101651676A CN 101651676 A CN101651676 A CN 101651676A CN 200910091939 A CN200910091939 A CN 200910091939A CN 200910091939 A CN200910091939 A CN 200910091939A CN 101651676 A CN101651676 A CN 101651676A
- Authority
- CN
- China
- Prior art keywords
- data
- terminal
- server
- sequence number
- bag sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides an online download method for large data volume files. The method relates to a server and a terminal. The server has control threads and data threads; the terminal has control threads and data threads too. The online download method for large data volume files comprises steps of: (1) the server and the terminal operate the control threads for protocol information interactionto realize handshake between control port protocols of the server and the terminal; (2) after the step 1), the server and the terminal operate the data threads to realize handshake between data portprotocols of the server and the terminal; (3) transmitting data files between the data port protocols of the server and the terminal via the data threads; (4) operating the control threads again aftertransmitting the data files. The invention is able to transmit data efficiently and reliably, and is especially suitable for online download of large data volume files.
Description
Technical field
The invention belongs to technical field of the computer network, specifically, the present invention relates to a kind of high-efficiency network method for down loading of large-data documents.
Background technology
Along with development of computer, the data volume of data file constantly increases, and especially high definition media is universal day by day, makes people often need download large-data documents.And how under certain network bandwidth environment, realize download that large-data documents (as the high definition media file, this class file average size reaches more than the 4G) is efficient, complete and local and to play will be a no small difficult problem.And use in downloading at present TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) procotol arranged comparatively widely.
Transmission control protocol (TCP) agreement is mainly used in and sets up a virtual connection between main frame, to realize the packet exchange of high reliability.The IP agreement can be carried out cutting apart of IP packet and be assembled, but whether can not be well understood to packet by the IP agreement successfully sends to object-computer.And Transmission Control Protocol can require to send an affirmation after packet is successfully sent to object-computer in its transmission mode; If do not receive affirmation in certain time limit, TCP will resend packet so.In addition, in the process of transmission, if receive unordered, lose and ruined packet, TCP can also be responsible for recovery, as long as most common form be mustn't go to affirmation exactly, just resends datagram, ends up to obtaining confirming as of the other side.
Transmission Control Protocol can provide a reliable delivering path, thus at present to higher some occasions of transmission data demand all based on Transmission Control Protocol, to guarantee correct, the integrality of data.If but in the face of great amount of terminals and server realization TCP transmission the time,, therefore greatly increased the various expenses of server because server will be responsible for realizing whole fault tolerant mechanisms, this will be a very big burden for server.
User Datagram Protoco (UDP) (UDP) is a part of ICP/IP protocol group.Its foundation is the approach for the no linkage function that a visit IP is provided to application program.TCP and UDP use IP.The design of UDP allows application program to create datagram, and they are addressed to the port of access application or process.The basic role of UDP is exactly the port address that increases a program process to an IP grouping.
UDP is simple much than TCP as can be seen, whether successfully receive data as long as data are sent regardless of transmitting terminal, so transmission speed is very fast.But this agreement does not have fault tolerant mechanism for the packet loss that may occur, and can't guarantee the integrality of transfer of data, promptly can't guarantee reliable transfer of data.
Consider high definition media characteristic capacious, download between terminal and the server is exactly the process of a Large Volume Data transmission, to consider not only that in this process the integrality of transfer of data also will take into account the speed problem of transfer of data, though TCP is reliable, but the failure tolerance of self can cause the transmission that repeats that causes data to a certain extent, and too much protocol handshake also exerts a certain influence to speed, and in the face of the mass data transmission, server load is heavier; Though UDP can more effective settlement server loading problem, has data transmission bauds faster, UDP is in the face of connectionless, and Network Packet Loss is not done fault-tolerant processing, therefore, can not provide reliable transfer of data.
Summary of the invention
The purpose of this invention is to provide a kind of at the network download method that guarantees to realize on the reliable basis of transfer of data the large-data documents high-efficiency transfer.And this network download method can be selected data downloading mode and fault tolerant mechanism automatically according to network packet loss rate, thereby has further guaranteed the high efficiency and the correctness of transfer of data, satisfies the needs under the environment of various heterogeneous networks packet loss.
For achieving the above object, the invention provides a kind of method of big data quantity network download, this method relates to server and terminal, described server has control thread and data thread, described terminal has control thread and data thread, and the network download method of described large-data documents comprises the following steps:
1) to carry out protocol information mutual for server and terminal operating control thread, finishes shaking hands between the control port agreement of server and terminal;
2) after step 1) finishes, server and terminal operating data thread, the shaking hands of FPDP of finishing server and terminal;
3) by data thread transmission data file between the FPDP of server and terminal;
4) behind the data file end of transmission, rerun the control thread.
Adopt such scheme, when shaking hands successfully, FPDP is directly carried out the download of data, control port is no longer interfered, only take over by control port again and carry out the download flow process of next file or enter waiting for that the user triggers state when the current file end of transmission, unnecessary shaking hands saved in the intermediate data transmission, thereby effectively improves message transmission rate.
In the technique scheme, in the described step 1), carry out protocol information when mutual, after the protocol information originating end is sending protocol information, do not receive the answer of receiving terminal to described protocol information in predefined time threshold, then described protocol information originating end is retransmitted this protocol information.
In the technique scheme, described time threshold is 200ms.
In the technique scheme, in the described step 3), also need to take to retransmit, reissue strategy, the corresponding downloading mode of Dynamic Selection according to Network Packet Loss situation self adaptation.When the count value of the internal counter of the application program of carrying out this network download when bag sequence number and this terminal of data message that terminal is received does not match, select triggering retransmission mechanism or the mechanism of reissuing to obtain the data message of losing according to the difference of the two, to guarantee the high efficiency and the fault-tolerance of transfer of data.
Further, described step 3) comprises following substep:
31) server sends datagram to terminal;
32) described terminal receives and the resolution data message, obtain the bag sequence number of the data message of receiving, and compare with internal counter that this terminal is carried out the application program of this network download, be 0 if described bag sequence number deducts the internal counter count value, then carry out the storage of data; If the difference that described bag sequence number deducts the internal counter count value then enters step 33 more than or equal to 3); If the difference that described bag sequence number deducts the internal counter count value then enters step 34 less than 3);
33) described terminal to server sends re-send request may, carry in the re-send request may and retransmit initial bag sequence number, the initial bag sequence number of described repeating transmission equals the currency of internal counter, the data message of being received that gets off of described end-on simultaneously abandons without exception, the difference that deducts the internal counter count value up to described bag sequence number is 0, and the pointer of described data file carries out offset according to current bag sequence number;
34) described terminal receives current data message, and file pointer is offset and stores data according to the bag sequence number then, and the internal counter value changes the current bag sequence number that receives into, and the middle bag sequence number of being skipped deposits in and reissues formation as reissuing the bag sequence number; Described terminal to server sends and reissues request, and the request of reissuing is carried and reissued the bag sequence number;
35) server is after receiving re-send request may, and the initial bag sequence number of the described repeating transmission that transmission comes up according to terminal is reorientated the internal file pointer and newly sent datagram from the lifting of the initial bag sequence number of described repeating transmission;
36) server is after receiving the request of reissuing, internal file continues down sequence offsets, server is reissued processing immediately to the Bale No. that requires to reissue on transmitting in sub-thread simultaneously, if terminal is received the bag sequence number less than current counter, judge this bag sequence number whether with reissue formation in a certain bag sequence number of reissuing equate, if equate, then from reissue formation, delete this and reissue the bag sequence number.
In the technique scheme, described step 34) in, reissuing the bag sequence number when surpassing a certain extreme value in reissuing formation, the terminal to server end sends the reduction of speed order to reduce the frequency that server sends packet.
In the technique scheme, described terminal is whenever finished the packet of some and is downloaded promptly to online reception heartbeat message of server transmission; If the continuous heartbeat message of not receiving terminal for three times of described server, then server stops this download.
Compared with prior art, the invention has the advantages that:
Strong point in conjunction with Transmission Control Protocol and udp protocol, both had reliability of data transmission under the Transmission Control Protocol, it is light to possess under the udp protocol server load again, the characteristics that transfer of data is fast, what deserves to be explained is that especially terminal can be according to the packet loss situation of heterogeneous networks, the adaptively changing data transmission policies, possesses adaptive capacity stronger under each network environment, especially under the Network Packet Loss serious (public network environment, packet loss is about 20%) or the environment of data congestion, can show stronger Data Receiving ability (seeing resolution chart 2 for details).Facts have proved that speed of download of the present invention on average can reach 70% to 80% of the network bandwidth.
In addition, core code amount of the present invention only accounts for 18.5KB, and code is simplified, and is efficient, especially meets the code space requirement of embedded platform.
Description of drawings
Fig. 1 is that the present invention realizes the program flow diagram that big data quantity is downloaded.
Embodiment:
Below in conjunction with accompanying drawing 1 and embodiment the present invention is described in further detail:
High definition media network download method based on embedded platform provided by the invention comprises that (1) both sides arrange each flag bit of application layer, adopt fat client mode; (2) function modoularization adopts two-wire journey mechanism, and wherein a thread is responsible for control protocol and is shaken hands, and another thread is responsible for FPDP and is shaken hands, and two ports adopt different fault-tolerant strategies, and are with strong points, effectively promote message transmission rate; (3) the operational mode analog D MA passage strategy of two-wire journey is taken over the transmission of data until end of file transfer fully by FPDP after control port is successfully shaken hands; (4) take to retransmit, reissue strategy according to the network packet loss rate self adaptation, select corresponding downloading mode, guarantee the high efficiency of transfer of data.
Both sides arrange the download that each flag bit carries out shaking hands of agreement and data in application layer, for effectively alleviating server load, adopt fat client mode, and terminal is responsible for the operation of the overwhelming majority, effectively alleviates server load.
Both sides fill the message flag position according to a preconcerted arrangement and send, and the recipient parses data equally according to a preconcerted arrangement, and is fixing, both sides arrange the type of the next unique definite message of the 3rd byte of application layer, simultaneously, for the convenient expansion of type of message from now on, can adopt secondary type of message catalogue; Arrange the 5th, the 6 byte for sending length of data package (comprising packet header), detecting the integrality of packet, if the data message length that parses and head H ead.length length violation think that then institute transmits messages civilian incorrect.
Distribute two independently thread carry out the operation of control port and FPDP respectively, control port is just taken over the transmission of data by FPDP after successfully shaking hands, the centre need not the interference of control port, saved a large amount of shaking hands the time than Transmission Control Protocol, finished again when transfer of data and take over current operation (promptly taking over CPU) by the control thread by control port.
The fault-tolerant employing timeout mechanism of control port is not promptly also received recipient's response after transmitting terminal is sending message 200ms, just judgement information sends overtime, carries out retransmit operation.
The fault-tolerant of FPDP can be selected fault tolerant mechanism automatically according to the situation of network packet loss rate, and be specific as follows:
(1) after receiving data message, parse Bale No. (flag bit in can application layer is according to a preconcerted arrangement judged the bag sequence number of being received), and compare with file internal counter (internal counter refer to terminal carry out the application program of this download be current) just at the set internal counter of downloaded files, if meet, it then is desired packet, can carry out the write operation of file, if Bale No. and file counter are not inconsistent, then packet loss takes place in explanation, carries out fault tolerant mechanism according to network packet loss rate and selects.
(2) when the Bale No. that parses from data message and internal file counter difference more than or equal to 3 the time, trigger retransmission mechanism, terminal abandons the arrival up to required Bale No. of the data message that received, and server is then reorientated file internal pointer and propelling data immediately according to transmitting the Bale No. that comes up.
(3) when the Bale No. that parses from data message and internal file counter difference less than 3 the time, mechanism is reissued in triggering, the data message that terminal acceptance has passed over, file pointer carries out offset storage according to the Bale No. that parses, terminal is set up and is reissued formation simultaneously, reissue request, server receives that not changing original after the request of reissuing sends pointer position, but utilizes another file pointer to reissue the quick location of bag and reissue in sub-thread.Different is with retransmission mechanism, reissue in the mechanism, terminal receives current data message, file pointer is offset and stores data according to the bag sequence number simultaneously, the internal counter value changes the current bag sequence number that receives into, effective like this saved data repeat transmit, promoted the efficient of transfer of data.
Terminal receive the Bale No. littler than file counter just can judge whether with reissue formation in a certain Bale No. conform to, if conform to, then explanation is reissued successfully, this Bale No. is made team operation.Further, for avoid network packet loss rate higher may cause reissue overflowing of formation, when reissuing formation above a certain extreme value, according to a preconcerted arrangement reissue grade, terminal can adopt dynamic adjustment reduction of speed strategy to send the reduction of speed order to server end, to adapt to the length of reissuing formation at present and to reduce the frequency that server sends packet.
Receive the tail bag when terminal, judge that current whether reissue formation be empty, if reissue formation this moment for empty, then send the request of finishing of downloading by the control port, the server response also stops to transmit data, and so far whole transmission course finishes; Then reissue operation if reissue the formation non-NULL, the judgement that then repeats to reissue formation is until for empty, shows that data have downloaded complete.
Further, also adopted data message intelligence feedback mechanism in the present embodiment: when the terminal data thread is whenever received 100 data messages of server (each message 1316Bytes), the control thread of terminal then sends an online reception heartbeat message to server, and server is judged whether normal online reception of terminal according to the heartbeat message that receives; The continuous heartbeat message of not receiving terminal for three times of server, server are thought and are stopped the terminal off-line this secondary data immediately and download.When this feedback mechanism can prevent effectively that certain network terminal from breaking down off-line, network congestion phenomenon that the data bulk deposition that causes causes at the end router end and waste server resource.
Claims (7)
1, a kind of network download method of large-data documents relates to server and terminal, and described server has control thread and data thread, and described terminal has control thread and data thread, and the network download method of described large-data documents comprises the following steps:
1) to carry out protocol information mutual for server and terminal operating control thread, finishes shaking hands between the control port agreement of server and terminal;
2) after step 1) finishes, server and terminal operating data thread, the shaking hands of FPDP of finishing server and terminal;
3) by data thread transmission data file between the FPDP of server and terminal;
4) behind the data file end of transmission, rerun the control thread.
2, the network download method of large-data documents according to claim 1, it is characterized in that, in the described step 1), carry out protocol information when mutual, after the protocol information originating end is sending protocol information, do not receive the answer of receiving terminal to described protocol information in predefined time threshold, then described protocol information originating end is retransmitted this protocol information.
3, the network download method of large-data documents according to claim 2 is characterized in that, described time threshold is 200ms.
4, the network download method of large-data documents according to claim 1, it is characterized in that, in the described step 3), when the count value of the internal counter of the application program of carrying out this network download when bag sequence number and this terminal of data message that terminal is received does not match, select to trigger retransmission mechanism or the mechanism of reissuing is obtained the data message of losing according to the difference of the two.
5, the network download method of large-data documents according to claim 1 is characterized in that, in the described step 3), comprises following substep:
31) server sends datagram to terminal;
32) described terminal receives and the resolution data message, obtain the bag sequence number of the data message of receiving, and compare with internal counter that this terminal is carried out the application program of this network download, be 0 if described bag sequence number deducts the internal counter count value, then carry out the storage of data; If the difference that described bag sequence number deducts the internal counter count value then enters step 33 more than or equal to 3); If the difference that described bag sequence number deducts the internal counter count value then enters step 34 less than 3);
33) described terminal to server sends re-send request may, carry in the re-send request may and retransmit initial bag sequence number, the initial bag sequence number of described repeating transmission equals the currency of internal counter, the data message of being received that gets off of described end-on simultaneously abandons without exception, the difference that deducts the internal counter count value up to described bag sequence number is 0, and the pointer of described data file carries out offset according to current bag sequence number;
34) described terminal receives current data message, and file pointer is offset and stores data according to the bag sequence number then, and the internal counter value changes the current bag sequence number that receives into, and the middle bag sequence number of being skipped deposits in and reissues formation as reissuing the bag sequence number; Described terminal to server sends and reissues request, and the request of reissuing is carried and reissued the bag sequence number;
35) server is after receiving re-send request may, and the initial bag sequence number of the described repeating transmission that transmission comes up according to terminal is reorientated the internal file pointer and newly sent datagram from the lifting of the initial bag sequence number of described repeating transmission;
36) server is after receiving the request of reissuing, internal file continues down sequence offsets, server is reissued processing immediately to the Bale No. that requires to reissue on transmitting in sub-thread simultaneously, if terminal is received the bag sequence number less than current counter, judge this bag sequence number whether with reissue formation in a certain bag sequence number of reissuing equate, if equate, then from reissue formation, delete this and reissue the bag sequence number.
6, the network download method of large-data documents according to claim 5, it is characterized in that, described step 34) in, reissuing the bag sequence number when surpassing a certain extreme value in reissuing formation, the terminal to server end sends the reduction of speed order to reduce the frequency that server sends packet.
7, the network download method of large-data documents according to claim 1 is characterized in that, described terminal is whenever finished the packet of some and downloaded promptly to online reception heartbeat message of server transmission; If the continuous heartbeat message of not receiving terminal for three times of described server, then server stops this download.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910091939A CN101651676B (en) | 2009-09-01 | 2009-09-01 | Online download method for large data volume files |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910091939A CN101651676B (en) | 2009-09-01 | 2009-09-01 | Online download method for large data volume files |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101651676A true CN101651676A (en) | 2010-02-17 |
| CN101651676B CN101651676B (en) | 2012-10-17 |
Family
ID=41673785
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910091939A Expired - Fee Related CN101651676B (en) | 2009-09-01 | 2009-09-01 | Online download method for large data volume files |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101651676B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102333337A (en) * | 2011-10-11 | 2012-01-25 | 中兴通讯股份有限公司 | Script transmission method, wireless network simulation system (iNES) and server |
| CN103051674A (en) * | 2012-11-23 | 2013-04-17 | 深圳市航天泰瑞捷电子有限公司 | Method and device for remotely upgrading wireless communication module as well as handheld unit (HHU) |
| CN104539917A (en) * | 2015-02-03 | 2015-04-22 | 成都金本华科技股份有限公司 | Method for improving definition of video image |
| CN104581422A (en) * | 2015-02-05 | 2015-04-29 | 成都金本华科技股份有限公司 | Method and device for processing network data transmission |
| CN108632929A (en) * | 2018-04-16 | 2018-10-09 | 北京京大律业知识产权代理有限公司 | A kind of big data polymerization towards quick service |
| CN109729178A (en) * | 2019-02-22 | 2019-05-07 | 苏州华必讯信息科技有限公司 | A kind of network download method of large-data documents |
| CN112187667A (en) * | 2020-09-16 | 2021-01-05 | 深圳希施玛数据科技有限公司 | Data downloading method, device, equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7558870B2 (en) * | 2005-02-22 | 2009-07-07 | Alcatel Lucent | Multimedia content delivery system |
-
2009
- 2009-09-01 CN CN200910091939A patent/CN101651676B/en not_active Expired - Fee Related
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102333337A (en) * | 2011-10-11 | 2012-01-25 | 中兴通讯股份有限公司 | Script transmission method, wireless network simulation system (iNES) and server |
| CN103051674A (en) * | 2012-11-23 | 2013-04-17 | 深圳市航天泰瑞捷电子有限公司 | Method and device for remotely upgrading wireless communication module as well as handheld unit (HHU) |
| CN104539917A (en) * | 2015-02-03 | 2015-04-22 | 成都金本华科技股份有限公司 | Method for improving definition of video image |
| CN104581422A (en) * | 2015-02-05 | 2015-04-29 | 成都金本华科技股份有限公司 | Method and device for processing network data transmission |
| CN104581422B (en) * | 2015-02-05 | 2017-09-15 | 成都金本华科技股份有限公司 | A kind of method and apparatus transmitted for network data |
| CN108632929A (en) * | 2018-04-16 | 2018-10-09 | 北京京大律业知识产权代理有限公司 | A kind of big data polymerization towards quick service |
| CN108632929B (en) * | 2018-04-16 | 2021-08-17 | 上海识装信息科技有限公司 | Big data aggregation method for quick service |
| CN109729178A (en) * | 2019-02-22 | 2019-05-07 | 苏州华必讯信息科技有限公司 | A kind of network download method of large-data documents |
| CN112187667A (en) * | 2020-09-16 | 2021-01-05 | 深圳希施玛数据科技有限公司 | Data downloading method, device, equipment and storage medium |
| CN112187667B (en) * | 2020-09-16 | 2022-09-13 | 深圳希施玛数据科技有限公司 | Data downloading method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101651676B (en) | 2012-10-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101651676B (en) | Online download method for large data volume files | |
| CN103036904B (en) | A kind of udp protocol that uses in a communication network carries out the method for reliable data transmission | |
| CN106878191B (en) | A kind of data transferring method, sending node, receiving node and data communication system | |
| US7899925B2 (en) | System and method for fast, reliable byte stream transport | |
| US10715282B2 (en) | Method and related device for improving TCP transmission efficiency using delayed ACK | |
| US8537664B2 (en) | Method and system for accelerating transmission of data between network devices | |
| US20170373804A1 (en) | Methods for enabling delay-awareness in the constrained application protocol (coap) | |
| CN108270682A (en) | A kind of message transmitting method, terminal, the network equipment and communication system | |
| US20060168262A1 (en) | System and method for reliable packet data transport in a computer network | |
| CN107204834A (en) | A kind of control method of the express network transmitting based on UDT agreements | |
| EP2228965A1 (en) | Gateway and method for connecting an IP network to a sensor network | |
| CN102820915B (en) | Satellite link system for improving TCP (transmission control protocol) transmission performance and use method of system | |
| US8605578B1 (en) | System and method for handling of destination host side congestion | |
| CN104093170A (en) | TCP-based data transmission method and TCP agent apparatus | |
| CN107360177B (en) | A kind of message transmitting method and device based on UDP | |
| CN103795632A (en) | Data message transmission method, related equipment and system | |
| CN101212401B (en) | Grid-oriented configurable data transmission method and system | |
| CN101505306A (en) | Inter-node reliable communication method in distributed system | |
| CN111970092B (en) | Multi-protocol redundant network asynchronous communication method supporting reliability adjustment | |
| Wang et al. | Aggressive congestion control mechanism for space systems | |
| CN101741747B (en) | NFS (Network File System) flow control method for UDP (User Datagram Protocol) protocol | |
| CN109729178A (en) | A kind of network download method of large-data documents | |
| CN106302426A (en) | A kind of udp protocol stack implementation method of band retransmission mechanism based on FPGA | |
| EP3367599B1 (en) | Method and system for transferring data within a layered architecture of network components | |
| CN117692367A (en) | RDP redundant data transmission method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20121017 Termination date: 20140901 |
|
| EXPY | Termination of patent right or utility model |