CN110474937A - Super large file fragmentation continuous transmission method and its system - Google Patents
Super large file fragmentation continuous transmission method and its system Download PDFInfo
- Publication number
- CN110474937A CN110474937A CN201810438851.2A CN201810438851A CN110474937A CN 110474937 A CN110474937 A CN 110474937A CN 201810438851 A CN201810438851 A CN 201810438851A CN 110474937 A CN110474937 A CN 110474937A
- Authority
- CN
- China
- Prior art keywords
- file
- value
- server
- client
- segmental
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
Abstract
The invention discloses a kind of super large file fragmentation continuous transmission methods, client extracts the corresponding MD5 value of front portion and rear portion to transmitting file and is sent to server, the server receives two parts of MD5 values and carries out Rapid matching with the file of preservation, after successful match, the client determines the complete MD5 value to transmitting file and is sent to server, the server receives the complete MD5 value and carries out precise search with the file of preservation, after retrieving same file, this is waited for that the transmitting file second passes to server by the client；It also discloses a kind of super large file fragmentation and resumes system；Through the invention can be after client and server disconnection and reconnection, according to transmission situation, quickly determination passes mode again, and improves transmission efficiency.
Description
Technical field
The invention belongs to field of computer technology, and in particular to a kind of super large file fragmentation continuous transmission method and its system.
Background technique
The file transmission working method of traditional mode is as follows: reading local file information；Construct file transmission request；Service
The request of device customer in response end transmission, keeps transmissions links before the end of transmission；Server, which receives client transmissions, to be terminated
Afterwards, the file information preservation is carried out；After server saves the file information success, background data base is recorded and returns to client；
This traditional operating mode occupies very big advantage when transmitting for small documents；But this mode is in super large
In document transmission process, good advantage cannot be all either obtained in user level or system level.Reason: conventional transmission
Mode is disposably to request, and can just be responded after needing server to receive, intermediate network flash or other reason once occur
Cause to connect disconnection with server, client can only initiate request since 0, and traditional transmission mode is obviously unable to satisfy this
The demand of reality.
Summary of the invention
In view of this, the main purpose of the present invention is to provide a kind of super large file fragmentation continuous transmission method and its systems.
In order to achieve the above objectives, the technical scheme of the present invention is realized as follows:
The embodiment of the present invention provides a kind of super large file fragmentation continuous transmission method, this method comprises: client extracts text to be passed
The corresponding MD5 value of the front portion and rear portion of part and be sent to server, the server receive two parts of MD5 values and
Rapid matching is carried out with the file of preservation, after successful match, the client determines the complete MD5 value to transmitting file and sends out
It is sent to server, the server receives the complete MD5 value and carries out precise search with the file of preservation, when retrieving phase
After file, this is waited for that the transmitting file second passes to server by the client.
In above scheme, this method further include: after it fails to match, the client is by file fragmentation to be passed and will be every
Segment file and corresponding MD5 value are separately sent to server, determine MD5 value after one segmental file of the every reception of server
And be compared with the corresponding MD5 value of this section of segmental file that the client is sent, if two MD5 values unanimously if save
The segmental file, on the contrary, the client retransmits the segmental file if two MD5 values are inconsistent.
In above scheme, this method further include: after the server has received all segmental files, the client kimonos
The MD5 value of each segmental file is merged into MD5 value small documents by business device, and determines the MD5 value of the MD5 value small documents, institute
It states client and sends server for the MD5 value and be compared, if the same think that file transmission is complete, the server
Data are written, if it is not the same, then thinking that file transmission is imperfect, the client is initiated request and transmitted from the beginning.
In above scheme, the client determines the complete MD5 value to transmitting file, specifically: the client is in determination
During the complete MD5 value of transmitting file, current disk utilization rate is judged before reading file content each time, if disk uses
When rate is higher than threshold value, reduce the size for reading file content next time immediately.
The embodiment of the present invention also provides a kind of super large file fragmentation and resumes system, which includes: client, server；
The client, for extracting the corresponding MD5 value of front portion and rear portion to transmitting file and being sent to
Server；It is also used to determine the complete MD5 value to transmitting file and is sent to server；Be also used to when server retrieve it is identical
After file, this is waited for that the transmitting file second passes to server；
The server, for receiving two parts of MD5 values and carrying out Rapid matching with the file of preservation；Work as successful match
Afterwards, it is also used to receive the complete MD5 value and carries out precise search with the file of preservation.
In above scheme, the client is also used to after it fails to match, by file fragmentation to be passed and by every segment file
And corresponding MD5 value is separately sent to server；It is also used to retransmit the segmental file if two MD5 values are inconsistent；
The server, be also used to it is every reception one segmental file after determine MD5 value and with the client send
The corresponding MD5 value of this section of segmental file is compared, if two MD5 values unanimously if save the segmental file, on the contrary, if two
A MD5 value is inconsistent, and the client retransmits the segmental file.
In above scheme, the server, after being also used to receive all segmental files, by the MD5 of each segmental file
Value is merged into MD5 value small documents, and determines the MD5 value of the MD5 value small documents；It is small to receive the MD5 value that the client is sent
It the MD5 value of file and being compared, if the same thinks that file transmission is complete, data are written in the server,
The client is also used to after server has received all segmental files, and the MD5 value of each segmental file is closed
And at MD5 value small documents, and determine the MD5 value of the MD5 value small documents；Server is sent by the MD5 value to be compared,
If the server compares not identical, then it is assumed that file transmission is imperfect, initiates request and transmits from the beginning.
In above scheme, the client, specifically for during determining complete MD5 value to transmitting file, each time
Current disk utilization rate is judged before reading file content, if disk utilization rate is higher than threshold value, reduces read next time immediately
The size of file content.
Compared with prior art, the present invention can disconnect in client and server and after reconnection, according to transmission situation
Quickly determination passes mode again, and improves transmission efficiency.
Detailed description of the invention
Fig. 1 is the flow chart that the embodiment of the present invention improves a kind of super large file fragmentation continuous transmission method.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
The embodiment of the present invention improves a kind of super large file fragmentation continuous transmission method, as shown in Figure 1, this method passes through following steps
It realizes:
Step 101: client extracts the corresponding MD5 value of front portion and rear portion to transmitting file and is sent to clothes
Business device；
Specifically, the client extraction can be according to the big of transmitting file to the front portion and rear portion of transmitting file
In general small specific determination can extract after the preceding 4m of transmitting file the corresponding MD5 value of 4m with after using client.
Step 102: the server receives two parts of MD5 values and carries out Rapid matching, successful match with the file of preservation
Step 103 is gone to, it fails to match goes to step 104；
Specifically, two parts of MD5 values are all identical as the file that server saves just belongs to successful match.
In fact, the server can also save the corresponding MD5 value of front portion and rear portion of each file, so as to
Subsequent Rapid matching；Certainly, in order to which client and server end is consistent, at two in file front portion and rear portion use
Size is identical.
Step 103: after successful match, the client determines the complete MD5 value to transmitting file and is sent to service
Device, the server receives the complete MD5 value and carries out precise search with the file of preservation, when retrieving same file
Afterwards, this is waited for that the transmitting file second passes to server by the client, terminates process.
Specifically, the client determines the complete MD5 value to transmitting file, specifically: the client determine it is to be passed
During the complete MD5 value of file, current disk utilization rate is judged before reading file content each time, if disk utilization rate is high
When threshold value, reduce the size for reading file content next time immediately.
It, should be to transmitting file then it is determined that the server storage area is existing if retrieving same file.
If not retrieving same file, from the beginning the client transmits file according to step 104.
Step 104: after it fails to match, the client is by file fragmentation to be passed and by every segment file and corresponding
MD5 value is separately sent to server, determined after one segmental file of the every reception of server MD5 value and with the client
Send the corresponding MD5 value of this section of segmental file be compared, if two MD5 values unanimously if save the segmental file, on the contrary,
The client retransmits the segmental file if two MD5 values are inconsistent.
Step 105: after the server has received all segmental files, the client and server is by each segmentation
The MD5 value of file is merged into MD5 value small documents, and determines the MD5 value of the MD5 value small documents, and the client is by the MD5
Value is sent to server and is compared, and if the same thinks that file transmission is complete, data are written in the server, if not phase
Together, then it is assumed that file transmission is imperfect, and the client is initiated request and transmitted from the beginning.
When the client transmits file from the beginning, it can also be transmitted using modes such as piecemeals.
It is all necessary that the client use is segmented every one section of biography (or one piece) when perhaps piecemeal transmits file to server
It assesses from initiating to the time between return, the optimum efficiency time is to respond for 2 seconds, if go-between is slack-off or services
When device response does not come, need to reduce the size of next section of (or next piece) file, when response was less than 2 seconds lower section (or
Lower piece of person) file then becomes larger and so on to being transmitted.
The embodiment of the present invention also provides a kind of super large file fragmentation and resumes system, which includes: client, server；
The client is for extracting the corresponding MD5 value of front portion and rear portion to transmitting file and being sent to clothes
Business device；It is also used to determine the complete MD5 value to transmitting file and is sent to server；It is also used to retrieve phase identical text when server
After part, this is waited for that the transmitting file second passes to server；
The server is for receiving two parts of MD5 values and carrying out Rapid matching with the file of preservation；After successful match,
It is also used to receive the complete MD5 value and carries out precise search with the file of preservation.
The client is also used to after it fails to match, by file fragmentation to be passed and by every segment file and corresponding
MD5 value is separately sent to server；It is also used to retransmit the segmental file if two MD5 values are inconsistent；
The server determines MD5 value after being also used to one segmental file of every reception and is somebody's turn to do with what the client was sent
The corresponding MD5 value of section segmental file is compared, if two MD5 values unanimously if save the segmental file, on the contrary, if two
MD5 value is inconsistent, and the client retransmits the segmental file.
After the server is also used to receive all segmental files, the MD5 value of each segmental file is merged into MD5 value
Small documents, and determine the MD5 value of the MD5 value small documents；Receive the MD5 value for the MD5 value small documents that the client is sent simultaneously
And be compared, if the same thinking that file transmission is complete, data are written in the server,
The client is also used to after server has received all segmental files, and the MD5 value of each segmental file is closed
And at MD5 value small documents, and determine the MD5 value of the MD5 value small documents；Server is sent by the MD5 value to be compared,
If the server compares not identical, then it is assumed that file transmission is imperfect, initiates request and transmits from the beginning.
The client is specifically used for reading file content each time during determining the complete MD5 value to transmitting file
The preceding current disk utilization rate of judgement reduces read the big of file content next time immediately if disk utilization rate is higher than threshold value
It is small.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (8)
1. a kind of super large file fragmentation continuous transmission method, which is characterized in that this method comprises: client is extracted to the previous of transmitting file
The corresponding with rear portion MD5 value in part and be sent to server, server two parts of MD5 values of reception and with preservation
File carries out Rapid matching, and after successful match, the client determines the complete MD5 value to transmitting file and is sent to service
Device, the server receives the complete MD5 value and carries out precise search with the file of preservation, when retrieving same file
Afterwards, this is waited for that the transmitting file second passes to server by the client.
2. super large file fragmentation continuous transmission method according to claim 1, which is characterized in that this method further include: work as matching
After failure, the client is separately sent to server by file fragmentation to be passed and by every segment file and corresponding MD5 value,
The server is every to be received and determines MD5 value and corresponding with this section of segmental file that the client is sent after a segmental file
MD5 value be compared, if two MD5 values unanimously if save the segmental file, on the contrary, if two MD5 values are inconsistent
The client retransmits the segmental file.
3. super large file fragmentation continuous transmission method according to claim 2, which is characterized in that this method further include: the clothes
After business device has received all segmental files, the MD5 value of each segmental file is merged into MD5 value by the client and server
Small documents, and determine the MD5 value of the MD5 value small documents, the client sends server for the MD5 value and compares
It is right, if the same think that file transmission is complete, data are written in the server, if it is not the same, then thinking file transmission not
Completely, the client is initiated to request to transmit from the beginning.
4. super large file fragmentation continuous transmission method according to claim 1 or 3, which is characterized in that the client determine to
The complete MD5 value of transmitting file, specifically: the client is read each time during determining the complete MD5 value to transmitting file
Current disk utilization rate is judged before file content, if disk utilization rate is higher than threshold value, reduces read file next time immediately
The size of content.
5. a kind of super large file fragmentation resumes system, which is characterized in that the system includes: client, server；
The client, for extracting the corresponding MD5 value of front portion and rear portion to transmitting file and being sent to service
Device；It is also used to determine the complete MD5 value to transmitting file and is sent to server；It is also used to retrieve same file when server
Afterwards, this is waited for that the transmitting file second passes to server；
The server, for receiving two parts of MD5 values and carrying out Rapid matching with the file of preservation；After successful match, also
For receiving the complete MD5 value and carrying out precise search with the file of preservation.
6. super large file fragmentation according to claim 5 resumes system, which is characterized in that the client is also used to work as
After it fails to match, server is separately sent to by file fragmentation to be passed and by every segment file and corresponding MD5 value；It is also used to
The segmental file is retransmitted if two MD5 values are inconsistent；
The server is also used to the section for determining MD5 value after one segmental file of every reception and sending with the client
The corresponding MD5 value of segmental file is compared, if two MD5 values unanimously if save the segmental file, on the contrary, if two
MD5 value is inconsistent, and the client retransmits the segmental file.
7. super large file fragmentation according to claim 6 resumes system, which is characterized in that the server is also used to connect
After harvesting all segmental files, the MD5 value of each segmental file is merged into MD5 value small documents, and determine the small text of MD5 value
The MD5 value of part；It receives the MD5 value for the MD5 value small documents that the client is sent and is compared, if the same think text
Part transmission is complete, and data are written in the server,
The client is also used to after server has received all segmental files, and the MD5 value of each segmental file is merged into
MD5 value small documents, and determine the MD5 value of the MD5 value small documents；Server is sent by the MD5 value to be compared, if
The server compares not identical, then it is assumed that file transmission is imperfect, initiates request and transmits from the beginning.
8. the super large file fragmentation according to claim 5 or 7 resumes system, which is characterized in that the client is specific to use
During determining the complete MD5 value to transmitting file, current disk utilization rate is judged before reading file content each time, if
When disk utilization rate is higher than threshold value, reduce the size for reading file content next time immediately.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810438851.2A CN110474937A (en) | 2018-05-09 | 2018-05-09 | Super large file fragmentation continuous transmission method and its system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810438851.2A CN110474937A (en) | 2018-05-09 | 2018-05-09 | Super large file fragmentation continuous transmission method and its system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110474937A true CN110474937A (en) | 2019-11-19 |
Family
ID=68503723
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810438851.2A Pending CN110474937A (en) | 2018-05-09 | 2018-05-09 | Super large file fragmentation continuous transmission method and its system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110474937A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114422503A (en) * | 2022-01-24 | 2022-04-29 | 深圳市云语科技有限公司 | Method for intelligently selecting file transmission mode of multi-node file transmission system |
| CN114615258A (en) * | 2022-03-28 | 2022-06-10 | 重庆长安汽车股份有限公司 | Method and device for uploading large files to file server in fragmented manner |
| CN116032913A (en) * | 2022-10-12 | 2023-04-28 | 浪潮云信息技术股份公司 | File uploading method and system based on distributed object storage |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105139623A (en) * | 2015-09-29 | 2015-12-09 | 光一科技股份有限公司 | File compression remote upgrading method for collector multistage transmission based on broadband carrier waves |
| CN105635324A (en) * | 2016-03-17 | 2016-06-01 | 新浪网技术(中国)有限公司 | Big file uploading and continuous uploading method and device for browser or server |
| CN106708876A (en) * | 2015-11-16 | 2017-05-24 | 任子行网络技术股份有限公司 | Similar video retrieval method and system based on Lucene |
| US20170180137A1 (en) * | 2015-12-21 | 2017-06-22 | Electro Industries/Gauge Tech | Providing security in an intelligent electronic device |
-
2018
- 2018-05-09 CN CN201810438851.2A patent/CN110474937A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105139623A (en) * | 2015-09-29 | 2015-12-09 | 光一科技股份有限公司 | File compression remote upgrading method for collector multistage transmission based on broadband carrier waves |
| CN106708876A (en) * | 2015-11-16 | 2017-05-24 | 任子行网络技术股份有限公司 | Similar video retrieval method and system based on Lucene |
| US20170180137A1 (en) * | 2015-12-21 | 2017-06-22 | Electro Industries/Gauge Tech | Providing security in an intelligent electronic device |
| CN105635324A (en) * | 2016-03-17 | 2016-06-01 | 新浪网技术(中国)有限公司 | Big file uploading and continuous uploading method and device for browser or server |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114422503A (en) * | 2022-01-24 | 2022-04-29 | 深圳市云语科技有限公司 | Method for intelligently selecting file transmission mode of multi-node file transmission system |
| CN114422503B (en) * | 2022-01-24 | 2024-01-30 | 深圳市云语科技有限公司 | Method for intelligently selecting file transmission mode by multi-node file transmission system |
| CN114615258A (en) * | 2022-03-28 | 2022-06-10 | 重庆长安汽车股份有限公司 | Method and device for uploading large files to file server in fragmented manner |
| CN116032913A (en) * | 2022-10-12 | 2023-04-28 | 浪潮云信息技术股份公司 | File uploading method and system based on distributed object storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9792297B2 (en) | File folder transmission on network | |
| CN110474937A (en) | Super large file fragmentation continuous transmission method and its system | |
| US9785664B2 (en) | Gathering transaction data associated with locally stored data files | |
| US9300721B2 (en) | Method, apparatus and system for file transfer based on file directory | |
| US20160157047A1 (en) | Bluetooth communication method and bluetooth communication device | |
| CN101616503B (en) | Method and device for sharing telephone number information | |
| CN103516580B (en) | A kind of method and system that message sink and forwarding are realized based on message queue | |
| CN101494834A (en) | Method, system and mobile communication terminal for retransmitting information | |
| CN102377684B (en) | The method of the pornographic video file transfer of a kind of automatic blocking-up, terminal and system | |
| CN105224676A (en) | The method of mobile terminal file migration and device | |
| CN103685343A (en) | File transfer method and file transfer system | |
| KR100624142B1 (en) | Gateway apparatus and method for wireless data service | |
| CN105165035A (en) | Transmission of multimedia message doubled with transmission of text message | |
| CN103648085B (en) | A kind of mobile terminal message transmission method overcoming network jitter | |
| CN102821148A (en) | Method and device for optimizing CIFS (common internet file system) application | |
| US9607614B2 (en) | Terminal, server and information pushing method | |
| CN102624628B (en) | A kind of home gateway and the method realizing packet fast-forwarding thereof | |
| CN105024905A (en) | Message pushing method and device | |
| CN109766347B (en) | Data updating method, device, system, computer equipment and storage medium | |
| CN104754760A (en) | Packet service reconstruction method and terminal | |
| CN114070647A (en) | Video transmission system and video transmission method | |
| CN101827068A (en) | Business scenario reduction method and device | |
| CN110471739B (en) | Instruction retry method and device | |
| CN114020703A (en) | Large file uploading method used in narrow-band network environment | |
| CN102739777A (en) | Data stream disconnection recovering method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191119 |
|
| RJ01 | Rejection of invention patent application after publication |