CN103685509A - Method for synchronizing file delta - Google Patents
Method for synchronizing file delta Download PDFInfo
- Publication number
- CN103685509A CN103685509A CN201310683085.3A CN201310683085A CN103685509A CN 103685509 A CN103685509 A CN 103685509A CN 201310683085 A CN201310683085 A CN 201310683085A CN 103685509 A CN103685509 A CN 103685509A
- Authority
- CN
- China
- Prior art keywords
- file
- check value
- side computer
- data block
- service end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a method for synchronizing file delta. The method comprises the steps that A1) a client side computer is connected with a server side computer through a network, the client side computer can access a file A, the server side computer can access a file B, and the file A is similar to the file B; A2) the file B is divided by the server side computer; A3) a weak verification value and a strong veritication value are respectively calculated for each byte formed in a dividing mode; A4) the server side computer sends the weak verification values and the strong verification values to the client side computer; A5) the files are searched for by the client side computer; A6) the client side computer sends a protocol of reconstructing the file A to the server side computer, and file synchronization is achieved. Through file delta synchronization, long-distance storage space can be managed and used through the network anytime and anywhere, and data can be transmitted, stored, downloaded, shared and backed up.
Description
Technical field
The invention belongs to computer documents simultaneous techniques field, especially relate to and a kind ofly according to file change content, carry out the synchronous method of data.
Background technology
Along with the rapid rising of mobile Internet and the universalness of mobile terminal, excited the transmission demand of customized information on contact, multiple terminals.And along with the maturation in on-line storage market and popularizing of idea, and client is more and clearer and more definite to the demand of on-line storage, how better this class Preserving Electronic Information, application, management to be become to a popular demand.Everybody wishes a quick and easy on-line storage space, realizes storage administration anywhere or anytime and application.That can without time without ground, by network, come beyond the clouds management and using remote storage space, realize transmission, the storage of data, the demand of downloading, sharing and back up highlights all the more.Utilize cloud memory technology, user can carry out digital information seamless synchronous between the multiple terminalss such as PC, mobile phone, PAD, and user can pass through PC transaction file, browses the file after renewal afterwards at mobile phone terminal.
Summary of the invention
Cloud memory function will realize good user and experience, and file synchronization speed and flow are the parts of most critical wherein, and file residual quantity synchronous method produces for addressing this problem.
The technical solution used in the present invention is: a kind of file residual quantity synchronous method, comprises step:
A1) client computer is connected by network with service end computer, described client computer can be accessed A file, and described service end computer can be accessed B file, and file A is close with B;
A2) service end computer is cut apart file B, is divided into the data block S of continuous nonoverlapping fixed size;
A3) by divided each piece byte, calculate respectively a weak check value and a strong check value, described weak check value is used for distinguishing different, and strong check value is used for confirming identical;
A4) service end computer sends to client computer by weak check value and strong check value;
A5) data block that all sizes of client computer search file A are S; Find a certain data block that has identical weak check code and strong check code with file B;
A6) client computer sends to service end computer to rebuild the agreement of A file; Each instruction, or be a data block not matching with any one data block of file B, or be that service end computer has this data block and the proof that do not need to send; Thereby realized the synchronous of A file.
Accompanying drawing explanation
Fig. 1 is client and service end networking schematic diagram;
Fig. 2 is synchronization principles schematic diagram.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in further details.
If client and service end belong to respectively two computer A lpha and Beta, computer A lpha can access A file, computer Beta can access B file, when file A similar with B, two computer A lpha are being connected with Beta and for example, by public network low speed (mobile network), as shown in Figure 1.
Residual quantity synchronized algorithm step:
The first step: Beta is divided into file B the piece S of continuous nonoverlapping fixed size, and last piece may be less than S byte;
Second step: each the piece byte by Beta, calculates two check values: the weak verification rolling checksum (adler-32) of 32 and one 128 strong verifications (MD5).Weak adler-32 is different for difference, and strong MD5 is identical for confirming;
The 3rd step: Beta sends to Alpha by these check values;
The data block (side-play amount can be optional, and not necessarily leaveing no choice but is the multiple of S) that all sizes of the 4th step: Alpha by search file A are S, finds a certain data block that has identical weak check code and strong check code with file B;
The 5th step: Alpha sends to Beta to rebuild the agreement of A file; Each instruction, or be a data block not matching with any one data block of file B, or be that Beta has this data block and the proof that do not need to send.
Last result is that Beta has obtained A file, as shown in Figure 2.
In residual quantity synchronized algorithm, use rolling digest algorithm, the summary that rolls has given x1 and to the check value of xn, also has the value of x1 and x n+1, just can be very fast calculate x2 to the characteristic of xn+1 check value.
What weak checking algorithm adopted is the adler-32 verification of Mark Adler, and we high-ranking officers test and are defined as:
s(k,l)=a(k,l)+2
16b(k,l)
S (k, l) is that xk is to the rollback verification of xl, for simply and fast we are defined as 2 by M
16, the important characteristic of verification is by using recurrence relation can calculate very efficiently successive value, the piece that is therefore S by the length that a rollback mode verification can be calculated all possible skew.The pattern of this incremental computations has greatly improved the computational speed of check value.
When file is carried out to stripping and slicing, block size has a direct impact network data transmission amount, and if block is excessive, and variance data transmission quantity is just larger, so file stripping and slicing principle is as follows:
Acquiescence is divided into file to be no more than 1000 pieces, and this parameter represents with def_block_cnt;
1) every optional size represents with min_block_size from this parameter of minimum 1k() according to multiple, be incremented to this parameter of maximum 512k(and represent with max_block_size); be that the optional value of block size is: block_list=1k, 2k, 4k; 8k; 16k, 32k, 64k; 128k; 256k, 512k divided by default tile number def_block_cnt, calculates each block size tmp_block_size by file size;
2) if tmp_block_size is less than or equal to smallest block size min_block_size:
Actual block size cur_block_size=min_block_size
Actual block number cur_blockl_cnt=file_size/cur_block_size;
3) if tmp_block_size is more than or equal to largest block size max_block_size:
Actual block size cur_block_size=max_block_size
Actual block number cur_blockl_cnt=file_size/cur_block_size
If tmp_block_size is less than largest block size max_block_size, block size cur_block_size gets in block_list larger than tmp_block_size, but approach the block size value of tmp_block_size most, recalculate afterwards piece number cur_blockl_cnt=file_size/cur_block_size.
Above content is in conjunction with concrete preferred implementation further description made for the present invention, can not assert that specific embodiment of the invention is confined to these explanations.For general technical staff of the technical field of the invention, without departing from the inventive concept of the premise, can also make some simple deduction or replace, all should be considered as belonging to protection scope of the present invention.
Claims (4)
1. a file residual quantity synchronous method, comprises step:
A1) client computer is connected by network with service end computer, described client computer can be accessed A file, and described service end computer can be accessed B file, and file A is close with B;
A2) service end computer is cut apart file B, is divided into the data block S of continuous nonoverlapping fixed size;
A3) by divided each piece byte, calculate respectively a weak check value and a strong check value, described weak check value is used for distinguishing different, and strong check value is used for confirming identical;
A4) service end computer sends to client computer by weak check value and strong check value;
A5) data block that all sizes of client computer search file A are S; Find a certain data block that has identical weak check code and strong check code with file B;
A6) client computer sends to service end computer to rebuild the agreement of A file; Each instruction, or be a data block not matching with any one data block of file B, or be that service end computer has this data block and the proof that do not need to send; Thereby realized the synchronous of A file.
2. file residual quantity synchronous method as claimed in claim 1, is characterized in that:
Described steps A 3) in, weak check value is adler-32, and strong check value is MD5.
3. file residual quantity synchronous method as claimed in claim 2, is characterized in that: what weak checking algorithm adopted is the adler-32 verification of Mark Adler, and verification is defined as:
s(k,l)=a(k,l)+2
16b(k,l)
S (k, l) is that xk is to the rollback verification of xl.
4. file residual quantity synchronous method as claimed in claim 1, is characterized in that: while carrying out file division described steps A 2), file is divided into and is no more than 1000 pieces, and every optional size is incremented to maximum 512k from minimum 1k according to multiple.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310683085.3A CN103685509A (en) | 2013-12-12 | 2013-12-12 | Method for synchronizing file delta |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310683085.3A CN103685509A (en) | 2013-12-12 | 2013-12-12 | Method for synchronizing file delta |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103685509A true CN103685509A (en) | 2014-03-26 |
Family
ID=50321784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310683085.3A Pending CN103685509A (en) | 2013-12-12 | 2013-12-12 | Method for synchronizing file delta |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103685509A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105100274A (en) * | 2015-08-31 | 2015-11-25 | 北京奇虎科技有限公司 | File uploading/downloading method and system in web environment, client and server |
| CN105554081A (en) * | 2015-12-09 | 2016-05-04 | 华为技术有限公司 | File difference transmission method and device |
| CN105872017A (en) * | 2016-03-18 | 2016-08-17 | 清华大学 | Method and apparatus for carrying out file differential encoding synchronization at web page side |
| CN107480267A (en) * | 2017-08-17 | 2017-12-15 | 无锡清华信息科学与技术国家实验室物联网技术中心 | A kind of method that file difference synchronizing speed is improved using locality |
-
2013
- 2013-12-12 CN CN201310683085.3A patent/CN103685509A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105100274A (en) * | 2015-08-31 | 2015-11-25 | 北京奇虎科技有限公司 | File uploading/downloading method and system in web environment, client and server |
| CN105554081A (en) * | 2015-12-09 | 2016-05-04 | 华为技术有限公司 | File difference transmission method and device |
| WO2017097106A1 (en) * | 2015-12-09 | 2017-06-15 | 华为技术有限公司 | Method and apparatus for transmitting file difference |
| CN105554081B (en) * | 2015-12-09 | 2019-01-18 | 华为技术有限公司 | A kind of transmission method and device of file delta |
| CN105872017A (en) * | 2016-03-18 | 2016-08-17 | 清华大学 | Method and apparatus for carrying out file differential encoding synchronization at web page side |
| CN107480267A (en) * | 2017-08-17 | 2017-12-15 | 无锡清华信息科学与技术国家实验室物联网技术中心 | A kind of method that file difference synchronizing speed is improved using locality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11569982B2 (en) | Blockchain compression using summary and padding blocks | |
| CN110019211A (en) | The methods, devices and systems of association index | |
| CN103873505B (en) | Method, system and the device uploaded offline to cloud storage service device | |
| CN103685509A (en) | Method for synchronizing file delta | |
| CN104104717A (en) | Inputting channel data statistical method and device | |
| US11620260B2 (en) | Record property synchronization in a network computing system | |
| CN103338249A (en) | Cache method and device | |
| CN104657401A (en) | Web cache updating method | |
| CN104579970A (en) | Strategy matching method and device of IPv6 message | |
| CN102946410A (en) | Method and device for network synchronization | |
| CN103581224A (en) | Method and device for pushing information | |
| CN104899236A (en) | Comment information display method, comment information display device and comment information display system | |
| CN103841180A (en) | Network data synchronization method and device based on operating instruction, terminal device and server | |
| CN105099796A (en) | Detection method and device for multimedia content replacement and resource spreading system | |
| US11159616B2 (en) | Email synchronization method and apparatus, and computer readable storage medium | |
| US9633035B2 (en) | Storage system and methods for time continuum data retrieval | |
| CN104503862B (en) | The method and apparatus for obtaining the check value using channel bag | |
| CN106372109A (en) | Internet resource file caching method and apparatus | |
| Ramaprasath et al. | Performance optimization of big data in mobile networks | |
| CN105872017A (en) | Method and apparatus for carrying out file differential encoding synchronization at web page side | |
| CN109144991B (en) | Method and device for dynamic sub-metering, electronic equipment and computer-storable medium | |
| CN110851794A (en) | Media file uplink method and device, storage medium and electronic device | |
| CN104063377A (en) | Information processing method and electronic equipment using same | |
| CN109617708A (en) | A kind of compression method burying a log, equipment and system | |
| US10133801B2 (en) | Providing data aggregated from various sources to a client application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 4, 01-11 building, 518000 / F, Changhong technology building, 18 South twelve Road, Nanshan District, Guangdong, Shenzhen Applicant after: Polytron Technologies Inc Address before: 4, 01-11 building, 518000 / F, Changhong technology building, 18 South twelve Road, Nanshan District, Guangdong, Shenzhen Applicant before: Shenzhen City Richinfo Technology Co., Ltd. |
|
| COR | Change of bibliographic data | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140326 |