CN101495983A

CN101495983A - Bidirectional data transfer optimization and content control for networks

Info

Publication number: CN101495983A
Application number: CNA2005800479053A
Authority: CN
Inventors: G·舍费; O·索梅克
Original assignee: OBVIOUS Ltd B
Current assignee: OBVIOUS Ltd B
Priority date: 2004-12-08
Filing date: 2005-12-08
Publication date: 2009-07-29
Also published as: ZA200705615B

Abstract

A method of transferring data objects over a network comprises intercepting a network transfer message with a passing object, creating a unique identifier for the object using a predetermined function, the same function having been used to provide identifiers for objects stored at predetermined nodes of said network, removing the object and sending on the network transfer message with the unique identifier in place of the object. Then, at the recipient end it is possible to obtain the unique identifier and use it as a key to search for a corresponding object in the local nodes. The search starts with a node closest to the recipient and steadily spreads outwards. The object when found is reattached for the benefit of the recipient and network bandwidth has been saved by the avoidance of redundant transfer since the object is brought to the recipient from the node which is the closest to him.

Description

The content control of network and bidirectional data transfer optimization

Invention field and background

The present invention relates to the content control and the two-way object transfer optimization of network, and relate in particular to such network bi-directional object transfer, it reduces object transfer redundant on network, and can carry out the operation of content and do not invade any privacy, thereby improved network utilization and obtained the wherein control of the data of transmission.

Present network service mainly is based on predefined object, such as file, webpage, e-mail annex or the like.Can come on diverse network, to share these objects by various communication meanss, comprise the Internet, Cellular Networks, Intranet or the like.Many to as if extensively shared, thereby by on existing the connection, transmitting again and again.These objects repeat to transmit the remarkable increase that has caused bandwidth lose, and the load and the stand-by period of therefore having improved server.Redundancy on the network causes network performance to reduce through regular meeting, and therefore need carry out extra investment on network infrastructure.

Traditional object transfer causes load to the network path from the transmitting terminal to the receiving end, causes the high offered load on these network paths, high stand-by period and low performance.

Present data communication network allows almost without any the transmission data object of restriction, and this can cause the transmission of invalid data, such as virus/worm/have material of copyright protection or the like.The mail of the very high number percent that in fact is transmitted is to duplicate for a plurality of of identical uncalled advertisement, so-called rubbish or advertisement matter.

Present object conversion for example in the conversion that is applicable between the dissimilar form of different operating system or hardware architecture, is usually directed to manual search and/or cpu intensive process.

Present solution

The electronics verification of cryptographic system key, hashing and file

Another problem related to the present invention is the e-file verification.Along with the quick growth of the use of e-file, it is not only consuming time that the content of each file in the desk checking file system becomes, but also the mistake during having caused checking, and be infeasible therefore.

In the computational analysis in early days, electronics verification file integrality begins to play the part of important role.Because the data of storing in suspicious dish are fragile, but also need be retained with testifying, often require the assayer to obtain the mirror image accurately of suspicious disk drive, to carry out the inspection of complexity.For this reason, need a kind of strong cryptographic hash function, it can provide a kind of useful and mode easily, the integrality of coming checking data for the examiner.That is to say that this hash function is the function of bit sequence in the file.If bit sequence has changed, mean that the someone has distorted this document, the hash function that is produced has also changed.So just can determine whether this driver was distorted.

Several known hashing algorithms that are used for cryptographic system are arranged.Comprise following algorithm:

● the eap-message digest hash function, MD2, MD4 and MD5 are used for message is upset to become littler value, are called eap-message digest.

● secure hash algorithm (sha), a kind of canonical algorithm produces bigger (160 bit) eap-message digest, and is similar to MD4.

At present, encryption key is mainly used in the file integrality verification in storer and the network system.

The mathematical principle of hash function

The mathematical theory of hash function provides following attribute:

● if file F has provided a hashed value H1, and each individual bit of H1 all is the function of all bits of F so.

● if file F has provided a hashed value H1, and an individual bit revising F so will cause a diverse hashed value.

● if file F has provided a hashed value H1, has provided another then and has been not equal to the hashed value H2 of H1, is the part (such as revising last 10 bytes) that impossible have a mind to revise F on calculating, and makes that the hashed value of the new file of revising is H2.

● the chance that two files of selecting at random have same Hash value is extremely small.For example, probability with MD5 hashed value of 128 identical bits of two files is 1/ (2 ¹²⁸), be substantially equal to 1/ (3.4 * 10 ³⁸), perhaps roughly be 340 ten hundred million ten hundred million ten hundred million part per billion probability.It can be compared with showing the scene in the life: announcement get (a kind of recreation of making wild with joy in Hong Kong of Hong Kong lottery ticket, select the number between 61 to 47 at random) probability of the first prize is 10,737,1/573rd, the getting probability that Pennsylvania, United States super 6 makes wild with joy and be 3,900 ten thousand of announcement/.Therefore, chance with identical MD5 hashed value of two files is similar to and gets 30 * 10 ³⁰Inferior Hong Kong lottery ticket first prize, two files have the probability of identical SHA-1 hashed value even littler, because the SHA-1 hashed value is 160 bits.

The WAN optimal solution

The WAN prioritization scheme makes the user to come mobile more information with lower expense and more performance.The multiple solution that illustrates below is developed to improve the efficient that WAN connects.

Basic buffer memory

Internet Forum has stood to repeat to transmit the poor efficiency of unaltered file for a long time on WAN.In order to address this problem, a large amount of file caches and file distribution scheme have been developed.The webpage buffer memory is adopted the use that reduces bandwidth by many ISPs, and it checks at first by before downloading webpage in request from actual website whether the server that closes on this user has the webpage copy of buffer memory, solves similar problem.If found this webpage at local cache, it is directly sent from local cache, thereby has avoided downloading by WAN once more the needs of webpage.

The grouping shaping

The grouping shaping is used to distribute limited bandwidth resources, to mate the right of priority of a company.Important or the business of delay-sensitive is sent out on WAN connects prior to flow more unessential, that more can tolerate delay.

Basic compression

Data compression at present is used to multiple application, comprises that digital music, cellular phone network and satellite video transmit.Many suboffices router is supported the data compression of various forms of network-orienteds, comprises the compression of IP packet header and payload.The data compression of simple form is to come the redundant mode in the alternative data flow to come work by identification and with littler symbol.

Supercompression

Supercompression combines the metadata cache solution of basic compression and pattern-recognition scheme and innovation.Although the supercompression scheme is similar to basic compression, the supercompression algorithm be use and divide into groups unknowable, thereby can obtain potential bigger compressibility.

In brief, supercompression does not adopt traditional scope with compression of basic compression to be restricted to the mode of file or grouping level.

Pass through the pattern of its business in the compressing member inspection of transmitting terminal.The question blank or the dictionary of a unique bit mode of transmitting terminal exploitation.Adopt identical algorithm, question blank/dictionary that the receiving end exploitation is identical.Transmitting terminal is seen a pattern of seeing before it next time, and it can delete this pattern, and it is replaced with little mark or symbol.This mark is much smaller than the pattern that it substituted.When receiving end is seen this mark, its with the dictionary of having developed with this mark or symbol " translation " time unique bit mode.Receiving end is delivered to object computer with the information that recovers then.

Past also is called supercompression " a kind of agreement that is used to delete the redundant network flow is technology independently ".

Although it is relevant with supercompression that term " buffer memory " is often used in, supercompression is not really to have a traditional buffer memory.Buffer memory in the supercompression relates to the nearest history of streams of trace data.Data stream is not divided into object, and in most of the cases, any object that relates in the stream was blended in the history according to the time that each segmentation reached of this object.Such mixing has prevented that object quickens completely.Yet it obtains the better compression to multiimage really.Referring to Fig. 1, it illustrates the form that data stream 2 may occur on network, and illustrate it then and how to occur 4 in traditional buffer memory, and last 6, how it appears in the supercompression history buffer.

Two-way time latency management

End-to-end connection stand-by period or two-way time, the effective throughput that connects for the Internet has far-reaching influence.In order to guarantee that all groupings that are transmitted all arrive their destination, developed transmission control protocol such as TCP, confirm when they successfully receive packet, to send.Have only after the sender receives affirmation signal from the recipient, the sender just sends more data.Therefore, the two-way time between 2 is long more, and it is just long more to send the required time of file.A kind of result of this scheme is that the distance of two end points connecting is long more, and the available bandwidth of this connection is just few more.Have several schemes can alleviate the problems referred to above:

● the management of tcp window size

● TCP begins management slowly

● forward error correction

● application protocol optimization

Multipath route based on strategy

Connection on the not all the Internet or path all provide identical characteristic, and the different routers between identical two set points can provide two results that difference is very big on the network.Some paths may have low latency, high-throughput characteristic, and other has that height is lost, high stand-by period characteristic.The multipath route makes the data transmission with different business that require advance through optimal path on the network.For example, can be routed through a more expensive low latency path the business of delay-sensitive, and electronic mail service can be routed path more cheap, lower throughput through, higher latency.

Low bandwidth file system (LBFS)

Referring now to Fig. 2,, it is a process flow diagram that shows the file system that is called LBFS, is used to avoid the redundant transmission of the data on low-bandwidth network.LBFS is a kind of network file system(NFS) that is in particular the low-bandwidth network design, and the suitable LAN of office or similar.In order to reduce bandwidth requirement, LBFS has adopted the intersection document similarity.In order to utilize the similarity between the file, the LBFS file server is divided into bulk with the file of its storage, and comes these bulks of index with hashed value.The big lasting file cache of reprinting books in a reduce format that LBFS client is also similar.When transfer files between client and server, the data bulk that LBFS identification recipient has had in alternative document, and avoid on network, transmitting this redundant data.On client and server, LBFS is set to one group of file of index, can avoid the data bulk that sends to identify on network.In order to save the transmission of bulk, LBFS depends on the collision of SHA-1 hash function and resists attribute.Two inputs to SHA-1 produce the probability of the probability of identical output well below hardware bit mistake.Thereby LBFS has followed the convention that the supposition of accepting extensively does not have Hash collision.If client and server all has the data bulk that produces identical SHA-1 hashed value, they suppose that these two bulks are actual identical bulks, and avoid in its content of transmission over networks.

The mode of operation of LBFS is similar to some to reciprocity by the time implementation, also has special supplementary features, and it has increased a file cache.This document buffer memory temporarily is stored in the file of a given position, and makes the client can not exist in buffer memory by network requests, and file that is not modified simultaneously or file fragmentation.

LBFS has following aspect:

● LBFS is implemented as a kind of certain protocol and revises.This is not the universal solution for all object transfer problems.

● the LBFS method depends on the specific protocol information of acquisition for this method, with operational example such as file path.It does not only realize the caching system based on object information itself.

● LBFS is applicable to a kind of inquiry rank method, and promptly it disturbs in Object Query to obtain its target.

● LBFS is applicable to the method that a kind of content is known.Client cache is known the type of the information that it is held.This is disadvantageous, because service supplier is not want the data of their transmission are responsible for.Therefore a kind of suitable method should not known data.

● LBFS can not utilize and the network path network path inequality from the transmitting terminal to the receiving end.That is to say it only is concerned about to send what data, rather than how to send.

● under object was not present in situation in the destination object buffer memory, LBFS can not reduce the transmission time.

● LBFS can not distribute network path from the transmitting terminal to the receiving end and the offered load between other network paths.

● LBFS need revise client and server, to carry out the flow optimization.

● LBFS is fragile for the cache hit of mistake.Determine cache hit because LBFS only depends on summary information, and can represent different payloads with identical summary information, thus the cache hit that may make a mistake just, and wrong payload may replace being sent out in original payload.

The identification of P2P object

To waiting until in the peer protocol, encryption key is used to object identification several.All files are given hashed value.This hashed value allows each user by using unique hashed value, can both find for the institute of a specific file actively, and is independent of any filename that each user may rise to this document.In addition, file is divided into the data sementation of 9.28MB.Each segmentation is given the hashed value of oneself.For example, the file of a 600MB can comprise 65 segmentations, and every part all receives its hashed value.From these part hashed values, create a hashed value then, and can be used in the network by the file of this hashed value sign for whole file.

Repeat transmission and detect (DTD)

Repeating transmission detection (DTD) is a kind of system that allows any network-caching to delete all redundant HTTP payload transmission potentially.DTD is a kind of specific HTTP modification, and it has added an eap-message digest field to the HTTP stem, thereby can detect redundant HTTP transmission.

The DTD protocol data-flow has been shown in table 1.

The difference of DTD is following aspect:

● DTD is implemented as a kind of certain protocol and revises.This is not the universal solution for all object transfer problems.

● the DTD method depends on the specific protocol information of acquisition for method of operating, for example URL.It does not only realize the caching system based on object information itself.

● DTD uses a kind of inquiry rank method, and promptly it disturbs in Object Query to obtain its target.

● the method that DTD uses a kind of content to know.Client cache is known the type of the information that it holds.This is disadvantageous.

● DTD can not utilize and the network path network path inequality from the transmitting terminal to the receiving end.

● the place one's entire reliance upon agency's that inquired about inner buffer of DTD.Under object was not present in situation in the destination object buffer memory, DTD can not reduce the transmission time.

● DTD can not distribute network path from the transmitting terminal to the receiving end and the offered load between other network paths.

● DTD need revise client and server, to carry out flow optimization.

● DTD is fragile for the cache hit of mistake.DTD only depends on summary information and determines cache hit, and can represent different payloads with identical summary information, the cache hit that can make a mistake, and wrong payload may replace being sent out in original payload.

The shortcoming of present solution

Basic buffer memory

The shortcoming of basic buffering scheme is:

● it is a kind of main scheme for network object.It can not be applied to all digital objects.

● it is that the WWW of HTML (Hypertext Markup Language) (http) or the scheme of any other similar network are used in a kind of main sensing.It can not be applied to all data communication network environment.

● it is a kind of scheme that depends on the certain protocol parameter, HTTP URL for example, and therefore can not be applied to all digital object transmission, and no matter their transmission context.

● basic buffer memory makes it become the scheme that a kind of content is known at the fact of HTTP URL of certain protocol parameter-for example.This can cause the network provider is that the content of the object that quickens is born legal responsibility.

● it can not utilize the network path that is different from the network path from sender to recipient.

● do not exist in the destination object buffer memory under the situation of object, it can not reduce the transmission time.

● it can not distribute network path from the transmitting terminal to the receiving end and the offered load between other network paths.

● it can not guarantee that the information of buffer memory upgrades at any time.

● its its inner buffer that places one's entire reliance upon.When needed information was not in inner buffer, it can not provide flow to quicken.One of top symptom is inefficient in zero-time.

● basic buffer memory depends on the fact of HTTP URL of certain protocol parameter-for example, prevents that its deletion has the redundant data transmissions of unmatched protocol parameter, for example for the different URL of same file.

The grouping shaping

The grouping shaping can not deleted the redundant data transmissions in the network.It just alleviates its symptom.

Basic compression

Basic compression scheme has following shortcoming:

● they can not delete the redundant data transmissions in the network.

● in most of the cases, they can not quicken whole redundant object.By the compression redundant information, rather than deletion, realize the minimizing of transmitting.

● it is the intensive scheme of a kind of CPU.

● it is unhelpful for incoercible object.Many file types have been compressed.

● it can not provide the minimum transfer stand-by period for redundant transmission.

Supercompression

The shortcoming of supercompression is:

● it can not delete the redundant data transmissions in the network.

● in most of the cases, they can not quicken whole object.In most of times, it does not accept whole object, and when it was accepted, it also only reduced the cost of redundant transmission.

● it is the intensive scheme of a kind of CPU.

● it is unhelpful for incoercible object.

● it depends on use and comes the buffer memory network packet based on the buffer memory of relatively little storer, so it is restricted to timely detection and excavates the quite communication redundancy of localization.

● it does not have about the application that generates (redundancy) network traffics or the information of server, so where it can not predicted data may be used to, and can not predict the potential further acceleration that data provided in the far-end buffer memory and optimize network traffics.

Two-way time latency management

Two-way time, latency management can not deleted the redundant data of transmitting in the network.It just attempts alleviating its symptom.

Multipath route based on strategy

Multipath route based on strategy can not deleted the redundant data of transmitting in the network.It just attempts alleviating its symptom.

LBFS

The shortcoming of basic buffer memory is:

● it is a kind of only for the scheme of the file in the NFS file system.It can not be applicable to all digital objects.

● it is a kind of only for the scheme of the environment that allows NFS.

● it can not be applicable to all data communication network environment.

● it is a kind of scheme that depends on specific NFS file path information, therefore can not be applicable to all digital object transmission, and no matter their transmission context.

● LBFS depends on the NFS file path, makes it become a kind of scheme of knowing content.This can cause bearing legal responsibility for the content of the object that quickens.

● it need revise client and server, so that can carry out flow optimization.

● LBFS is fragile for the cache hit of mistake.LBFS only depends on summary information and determines cache hit, and can represent different payloads with identical summary information, the cache hit that can make a mistake, and wrong payload may replace being sent out in original payload.

The identification of P2P object

The identification of P2P object can not deleted the redundant data of transmitting in the network.

The shortcoming of DTD:

The shortcoming of DTD is:

● DTD is a kind of only for the scheme of http protocol.It can not be applicable to all digital objects.

● it can not be applicable to all data communication network environment.

● it is a kind of scheme that depends on specific URL path, therefore can not be applicable to all digital object transmission, and no matter the transmission context.

● the fact that DTD depends on URL makes it become the scheme that a kind of content is known.This can cause bearing legal responsibility for the content of the object that quickens.

● it need revise client and server, so that can carry out flow optimization.

● it can only optimize complete object, can not optimize the part object.

● DTD client need use some heuristics, such as to comprise "? " URL do not send extra HEAD request.

● some webserver is forward abstract never.Be noted that and do not require that http server sends the example summary, does not have mechanism can find whether server can forward abstract at present yet.Thereby the client can stand the problem for given server, and can not obtain any benefit.

● DTD is fragile for the cache hit of mistake.DTD only depends on summary information and determines cache hit, and can represent different payloads with identical summary information, the cache hit that can lose efficacy, and wrong payload may replace being sent out in original payload.

Have understanding widely, need a kind of network data transmission system that can avoid above-mentioned restriction, this also is very favorable.

Summary of the invention

According to an aspect of the present invention, provide the device that is used at the transmission over networks data object, having comprised:

Transmission place on described network:

Interception unit, be used to tackle go on the recipient road separately through object; And

Recognition unit, be associated with described interception unit, be used to generate described digital network association through object, described interception unit is configured to replace described process object to transmit on described network with described digital network association, and further uses described digital network association to come the described object of mark in dictionary; And

Dictionary is used to store the described process object about described digital network association; And

Reception place on described network:

Search unit, be used for using described digital network association search to discern described corresponding object in the corresponding object of at least one dictionary, thereby described corresponding object is offered recipient separately from an approaching relatively dictionary, and need not carry out redundant Network Transmission; And

The correlating validation unit is associated with described search unit, is used to manage related information and prevents erroneous matching between the related and described data object of described digital network.

Preferably, described search unit is configured to provide:

A) local search,

B) if described local search is unsuccessful, so in nearest network node search, and

C), continue so in the search of the network node place of successive distances if described local search is unsuccessful.

Preferably, described digital network association comprises unique data identifier, only element identifier and validity timestamp.

Preferably, described recognition unit is configured to use hash function and local reference number, to generate described unique data identifier.

Preferably, described only element identifier is the predefined identifier that is set to represent transmitting element.

Preferably, described validity timestamp has defined the minimum time frame, and wherein the unique data identifier is associated with data object on transmitting element.

Preferably, described hash function is injection basically.

Preferably, when having provided the unique data identifier, described dictionary can extract data object.

Preferably, described dictionary is configured to use predefined algorithm, as " most recently used " (LRU), and with the replacement of management dictionary entry.

Preferably, the described only element sign of described correlating validation unit storage transmitting element, the local reference number of the object on transmitting element, and the described validity timestamp that is associated with described local unique data identifier, with the validity of definite association of being stored, and in order to prevent the erroneous matching of data identifier to data object.

Preferably, the copy through object is stored on a plurality of nodes of described network separately, and each copy all is associated with described unique identifier separately, thereby uses described digital network association separately to extract described copy from each node.

Preferably, determine the distance of node, and wherein said networking parameter comprises at least one in the group: physical distance, bandwidth, two-way time, stand-by period, hop count and economic cost with the networking parameter.

Preferably, described interception unit was configured to before buffer memory the process object fragments.This device further comprises the Load Balance Unit that is associated with described search unit, thereby described Load Balance Unit can extract the different segmentations of described object from the different nodes of described network, thereby the network utilization of relative equilibrium is provided.

This device can comprise the content control unit, be configured to store unique identifier relevant and the rule that is used for described control with the data object of wanting to control its distribution, thereby can use given unique identifier to search for described control module, to extract the respective rule that to use the further distribution of described object.

Preferably, described rule is to comprise any one of group that transmission blocking, object modification and object are replaced.

Preferably, described transmission place further comprises reception place recognition unit, be configured to by sending ICMP message to described recipient, be identified at a system unit in described reception place, and allow described system unit can discern and revise response, thereby described system unit is defined as parts in described reception place to described ICMP message.

Preferably, described reception place further comprises transmission place recognition unit, be configured to by sending ICMP message to described recipient, discern the system unit that approaches described recipient most, and allow response first system unit of process can discern and revise response to described ICMP message, thereby described system unit is defined as approaching most described recipient's system unit.

According to a second aspect of the invention, provide a kind of method, having comprised at the transmission over networks data object:

Transmission place on described network:

Interception go on the receiving end road separately through object;

Generate the digital network association of the segmentation of described process data,

Replace described process data sementation on described network, to transmit with described digital network is related, and

Use described digital network association to come the described object of mark in dictionary; And

Come the described data object section of mark in dictionary with described digital network association; And

Reception place on described network:

Use the corresponding object in described at least one dictionary of digital network association search, discern described corresponding object, thereby described corresponding object is offered recipient separately from an approaching relatively dictionary, and need not carry out redundant Network Transmission.

Preferably, described search begins to carry out in this locality, carry out at the node that approaches described expectation recipient then, and if still do not find corresponding object, so described search extension is arrived the farther reposefully node of distance.

This method is included in storage in the described dictionary with before described establishment digital network is related, becomes pre-sizing with described through object fragments.

Preferably, use the networking parameter to determine the distance of node, and wherein said networking parameter comprise in the following group one of at least: physical distance, bandwidth, two-way time, stand-by period, hop count and economic cost.

Preferably, on a plurality of nodes to described expectation recipient's different paths, carry out described search, thereby guaranteed relative network utilisation equilibrium.

According to a third aspect of the invention we, provide a kind of node of communication network, be configured to be minimized in object transfer redundant on the described network, this node comprises:

Interception unit, the network service that is used to tackle the carrying data object;

Dictionary is used to store described data object,

Indexing unit with described dictionary is associated is used for

A) generate the digital network association that is stored in each data object in the described dictionary, thereby for each described data object provides electronic signature, described electronic signature comprises the unique data sign in conjunction with local reference number;

B) first authentic copy of the described unique data sign of storage and described object association; And

C) in described network service, replace described data object with second copy of described unique data sign, continuing at described network communication, and

Extraction unit with described dictionary is associated is used for

D) receive inquiry from described network to the object that is identified with the digital network association,

E) signature of the storage in more described electronic signature and the described dictionary.

F) if found coupling, and effective from transmitting element to the checking of the data sementation of storage, extract object so corresponding to the signature of described coupling, and

G) if do not find coupling, send described inquiry to a neighbor node so.

According to a forth aspect of the invention, provide a kind of dictionary, be associated with the node of communication network, described dictionary is configured to be minimized in object transfer redundant on the described network, and this dictionary can be stored described data object, and further is associated in:

Interception unit is used to tackle network service;

Indexing unit is associated with described dictionary, is used for

A) generate the digital network association of each data object be stored in described dictionary, thus for each described data object provide data sementation electronic signature, transmit the identifier and the validity timestamp of network node;

B) first authentic copy of the described identifier that is associated with described object of storage; And

C) triplicate with described identifier replaces described data object in described network service, to continue at described network communication; And

Extraction unit is associated with described dictionary, is used for

D) from the inquiry of described network reception to the data object of usefulness digital network association identification,

E) signature of storing in more described electronic signature and the described dictionary,

F) if found coupling, and effective from transmitting element to the checking of the data object of storage, extract object so corresponding to the signature of described coupling, and

G) if do not find coupling, send described inquiry to a neighbor node so.

According to a fifth aspect of the invention, provide a kind of method, having comprised at the transmission over networks data object that comprises a plurality of nodes:

Pass through object arbitrarily in the location storage that is bordering on transmitting terminal that connects,

Generate the digital network association of described object;

The digital network association store that will be associated with described object is in dictionary;

Expectation recipient to described object sends described digital network association;

Be bordering on described expectation recipient's node connecting, use the digital network of described transmission related come with described dictionary in the identifier of objects stored compare, to find coupling;

If found coupling, and effective from transmitting element to the checking of objects stored, extract corresponding storage object so, and send to described expectation recipient;

If do not find coupling, repeat described coupling at further node so, up to finding coupling.

Preferably, use network parameter to determine the distance of node, network parameter comprises at least one in the group, and this group comprises physical distance, bandwidth, two-way time, stand-by period, hop count and economic cost.

According to a sixth aspect of the invention, provide a kind of device that on network remote identification approaches the system unit of given position most that is used for, this device comprises:

Message builder is used for generating for the discernible response request of system unit separately, and

Transmitting element, be used for sending described response request to described given position, thereby described response request is received by described given position, and send it back a response, described response is discernible for described system unit, thus first system unit that receives described response with himself to described device identification.

Preferably, described response request is an ICMP message, comprises first field with first numeral and second field with second numeral, this second numeral is the predefined function of described first numeral, and described predefined function is used to discern described response by described system unit.

According to a seventh aspect of the invention, provide a kind of on network remote identification approach the method for a given position most, this method comprises:

Generation is for the discernible response request of system unit separately,

Send described response request to described given position, thereby described response request is received by described given position, and send it back a response, described response is discernible for described system unit, thereby first system unit that receives described response identifies himself.

This method preferably includes: the method that generates described response request comprises with first numeral and is that second numeral of the predefined function of described first numeral is inserted in the described response request that described predefined function can allow described system unit discern described response.

According to a further aspect of the invention, a kind of device that is used to search and receive the immediate adjacent destination node in place is provided, wherein send the place and further comprise reception place recognition unit, be configured to by sending the system unit that ICMP message is discerned described reception place to described recipient, and allow the response of described ICMP message is discerned and revised by described system unit, thereby described system unit is defined as parts in described reception place.

According to another aspect again of the present invention, a kind of device that is used to search and send immediate reception place, place is provided, wherein said transmission place further comprises reception place recognition unit, be configured to by sending the system unit that ICMP message is discerned described reception place to described recipient, and allow the response of described ICMP message is discerned and revised by first system unit of described response process, thereby described system unit is defined as approaching most described recipient's system unit.

According to a further aspect of the invention, provide a kind of device that is used to search and receive the immediate adjacent destination node in place, this method comprises:

By sending the system unit that ICMP message is discerned described reception place to described recipient,

Identification is to the response of described ICMP message, and

Modification is to the response of described ICMP message, thereby described system unit is defined as parts at place, described reception place.

According to another aspect more of the present invention, a kind of method that is used to search and send immediate reception place, place is provided, this method comprises:

Discern the system unit that approaches described recipient most by sending ICMP message to described recipient, and

Permission is discerned and is revised by first system unit of described response process the response of described ICMP message, thereby described system unit is defined as approaching most described recipient's system unit.

Unless other definition is arranged, all technology used herein have the identical implication of being understood with those skilled in the art in the invention with scientific and technological word.Here the material that provides, method and example only are illustrative, and are not restrictive.

That the implementation of method and system of the present invention relates to is artificial, automatically or the execution of the two combination or finish the task or the step of some selection.And the actual device of the preferred embodiment of the method according to this invention and system and equipment can be by the softwares on any operating system of hardware or any firmware, and perhaps it is in conjunction with realizing several selected steps.For example, as hardware, selected step of the present invention may be implemented as chip or circuit.As software, selected step of the present invention may be implemented as a plurality of software instructions, and they use the operating system that is fit to arbitrarily to carry out by computing machine.Under any circumstance, the selected step of method and system of the present invention can be described to be carried out by data processor, such as the computing platform that is used to carry out a plurality of instructions.

Accompanying drawing is described

Here only present invention is described in conjunction with the accompanying drawings in the mode of example.Detailed certain figures mark for accompanying drawing, be stressed that the specific markers that illustrates is exemplary, and only be for the preferred embodiments of the present invention are described, and providing of they is to be the most useful of principle of the present invention and notion and the description of easy understanding in order to provide those to be believed.In this, attempt basic understanding more of the present invention is not required illustrates in greater detail structuring details of the present invention, makes for the description of accompanying drawing that those skilled in the art are very clear and how to realize several form of the present invention in practice.

In the accompanying drawings:

Fig. 1 is a reduced graph, shows the comparison between conventional buffer memory and the supercompression buffer memory, and they are all as prior art before the present invention;

Fig. 2 is a reduced graph, illustrates the data stream among the low bandwidth file system LBFS of prior art;

Fig. 3 is a reduced graph, illustrates the representative network that comprises a plurality of nodes, and the object transfer accelerating system according to the present invention on it is useful;

Fig. 4 is a reduced graph, illustrates tackling the process of object network node according to a preferred embodiment of the invention from the sender;

Fig. 5 is a reduced graph, illustrates the network node of Fig. 4, and interception is before being sent to the recipient specifically, need be by the digital network association of reconstruct;

Fig. 6 is a rough schematic view, and the network node object that illustrates Fig. 4 is divided into segmentation, to handle it by segmentation;

Fig. 7 is a simplified flow chart, illustrates use acceleration according to a preferred embodiment of the invention, sends and receive the program of network object;

Fig. 8 is a rough schematic view, and first computing machine that illustrates according to a preferred embodiment of the present invention begins to the second computing machine connection object;

Fig. 9 illustrate according to a preferred embodiment of the present invention in this locality or the program of sender node;

Figure 10 is a rough schematic view, illustrates the activity at far-end or receiving terminal node according to a preferred embodiment of the present invention;

Figure 11 is a rough schematic view, illustrates the example for the two-way object transfer of a plurality of network nodes according to a preferred embodiment of the present invention;

Figure 12 is a rough schematic view, illustrates object transfer as shown in figure 11, but has extra load balancing feature;

Figure 13 is a rough schematic view, illustrate according to a preferred embodiment of the present invention simply obtain Object Operations from network node;

Figure 14 is a rough schematic view, illustrates the architecture of network node according to a preferred embodiment of the present invention;

Figure 15 is a reduced graph, illustrate internal data flow according to a preferred embodiment of the present invention, and explain, at as home server shown in Figure 14 at intranodal with dictionary, initial object interception and storage, the operation of network node;

Figure 16 is a reduced graph, when illustrating network node as Figure 14 as far-end server, and the operation under the object reconstruction situation;

Figure 17 is a reduced graph, illustrate according to a preferred embodiment of the present invention losing the network infrastructure that may relate under the scene;

Figure 18 is a rough schematic view, illustrates according to a preferred embodiment of the present invention, and when object is not stored in the far-end network node of estimating the recipient place, but the program when needing the extra network node of search.

Figure 19 is a reduced graph, illustrates program according to a preferred embodiment of the present invention, has wherein occurred not hitting at the far-end network node, therefore obtains this object from nearer network node, weighs degree of closeness with cyberspeak;

Figure 20 is a reduced graph, illustrates situation according to a preferred embodiment of the present invention, has wherein occurred not hitting in far-end network node and all neighbor networks nodes, thereby has finally obtained this object from the network node in sender's place;

Figure 21 is a reduced graph, and the copy transmissions that illustrates prior art detects the data stream among the DTD;

Figure 22 is a reduced graph, illustrates contents controlling method; And

Figure 23 is a reduced graph, illustrates the detection method at the far-end network node that approaches the recipient;

Figure 24 is a reduced graph, illustrates at the far-end network node that approaches the recipient employed field in the ICMP stem of detection method;

Figure 25 is a simplified flow chart, illustrates the correlating validation program at correlating validation system place;

Figure 26 is a rough schematic view, illustrates preferred systems message;

Figure 27 is a simplified flow chart, illustrates the program of the object that is modified in transmission over networks;

Figure 28 is a rough schematic view, illustrates the object transfer of using central server in order to realize the load balancing characteristic.

The description of preferred embodiment

The present invention includes a network bi-directional data transfer optimization system, its node place on network is stored in the part of object or object in the dictionary, and uses unique identifier to search the object that is stored in such dictionary.

Word used herein " dictionary " refers to be buffered wherein storer at different nodes place object.At all nodes, same identifier always produces same result, searches just as the speech in the dictionary and always produces identical result.

Object by network is blocked and substitutes with unique identifier.Then, use this unique identifier to identify the nearest copy of this object at receiving end.

From an alternative viewpoint, provide a kind of method at the transmission over networks data object.This method comprises that interception is through object, the function that use is predetermined and the authorization information of sending entity come together to create the unique identifier of this object, use same function to come to provide identifier, and send this unique identifier and come the surrogate data method segmentation as objects stored in the network node at other node places of network.

Then, just can obtain this unique identifier, and search for corresponding objects in the local dictionary used as key at receiving end.Search from the immediate dictionary of this recipient, and expand outwardly stably.When finding this object, for recipient's interests send it, and by avoiding redundant transmission to save the network bandwidth, therefore should to as if from bringing with recipient nearest network node.

The purpose of this system is to have following ability:

● its minimizing redundant data transmission.

● it is by the data object that is applicable to all kinds of equality, wherein data object is the bit set that has the end of the beginning of definition and definition arbitrarily, include, but are not limited to any segmentation of any segmentation, record or record of any segmentation, webpage or webpage of any segmentation, subregion (sector) or subregion of any segmentation, stem or stem of any segmentation, message or message of any segmentation, grouping or the grouping of file or file, and their combination in any.

● it is included, but are not limited to by the data communication network environment that is applicable to all kinds of equality, wireless network, internet, satellite network, digital RF network, Cellular Networks and the cable system that comprises the digital content transmission.

● it preferably is applicable to all digital object transmission method, agreement and system, and no matter their transmission context.It preferably can delete redundant transmission, and no matter outside attribute, these attributes include, but are not limited to, and send agreement, transmitting medium, object name, object path, object factory and object reference.

● it preferably can quicken whole objects, and no matter order and mode that the segmentation of object is transmitted on network, and no matter mixing mutually with other objects and information during the transmission on the data network.

● it is the scheme that a kind of non-content is known.It should be able to transmit redundant deletion scheme, and does not allow the supplier be that the content that it quickens is born any legal liabilities.

● it is the intensive scheme of non-CPU preferably.

● it is preferably all useful for compressible and incompressible data.

● it preferably can minimize for the redundant professional transmission time.

● it preferably can utilize the network path that is different from the network path from the transmitting terminal to the receiving end.

● it can even reduce the transmission time in those objects are not present in more highly difficult example in the local storage.

● it preferably can distribute network path from the transmitting terminal to the receiving end and the offered load between other network paths.

● it not exclusively depends on the inside local storer of oneself.It preferably provides flow to quicken, though when the information that needs not in the middle of the local storage of node, but be arranged in the adjacent equipment even the equipment of far-end.

● it can carry out Web content control, and can not produce any privacy concern, and can in fact not become the scheme that a kind of content is known, just, it can not influence the content of being transmitted yet under the situation of external information that need not be extra.The external information that needs makes it possible to simply mate, and still can not identify real content.

● it can on-the-fly modify content according to configuration, for example, can cause area 3 codings in the dvd of Israel movie request, and can cause area 1 coding for the request from the U.S. of identical content.

● it can detect the far-end network node that is close in receiving end.

● it is not fragile for the mismatch between data identifier and the object.

Describe in conjunction with the accompanying drawings and accordingly, will be better appreciated by principle and operation according to data transmission of the present invention and control system.

Before at least one embodiment of the present invention is explained in detail, be appreciated that the invention is not restricted to be set forth in the following description or accompanying drawing shown in the application that concrete structure and parts are arranged.The present invention can comprise other embodiment, perhaps can realize in every way or implement.And, be appreciated that the wording and the term that adopt are for purpose of description here, and should be considered to restrictive.

Referring now to Fig. 3,, it shows the example networks that has in a large number the node 10.1...10n that is linked by connection.Sender 12 joins with a particular sections spot correlation of network one end, and other local particular sections spot correlations connection of recipient 14 and network.Some possible paths are arranged from sender to recipient, and the grouping that carries object from sender to recipient usually, sent by one or more such possible paths, with the bandwidth on all middle connections that are all in selected path.The grouping quilt is route independently, thereby in fact bandwidth is used up by several routes.Yet as previously mentioned, the many objects that are sent out on network are such as the image that is embedded in the popular webpage, by transmission again and again.Have some network caching solutions, but they are incomplete.

Referring now to Fig. 4,, it shows network node according to a preferred embodiment of the invention.This node is a part that is used in the whole network scope system of transmission over networks object.This system is designed to control the transmission of digital content, guarantees that the object that repeats to send on network is stored in the place, strategic location, and can be discerned easily by electronic signature is provided, to be extracted.The notice about this object to the recipient has been carried this signature, and can extract this object from the dictionary that approaches most this recipient, no matter where this object is from being sent out.Like this, this object only is being sent out on necessity distance by minimum on the network, wherein, proximity defines with network parameter, and these parameters include, but are not limited to physical distance, bandwidth, two-way time, stand-by period, hop count or economic cost or any other useful parameter.

Should be appreciated that this optimum system choosing ground works on during leaching process, if thereby this object is only found for recipient network node place far away relatively, it is by the network node in the middle of being stored in addition so, and total result is exactly: widely used object-be stored in many positions on the Internet such as being embedded into image in the particularly popular webpage.

In Fig. 4, node 20 comprises interception unit 22, its interception to recipient's separately road through object 23, and they are stored in the dictionary 24.What be associated with interception unit 22 is recognition unit 26, and it generates the digital network association 28 of this object.This digital network association is to be made of unique data identifier, only element identifier and validity timestamp.This unique data identifier is the function of all bits of this object preferably, and is such according to the definition of the hash function of mentioning in background, to reduce the probability of the mismatch between identifier and the object.In a preferred embodiment, recognition unit 26 is configured to use hash function to generate electronic signature, and it can use with local reference number, as the unique data identifier.Injection means that this function is man-to-man, that is to say, same output must be produced by same input.In fact the injection attribute is a kind of ideal situation, has very small probability, and specific output meeting is produced by the input more than.For a kind of effective hash function, such probability must be far smaller than 0.5%, and the term here such as injection or basic injection will be done and correspondingly explain and understand.

In order to create injective function, only element identifier and reference number are affixed to the output of hash function, and are that this particular network node is created unique identifier.On this particular network node, if hash function has generated an electronic signature that has existed, so by the represented object of this electronic signature by be stored in local dictionary in object of having before each of same signature compare, if do not find this object in existing object, it can be given a new unique reference number.If find that this object is identical with an existing object, it is given and finds it is the identical identical unique reference number of object.

Description that will be very detailed is to the requirement of such electronic signature below.Then that related 28 usefulness of digital network are for referencial use, come search content control system 29.If in content control system, found this digital network association 28, so this object is carried out the content control operation of configuration, wherein the content control operation includes, but are not limited to, and transmission blocking, object modification and object are replaced.Interception unit 22 usefulness digital network associations 28 are replaced through object 23, thereby this identifier has replaced this object, realize that it continues advancing on the Internet.Digital network association 28 also is used to the object 23 in the mark dictionary, in order to following extraction.

This node also preferably includes search and extracts (retrieval) unit 32, it can use such digital network association messages to extract the object that is stored in the dictionary 24, and comprising correlating validation unit 33, it can verify whether the object that is extracted is the definite copy that sends object in fact.

In use, object 23 is sent at network from sender 12, and arrives node 20.At node 20, this object 23 is stored in the dictionary 24.For this object generates digital network association messages 28.This digital network association also is used in this dictionary the index as this object.Now this digital network association messages is transmitted on the network in its stroke.

Referring now to Fig. 5,, it is the same with Fig. 4, except its explanation be processing at the digital network association messages of node 40, this node approaches the expectation recipient 14 of object 23.Identical among this node select ground and Fig. 4 is being identical at buffer memory aspect object at least, and the parts identical with Fig. 4 are given identical Reference numeral, unless necessary in order to understand the present invention, otherwise just no longer mention.

Digital network related 28 arrives node 40.Like this, replace the transmission forward of common grouping, the correlating validation unit 33 that is associated converts the unique data identifier to local data identifier, and this identifier is by the reference of search unit 32 as the search content control system.After having carried out all control operations that need, search unit 32 with for referencial use, is searched for dictionary 24 with this unique data identifier.If found the object 23 corresponding to this unique data identifier in dictionary, it is affixed to this message, and this message is sent to the expectation recipient forward.

If do not find object 23 in the dictionary of node 40, node sends inquiry so towards periphery, to attempt following the tracks of this object.If in of these nodes, found this object, so once more it is transmitted toward the recipient.Yet, preferably, on the dictionary of intermediate node, also known this object, as below will explaining in detail, thereby improved the availability of object on network.In this way, this system has guaranteed that widely used object is extensively obtainable on network.

Referring now to Fig. 6,, it is a reduced graph, illustrates according to the storage of present invention further optimization embodiment and transmits the process of a big object.As long as this is relatively little to liking, the work that said method just can be satisfied with, even because found this object at relative network node place far away with the recipient, it also can be sent to the recipient, and can exceedingly not increase the load that network connects.Under any circumstance, as long as find the network node of this object nearer, so just saved the network bandwidth thereon than sending node.If yet should be big to liking, even send this object from nearer node so, in this connection, also will use up appreciable bandwidth.And, send this object with a piece and mean for whole transmission and have to use a single path.In fact, object is packaged into a plurality of groupings, and to send each grouping with different routes be more effective.That is to say, preferably attempt parallel several different connections and the node of using.Like this, in Fig. 6, big relatively object 50 at mark and before inserting dictionary by segmentation.The preferably pre-sizing of segmentation 52, this size is selected for the unitarity of whole network.Each segmentation is provided to its identifier, and the unit that is used as separation is inserted in the dictionary.During operation, these segmentations become and are distributed on many network node 54.1...54.n.Then, the segmentation of these separation is searched for respectively and is extracted, and is stressed that, the heterogeneous networks node as much as possible that connects from the difference to recipient's node 56 extracts these different segmentations.Therefore, can realize the load balancing that network connects.

Referring now to Fig. 7,, it is a simplified flow chart, illustrates the method at the transmission over networks data object.This method comprises the phase one 58, detects the far-end network node that closes on receiving end.Subordinate phase 60 comprises that interception will be transferred to recipient's object by network from the sender.Then, the object that is blocked is passed to recognition unit in the stage 62, creates unique digital network association to use aforesaid predefined function.This function is the arbitrary function that is used for creating the well known function group of the electronic signature that combines with above-mentioned unique node identifier and reference number.Use same predefined function to come to provide identifier for the object of other nodes of being stored in network.

In the stage 63, the digital network association is used as with reference to being used for searching at content control system.If in content control system, found this digital network association, so this object is carried out the content control operation that configures.This content control operation includes, but are not limited to, and transmission blocking, object modification and object are replaced.

In the stage 64, this object is inserted in the dictionary.

In the stage 66, this digital network association is sent on its stroke on the network.

Then, at the receiving end of network, this digital network is associated in the extraction stage 67 and is used as the key of searching for corresponding object.It would be desirable and finding this object, but be not so in fact usually with the nearest node of recipient.Thereby, extract and expanded to farther node stably, up to finding this object.

In the stage 68, the digital network association is used as with reference to being used for searching at content control system.If in content control system, found this unique identifier, then this object is carried out the content control operation that configures.This content control operation includes, but are not limited to, and transmission blocking, object modification and object are replaced.

Be sent at this object of stages 70 then and estimate the recipient.

As mentioned above, can carry out extraction by never getting different segmentations, thereby guarantee the use of the relative equilibrium of network with the different nodes on the path to the stream that is divided into several objects.

More detailed now consideration the preferred embodiments of the present invention.As explained above, a kind of method that content control and the transmission of accelerator object word are provided is provided, and has reduced the bandwidth use simultaneously.This method comprises:

A. tackle digital object at local digital object dictionary place,

B. use the mathematical hashes function of the object tackled, create and comprise that the object checking is related with the digital network of identification information,

C. with the related reference of this digital network as the search content control system.If in content control system, found this unique identifier, so this object is carried out the content control operation that configures,

D. the object with interception is inserted in the dictionary,

E. use the sign of generation to come to extract the index of objects stored and send to network with being used for related this object that replaces of above-mentioned digital network from dictionary,

F. at the receiving end of network, this method continues to carry out by the match objects of using this digital network association to search in the far-end dictionary.

If g. do not find match objects,, further attempt extracting by obtaining this object from succession farther neighbor networks node.When the neighbor networks node obtains object, so as mentioned above, load-sharing mode can be activated.Load-sharing mode makes it possible to obtain from different network nodes simultaneously the segmentation of stream.If there is not nearer network node to hold this object, this object will be finally by the originating network node transmission from the sender so.

H. this method is by coming the search content control system to continue execution with for referencial use with this digital network is related.If in content control system, found this unique identifier, so this object is carried out the content control operation that configures,

I. corresponding object is sent to from the far-end dictionary estimates the recipient.

Definition

It below is non-limiting nomenclature to the term that uses among the application.

Digital object: the bit set of the ending of any beginning with definition and definition, but be not restricted to any segmentation of any segmentation, record or record of any segmentation, webpage or webpage of any segmentation, subregion or subregion of any segmentation, stem or stem of any segmentation, message or message of any segmentation, grouping or the grouping of file or file, and their combination in any.

Network node: software and/or the hardware of realizing this object transfer method.

Data communication network: the network of the transmission data of any kind, include, but are not limited to, wireless network, generally comprise the Internet the network based on TCP/IP, satellite network, digital RF network, cellular network, comprise the cable system that digital content transmits.

Far-end/this locality: the far-end network node more approaches the destination of digital object than local network node.Proximity includes, but are not limited to physical distance, bandwidth, two-way time, stand-by period, hop count or economic cost or any other useful parameter with networking network parameter-definition, networking network parameter.

The mathematical hashes function: one for all the feasible and effective injective function of content object, wherein each effective bit sequence has only a kind of hash to represent, includes, but are not limited to MD4, MD5, SHA1, SHA256, SHA384, SHA512.These are illustrated in and are also referred to as electronic signature and unique identifier herein.

Object id creates: uses digital object sign that the mathematical hashes function creates or from the procotol of transmitting this object, knows, or calculate by network node.

Object interception: use transparent or opaque method well known in the art, carry out during the transmission interception to digital object at a given node.

Load-sharing mode: at load-sharing mode, object fragments is obtained to obtain the identical mode of whole object.Load-sharing mode can obtain object fragments from different nodes simultaneously, to realize that ratio is only by being placed on whole load the network path utilization factor of the balance more that is obtained on the network path from sender to recipient.

The content control operation: the content control operation includes, but are not limited to, and transmission blocking, object modification and object are replaced.

Use the data transfer optimization of two-way object transfer

Referring now to Fig. 8,, it is a rough schematic view, illustrates first computing machine 70 and initiates the transmission of object 71 to second computing machine 72.

Mark

74 and 76 is represented two network nodes, and wherein 74 is nodes local or sender, and the 76th, recipient's node or distant-end node.

Quicken transmission course as follows:

A. first computing machine 70 is initiated to the object transfer of second computing machine 72.

B. this digital object transmission of local node 74 interceptions.For this object 71 generates digital network association 78.This digital network association comprises unique data identifier, only element identifier and proving time stamp.Construct this unique data identifier with local reference number with the electronic signature that the output of the mathematical hashes function of for example SHA512 is calculated.

C. with of the reference of this unique data identifier, this object is carried out the content control operation as the search content control system.

D. use this unique data identifier with the interception object storage in local dictionary.

E. local node 74 sends the digital network association now, replaces object itself.

F. distant-end node 76 is tackled the message transfer of revising.This digital network association is used to search for corresponding local unique data identifier.

G. distant-end node 76 should this locality unique data identifier as the reference of search content control system, this object is carried out content control operation.

H. distant-end node 76 uses these this locality unique data identifiers to search for and verify object in its dictionary.

I. this object is sent to its original purpose ground then.

Referring now to Fig. 9,, it illustrates in this locality or the program of sender node 74.The operation of local dictionary is as follows:

A. the transmission of digital object is blocked.For this object 71 generates digital network association 78.This digital network association comprises unique data identifier, only element identifier and proving time stamp.Construct this unique data identifier with local reference number with the electronic signature that the output of mathematical hashes function is calculated.

If B. this object is greater than predetermined size, this object is divided into segmentation so.Generate the digital network association respectively for each segmentation then.The sign of object is done as a whole all connections of the sign of segmentation separately that just become.

D. subsequent stage comprises digital object is stored in the local dictionary, has the unique data identifier as key.

E. the related object self that replaces of digital network is sent out.

Referring now to Figure 10,, it is a rough schematic view, illustrates the operation at far-end or receiving terminal node 76.

Distant-end node is carried out following operation:

A. the message transfer that was modified at local node 74 is blocked.The digital network association is used to search for corresponding local unique data identifier.This digital network association comprises unique data identifier, only element identifier and proving time stamp.The unique data identifier is with local reference number and use the electronic signature of the output calculating of mathematical hashes function to construct.

B. with of the reference of this unique data identifier, this object is carried out the content control operation as the search content control system.

C. at far-end dictionary 76, use this this locality unique data identifier to search for and verify digital object 71.

If D. found object 71, this object is sent to the destination that it is estimated subsequently so.

Referring now to Figure 11,, it is a rough schematic view, illustrates the situation of the object transfer with a plurality of network nodes.

Computing machine 70 is initiated to the object transfer of passing through network of computing machine 72.This network comprises a plurality of network node S1, S2...S10.

Come the optimization objects transmission as follows:

A. computing machine 70 is initiated to the object transfer of computing machine 72.It sends digital object as before.

B. local network node (S1) is tackled this digital object transmission.The object of interception is stored in the local dictionary, and is this Object Creation digital network association 78.

C. with of the reference of this unique identifier, this object is carried out the content control operation as the search content control system.

D. then, local network node (S1) sends digital network association 78, replaces object itself.

E. far-end network node (S6) interception digital network association.

F. the far-end network node uses this digital network association to verify and search for corresponding local unique data identifier.

If G. in the distant-end node dictionary, do not find this digital object, so this distant-end node attempt within the object transfer path and outside any other digital nodes extract this object.Be discussed below the various selections that realize such extraction algorithm.Distant-end node generates the request of network associate message, and it preferably includes the validity timestamp that provides in the unique association identifier that provides in the digital network association, the digital network association and its unit id.

If H. in these network nodes has found this object, by using, send it to the network node (S6) of request so to the replying of network associate request message.To replying of network associate request message preferably include the original unique association identifier that provides in the network associate message request, at the unique association identifier of the object of being asked on the response unit, validity timestamp and this data object itself on the response unit.Response unit uses the validity timestamp that provides in the network associate message request to upgrade its correlating validation database.

I. request object node (S6) upgrades its correlating validation database with original associated identifiers, validity timestamp and response unit associated identifiers and validity timestamp.

J. should this locality unique identifier as the reference of search content control system, this object is carried out content control operation.

K. the output of content control system is addressed to then and estimates the recipient.

Consider the problems referred to above in further detail, the matter of utmost importance of appearance is about when the optimal strategy that is used to extract object part when nearest network node does not find object part.Many possible strategies are arranged, and they have recommended the method that addresses this problem to the technician.A preferred strategy is that query messages is issued all neighbor nodes.From then on inquiry determines which node comprises which part of object.After having determined which part which node has, send request with all request mode of balance between different nodes to specific node then to each unitary part.If thereby find five different nodes to comprise this object, they all are positioned at same distance but on different paths, ask so by the distribution of equality, and each node is required five/part.If two in five nodes in fact on same paths, and system's success judge this point, these two nodes can be used as a separate nodes and treat so, and comprise these binodal four nodes each all be required four/part, the rest may be inferred.

Another kind of selection can be the request that sequentially sends for different piece.

The another kind of selection is that request with for the different segmentations of object sends to different neighbor nodes.

Another kind of select to be to use safeguard overall validation database and can considered that load balancing needs and the situation of other network parameters under request is directed to the server node of the neighbor networks node of the object that comprises needs.

Preferred implementation comprises some of above-mentioned strategy or whole combinations.In a kind of specific preferred embodiment, be used to determine the effective choice of neighbor node in the obtainable knowledge in far-end network node place of the geographic distribution of neighbor node.

Referring now to Figure 12,, it is a rough schematic view, illustrates object transfer as shown in figure 11, but has the supplementary features of load balancing.As mentioned above, load balancing is used to guarantee to obtain from different network nodes the different segmentations of the object of wanting.

As before, computing machine 70 is initiated to the object transfer of passing through network of computing machine 72.Along this network is network node S1, S2...S10.

Optimize data transmission procedure as follows:

A. computing machine 70 uses a procotol to be initiated to the object transfer of computing machine 72.

B. local digital node (S1) is tackled this digital object transmission.As mentioned above, object is divided into segmentation virtually.For each segmentation, calculate the digital network association.Use aforesaid unique identifier to come object fragments is carried out the content control operation.

C. local network node (S1) sends digital network association 78 then, replaces object itself.

D. far-end network node (S6) interception digital network association.

E. far-end network node (S6) uses this object identifier to verify and this object of search in its dictionary.

If F. do not find this digital object in distant-end node S6, this distant-end node is according to predetermined load balancing configuration so, attempt within the object transfer path and outside any other digital nodes extract this object.The far-end network node is attempted extracting by from the different object fragments of different object server requests.

Be noted that this system does not need to know specific route,, thereby need to use different routes because this system's utilization is deployed in the different nodes at the diverse location place on the network from the beginning.Yet, in the time of if desired, can enable routing iinformation configuration (artificial or automatically).

G. as previously mentioned, different object fragments are sent to request object node (S6).The request object node object that reconstruct is asked from these segmentations.

H. request object node (S6) is with the original message of this object reconstruction.

I. should this locality unique identifier as the reference of search content control system, this object is carried out content control operation.

J. the output of content control system is addressed to the expectation recipient then.

Referring now to Figure 28,, it is a rough schematic view, illustrates object transfer as shown in figure 11, but has the supplementary features that realize load balancing with central validation service device CS.As mentioned above, load balancing is used to guarantee to extract from different network nodes the different segmentations of the object of wanting.

As before, computing machine 70 is initiated to the object transfer of passing through network of computing machine 72.Along this network is network node S1...S5 and central validation service device S10.

Optimize data transmission procedure as follows:

F. computing machine 70 uses a procotol to be initiated to the object transfer of computing machine 72.

G. local digital node (S1) is tackled this digital object transmission.As mentioned above, object is divided into segmentation.For each segmentation, calculate the digital network association.Use aforesaid unique identifier to come object fragments is carried out the content control operation.

H. local network node (S1) sends digital network association 78 then, replaces object itself.

I. far-end network node (S6) interception digital network association.

J. far-end network node (S6) uses this object identifier to verify and this object of search in its dictionary.

If E. do not find this digital object in distant-end node S6, this distant-end node is to the inquiry of central validation service device (CS) transmission to network associate (QNA) message, to determine to provide the optimal network node of required object so.

F. central validation service device (CS) is made with network associate guiding (NAL) message and being replied.This network associate guiding message points out that to distant-end node network node (S5) is the optimal network node that can provide required object this moment.

G. distant-end node guides the node that provides the message to extract object from network associate.

J. then, the output of content control system is addressed to and estimates the recipient.

Consider the problems referred to above in further detail, the matter of utmost importance of appearance is the optimal strategy that is used to upgrade the central server validation database.Many possible strategies are arranged, and they have recommended the method that addresses this problem to the technician.A kind of preferred strategy is a new database more when the inquiry of network associate message is arrived.Can upgrade this database with the authorization information on the local node.

Another kind of selection is the neighbor node transmission inquiry to a tabulation.Determine that from this inquiry which node comprises this object.

The another kind of selection is to periodically update central server.

Preferred implementation has comprised the part or all of combination of above-mentioned strategy.

Referring now to Figure 13,, it illustrates a kind of simple object acquisition operation.The far-end network node is needed as another network node and obtains object, and the latter's node is the node that need to approach the recipient of this object.This node receives the request of network associate 78, the operation below carrying out then:

A. the digital network association is passed to the correlating validation system, and is used to the corresponding local unique data identifier of search in validation database.

If B. found object, from local dictionary 76, obtain it.

C. then, in the replying of network associate request message this object is being sent to the request network node.

Be noted that according to this architecture object does not exist can be with showing that the message that this object does not find indicates.Substitute, can use a kind of timeout feature, thereby if before extracting response this request timed out, so just suppose that this object does not exist.Referring now to Figure 14,, it is a rough schematic view, illustrates the architecture of network node according to a preferred embodiment of the invention.

Network node 90 comprises with lower module:

Tou Mingdaili ﹠amp; Protocol analyzer 92, it can carry out message block, object extraction and reconstruct.

Dictionary 94.This dictionary comprises the mapping between the data object of storing in unique data identifier and its.Come storage object with hash table.Be input to hashed value in the dictionary from unique data identifier self.And dictionary preferably keeps most recently used (LRU) dictionary list item, to carry out effective refuse collection, in other words empties redundant object exactly from dictionary.Each object is preferably divided to be equipped with a timestamp, and it has defined a minimum time frame, and the dictionary list item can not be deleted by Garbage Collection Mechanism in this time.In case this time frame finishes, this list item just may be deleted by Garbage Collection Mechanism.

Unique identifier system 96.As mentioned above, unique identifier system comprises and can be one or more functions of Object Creation unique identifier or given group of functions.

Storer 98 is used for the storage system of the stored digital object of physical store.

Common store connection protocol (USCP) 100, it is the protocol element that can make digital object node mutual communication.That is to say, for example the object requests between its network enabled node.

Content control system 95.Content control system be included in the unique data identifier and the operation that need carry out related object between mapping.The content control operation includes, but are not limited to, and object transfer obstruction, object modification and object are replaced.

The correlating validation system is configured to use the digital network association messages that is blocked, and so that local unique data identifier to be provided, this identifier is verified with the original related object of object, and prevented wrong object coupling.The correlating validation system is further configured to safeguarding a correlating validation database, and it manages all current far-end associations of knowing.

Referring now to Figure 15,, it is a reduced graph, show the internal data flow in the node of realizing the object transfer method according to a preferred embodiment of the invention, and explained that the network node shown in Figure 14 is for initial object interception be labeled as the operation of home server.Identical part is given identical Reference numeral among the figure in front, and is necessary except for understanding present embodiment, otherwise no longer mentions.The optimized operation of object with the viewpoint of data stream is as follows:

A. the transmission of the digital object on the network is tackled by Transparent Proxy 102.

B. digital object is extracted by protocol analyzer 104.

C. according to the object that is extracted, the unique data of unique identification system 96 calculating objects identifies.This step divides following two stages to carry out.Object preferably is divided into virtual segmentation, and wherein each segmentation can be regarded as an object.For each segmentation, calculate an independent unique data identifier.

D. according to the unique data identifier, content control system 95 is carried out suitable object control operation.

E. then, digital object is added in the local dictionary 98.Object validity timestamp is updated, thereby has prevented before far-end server can be asked this object, and this object is just deleted.That is to say, can guarantee that when far-end server began to seek this object, it was present in the somewhere on the network.

F. from the unique data identifier, object validity timestamp and the only element identifier that calculate, construct a digital network association messages.This digital network association messages replaces object self now, is sent by USCP module 100.

Referring now to Figure 16,, it illustrates the operation of the network node under the object reconstruction situation of Figure 14, and at this moment distant-end node is as far-end server.Identical part is given identical Reference numeral among the figure in front, and no longer mentions, and is necessary except for understanding present embodiment.

The operation of object reconstruction program is as follows:

A. the digital network association is preferably tackled by USCP module 100, and comprises three parts, unique data identifier, object validity timestamp and only element identifier.

B. correlating validation system 91 uses this digital network association to search for corresponding local unique data identifier.

C. based on local unique data identifier, content control system 95 is carried out suitable object control operation.

D. according to local unique data identifier, attempt from dictionary 94, obtaining digital object.

If E. found object, Transparent Proxy 102 sends to this digital object on its original purpose ground.

Referring now to Figure 17,, it illustrates related network infrastructure in a scene, does not have object can be associated with the digital network association messages at the far-end network node in this scene.

General introduction-miss occasion in dictionary

If the network node S6 of receiving end do not comprise can be related with the digital network association messages object, network node S6 searches the nearest server that comprises really corresponding to the object of this digital network association messages so.Network node S6 comprises the tabulation of other network nodes, and this tabulation is sorted with measuring of a kind of parameter Network Based.This network node tabulation also comprises the transmission network node for each object transfer, that is to say, this network node tabulation comprises the originating network node, and it is a network node that must comprise requested object.This starts or sends preferably one section sufficiently long minimum time frame of copy of object of reservation of network node, is long enough to make the far-end network node can obtain this object data.This way has guaranteed that at least one network node in the somewhere on the network holds the object of being asked, and is satisfied up to this request.At this moment, the network node of far-end network node in its tabulation sends the request of network associate message, and extracts this object from nearest (with the angle of tolerance) network node of holding this object.In the worst situation, have only a network node to hold this object, just the originating network node is not saved transmission bandwidth in this case.

The method that this search of several preferred realizations is arranged.

First kind of selection is to have as defined above, a predefined adjacent tabulation.After all it fails to match for all nodes in the tabulation, on initial the connection, pellucidly the network associate message request is sent it back the initial network node.

The another kind of selection is that beginning just sends the network associate message request on initial the connection.Each network node on the network associate message request path all attempts satisfying this request.Then, when coupling having occurred, actual object or segmentation are addressed to requesting node, have replaced the network associate message request is sent it back origination node along this path.

Preferred implementation comprises the combination of above-mentioned selection.

Referring now to Figure 18,, it is a rough schematic view according to a preferred embodiment of the invention, illustrates the far-end that is not stored in expectation recipient place when object Network nodeWhen middle, need search other Network nodeProgram.

As before, first computing machine 70 is initiated to the object transfer of second computing machine 72.S1 and S2 are two network nodes, and wherein S1 is local or sender node, S2 be far-end or receiving terminal node.

The object transfer optimizing process is as follows:

A. message transmits:

● as previously mentioned, first computing machine 70 is initiated to the object transfer of second computing machine 72.

● local network node (S1) is tackled this digital object transmission.Calculating object sign, i.e. unique data identifier.With of the reference of this unique data identifier, this object is carried out the content control operation as the search content control system.Some object can be blocked by content control or restriction in addition.If object transfer does not get clogged, use object storage that this unique data sign will interception in local dictionary.

● local network node (S1) sends the digital network association messages now, replaces sending primary object.Simultaneously, object is stored in S1, and the unique data identifier is used as the index that extracts this object from the dictionary of this node.

● remote digital node (S2) is tackled this digital network association messages.This digital network association is used to search for corresponding local unique data identifier.

B. in this example, in far-end network node Sn, do not find corresponding local unique data identifier.Thereby far-end network node S2 sends to local network node S1 has the network associate message request of miss object association identifier.

C. local network node sends with primary object replying the network associate request message now.This validation database is updated, and as required this object is carried out the content control operation.This far-end network node sends to this object its original purpose ground this moment.

What be noted that top example relates to is the situation of having only the simplification of two network nodes.Have more node as a rule.

Referring now to Figure 19,, it is a reduced graph, illustrate program according to a preferred embodiment of the invention, wherein, object is not stored in the far-end network node of estimating the recipient place, but needs the other network node of search, therefore, obtain this object from a nearer network node, proximity is weighed with cyberspeak.

At first, as before, first computing machine 70 is initiated to the object transfer of second computing machine 72.Network comprises intermediate network node S1...Sn, and wherein, S1 is the network node of sender this locality, and Sn is the far-end network node from sender's angle, still, and for estimating that the recipient is local.

The object transfer optimizing process is as follows:

A. message transmits:

● local network node (S1) is tackled this digital object transmission.For example using, the output and the local reference number of the mathematical hashes function of SHA512 come calculating object sign, i.e. unique data identifier.With of the reference of this unique data identifier, if necessary, this object is carried out the content control operation as the search content control system.If object transfer does not get clogged and is not limited in addition yet, use object storage that this unique data sign will interception so in local dictionary.

● local network node (S1) sends the digital network association messages now, replaces in sending primary object self.

● remote digital node (Sn) is tackled this digital network association messages.This digital network association is used to search for corresponding local unique data identifier.

B. in this example, in far-end network node Sn, do not find corresponding local unique data identifier.Thereby far-end network node Sn sends to local network node S1 has the network associate message request of miss object association identifier.Message is transmitted, and comprises the network node of this object up to arrival.The selection of listing above that is used to pass on the network associate message request and obtain object can be applied in here.

C. be found the network node that finally comprises this object, this moment is with replying to the request network node of network associate request message sent this object.Preferably, this object also is stored in intermediate node, thereby has improved the availability of this object on network.In this manner, system has guaranteed that widely used object is extensively obtainable on network.Then, the request network node upgrades its dictionary and checking, also this object is forwarded to next request network node, finally arrives the far-end network node up to this object.The far-end network node comes the reconstruct original data stream with this object at this moment.The data stream of reconstruct is sent to the expectation recipient on original purpose ground then.

Referring now to Figure 20,, it is a reduced graph, illustrates situation according to a preferred embodiment of the invention, and wherein, object is not stored in any far-end network node.

The same with the front, first computing machine 70 is initiated to the object transfer of second computing machine 72.Network node S1...Sn represents network node, and wherein S1 is a local network node, and Sn is the far-end network node.

Object transfer Optimizing operation under this scene is as follows:

A. message transmits:

● as previously mentioned, first computing machine 70 is initiated to the object transfer of PC2.

● local network node (S1) is tackled this digital object transmission fully as before.For example using, the output and the local reference number of the mathematical hashes function of SHA512 come calculating object sign, i.e. unique data identifier.With of the reference of this unique data identifier, this object is carried out the content control operation as the search content control system.If object transfer does not get clogged or is limited in addition, use object storage that this unique data sign will interception so in local dictionary.

● local network node (S1) sends the digital network association messages now, equally as before, replaces in sending primary object self.

B. in this example, in far-end network node Sn, do not find corresponding local unique data identifier.Thereby the far-end network node is at this moment to the neighbor networks node, and transmission has the network associate message request of miss object association identifier, and these neighbor nodes use it to check whether they have this object.Their also this objects not in this example, and message continues to be transmitted around the node of network have been located a network node that comprises this object up to final.In this case, the final local network node that starts that arrives of this request is because have only this network node to have this object.

C. local network node receives the network associate message request, and in response, with to the replying of network associate request message, sends this object to the request network node.The network node of request upgrades oneself now, and continues this object is forwarded to next request network node, finally arrives the far-end network node up to this object.This far-end network node is with this object reconstruction original data stream, and, then the data stream of reconstruct is sent to its original purpose ground.

Content control system

Referring now to Figure 22,, it is a reduced graph, illustrates the contents controlling method for the use of originating network node according to a preferred embodiment of the invention.Control system 29 comprises the database 144 of holding the table with two fields.In first field various unique data identifiers, and, in second field corresponding restriction and rule.The content control system operation is as follows:

A. receive a transmission.

B. digital object is blocked.

C. as mentioned above, use mathematical hashes function and local reference number to calculate the unique data sign.

D. according to rule list,, object is carried out the content control operation with of the reference of unique data identifier as the search content control system.The content control operation includes, but are not limited to, transmission blocking, object modification and object are replaced, and the content control rule includes, but are not limited to, purpose coupling, source coupling, marking matched, transmission match time, repeated matching number, any other firewall class coupling or any combination of these match-on criterions.

E. the result of content control operation is transferred to the destination of expectation then.

Content control system may be implemented as independently server or network components.

The technician will be understood that how this contents controlling method is applied to receiving end, and unique difference is that the data transmission that receives is a unique identifier, rather than origin object.

Be used to detect the method for the far-end network node that closes on receiving end

Referring now to Figure 23,, it is a rough schematic view, illustrates program according to a preferred embodiment of the invention, is used to detect the far-end network node that approaches the recipient.That is to say that the network node of present embodiment is estimated to be located in or to be associated with the node that strategy is gone up the network of selecting, but it can not be considered to all nodes that they must be located in network.Even they are positioned at all nodes of network, be necessary still to determine which node is the node that approaches the recipient most, and the system description of describing in conjunction with Figure 23 a kind of method of determining to approach most recipient's node.With the immediate node select of recipient ground be the node that starts the search of the object of needs, therefore, that can affirm determines its needs.

This process operation is as follows:

A. this locality or originating network node (S1) are also referred to as examination logical (ping) request to estimating that the recipient sends the ICMP request message.This ICMP request comprises two fields, an identifier field and a sequence-number field.System preferably above-mentioned two fields is set, so that can identify ICMP message that this is not a routine but system message in the mode that describes below.ICMP identifier and sequence-number field preferably include a random number x, and it preferably has predefined bit number, and second portion, and it comprises the result of function f (x), the number of selecting immediately above this function is applied to, for example XOR.

B. random number x is preferably stored with the related ground of the source and destination of the connection that will quicken.The source and destination of ICMP request preferably are configured to the source and destination of the connection that will quicken.

C. as the result who receives the ICMP request, far-end recipient (PC2) preferably provides an ICMP response, and this response has identical ICMP identifier and sequence-number field.

D. each node select ground receives this ICMP response, and can be a system message by the application f of the random number in the ICMP identifier (x) is identified it.If the result who obtains is consistent with sequence-number field, this message obviously is a system message so.X and f (x) are stored in the field of ICMP identifier.This two number can be placed in the different fields or be restricted to predetermined bit number, and section that can predefine ICMP is stored x and f (x) respectively.

If E. the result of f (x) equal to store in the ICMP sequence-number field or canned data by any way, as mentioned above, this message is regarded as system message so, and we can say that this node is confirmed as and the immediate distant-end node of receiving end, and it stores source and destination IP.It is determined like this because this response is only handed on by regular node, rather than is transmitted by the system node that it is defined as system message.Source and destination IP preferably be used to have this given IP address right, the far-end network node is appointed as and the immediate network node of receiving end.

F. be confirmed as like this and the immediate distant-end node of receiving end, use another function g (x) to change ICMP response identifier and sequence-number field or other memory location then, this function can be applied to random number, for example XOR+1.Modified ICMP response identifier and sequence-number field or other positions comprise the result of the initial random number and second function now.What this modified response was used as ICMP message now continues to send it back initial or local network node.

G. local node is tackled this ICMP response, thereby uses the value of this ICMP identifier and sequence-number field to find out whether there is the acceleration node on the path.This ICMP response can be dropped then.

A kind of optional implementation is to send 4 layer network information, for example agreement, port numbers or the like in the ICMP payload.Can dispose such implementation as required.

Referring now to Figure 25,, it is a simplified flow chart, illustrates the correlating validation program of locating in correlating validation system (AAS) according to further preferred embodiment of the present invention.This correlating validation system is configured to use the digital network association messages of interception, so that an empirical tests local unique data identifier and object coupling that prevents mistake corresponding to the original object that is associated is provided.This correlating validation system further is configured to safeguard a correlating validation database 180, current all the far-end associations known of this data base administration.Clauses and subclauses in the correlating validation database preferably include the only element identifier of electronic signature, far-end unit, unique local reference number, validity timestamp and the local reference number of far-end unit.This process comprises:

A. search for all data base entries, with the same in the digital network of interception is related, the same electronic signature 155 of they indications.

B. from the item list that finds at step a, search for all data base entries, with the same in the digital network of interception is related, the same only element identifier 154 of they indications.

C. search database clauses and subclauses from the item list that finds at step b, with the same in the digital network of interception is related, they indicate same local reference number 156.

If d. find such clauses and subclauses, and the validity timestamp is effectively, uses the combination of local reference number of storing in data base entries and the electronic signature that provides so, generates local unique data identifier.

If e. find such clauses and subclauses, and the validity timestamp is effectively, so according to the new validity timestamp 152 that provides in the digital network association messages, comes the validity timestamp in the new database entry more.

F. otherwise, to not existing corresponding to the association of the object signaling empirical tests of this digital network association messages.

Referring now to Figure 26,, it is a reduced graph, illustrates the preferred systems message of using in the data transfer optimization process.Preferably, digital network association messages 150 comprises unique association identifier 151 and validity timestamp 152.Preferably, unique association identifier comprises unique data identifier 153 and only element identifier 154.Preferably, the unique data identifier comprises the electronic signature and the local reference number 156 of data sementation 155.Preferably, only element identifier 154 is the predefined identifiers that are set to represent to send network node.Preferably, validity timestamp 152 has defined a minimum time frame, and wherein the unique data identifier is associated with data sementation on the transmitting element.Preferably, network associate message request 160 comprises unique association identifier 151, validity timestamp 152 and request unit id 161.Preferably, unique association identifier and validity timestamp are to obtain from the digital network association messages of trigger network association request.Preferably, the network associate request message is replied 170 and is comprised the related id of original related id, response unit, the validity timestamp of response unit and the data object that is associated.Preferably, original related id obtains from the network associate message request.

Referring now to Figure 27,, it is a reduced graph, illustrates the further preferred embodiment according to the present invention, in the process of the modified object of transmission over networks.If to as if with its original form in transmission over networks, it preferably is segmented into littler object, and is transmitted according to aforesaid method.In this stage, the object fragments sequence has been known in system, for example dna1, dna2, dna3, or the like.When a modified object is sent to network, the segmentation of this modification has produced and has been different from the desired DNA of system.System can detect modified segmentation, and attempts adding that with original data sementation change represents modified segmentation.If the difference between the data of raw data and modification is relative less, original DNA is coupled with the difference of data and is sent out, and system continues expectation and receives original fragment sequence.Otherwise this system sends the new DNA that creates, and the input data in future are considered as new data sequence.

Use above-mentioned technology may improve the efficient of connection object, these objects in time and partly are modified.Each segmentation is independently handled, thus the position that unmodified segmentation can be placed before them be assembled, and the new segmentation of revising assembled from the position that is placed recently, reduced the universe network load once more.

The embodiment here provides some with respect to the following at least advantage of prior art systems:

● they have removed redundant data transmission on the network substantially.

● they have reduced the overall bandwidth loss of network.

● they have reduced the overall server load of network.

● they have reduced the universe network stand-by period.

● they provide minimum, are almost for 0 redundant data transmissions time.

● these embodiment are applicable to all types and the size of data object.

● these embodiment are applicable to data communication network environment arbitrarily.

● these embodiment are applicable to all digital object transmission, no matter their transmission context.

● these embodiment can remove redundant transmission, no matter external attribute, for example HTTP URL, file path, URL mime type, filename or the like.

● these embodiment can quicken whole object, no matter segmentation order and the mode on network, transmitted, also no matter during transmission on the data network with the mixing of other objects and information.

● the solution that these embodiment provide a kind of non-content to know.Can not be from identifier this content of reconstruct.These embodiment can transmit redundant cleaning scheme, and the network provider are not exposed under any legal liabilities of bearing for the content of quickening.

● it is the intensive scheme of a kind of non-CPU.Generating and comparing identifier and transmit searching request needs cpu power, but this lacks a lot than the power of being saved in having less redundant business.

● these embodiment are same useful to compressible and incompressible data, and for data be actually compression or be not without any difference.

● these embodiment can utilize the network path different with the network path from sender to recipient.

● these embodiment can reduce the transmission time, even under object is not present in situation in the dictionary of purpose object network node, also are like this as long as this object is present in the middle buffer memory that more approaches the purpose buffer memory than source network node.

● these embodiment can distribute network path from the transmitting terminal to the receiving end and the offered load between other network paths smoothly.

● these embodiment can carry out Web content control, and can not produce any privacy concern, and can in fact not become the scheme that a kind of content is known, just it can not influence the content of being transmitted, external information that need not be extra yet.The external information that needs makes it possible to simply mate, and still can not discern real content.

● these embodiment can on-the-fly modify content according to configuration, for example, can cause area 3 codings in the dvd of Israel movie request, and can cause area 1 coding for the request from the U.S. of identical content.

● these embodiment can detect the far-end network node that is close in receiving end.

Hope will be developed many relevant data transmission, network and buffer memory equipment and system in the lifetime of this patent, and the scope of the corresponding word here to be hoped be to comprise all such priori new technologies.

Should be appreciated that, for clear and in the embodiment context that separates, some feature of the present invention of description, can also be in an independent embodiment combined providing.Opposite, for simplicity, various features that describe in independent embodiment context, of the present invention can also be provided respectively or be provided in the branch combination mode that is fit to arbitrarily.

Invention has been described although combine certain embodiments, and clearly, those skilled in the art are appreciated that many displacements, modification and modification.Accordingly, expectation comprises all such displacements, modification and modification, and they fall in the spirit and broad scope of additional claim.All publications, patent and the patented claim of mentioning in this manual all is attached in this instructions by reference fully, and its degree is equal to as each independent publication, patent or patented claim and will be included in here by reference by special and independent pointing out.In addition, quoting or identifying of any reference among the application should not be construed as and admit that such reference is obtainable as prior art of the present invention.

Claims

1. device that is used at the transmission over networks data object comprises:

Place, transmission place on described network:

Interception unit, be used to tackle and buffer memory go on the recipient road separately through object; And

Recognition unit, be associated with described interception unit, be used to generate described unique identifier through object, described interception unit be configured to described unique identifier replace described through object on described network, transmitting, and further use described unique identifier to come the described object of mark in described buffer memory; And

Place, reception place on described network:

Search unit, the corresponding object that is used for using described unique identifier to search at least one node dictionary is discerned described corresponding object, thus described corresponding object is offered recipient separately from an approaching relatively node dictionary, and need not carry out redundant Network Transmission.

2. according to the device of claim 1, wherein said search unit is configured to provide:

A) local search,

C), continue network node search so in successive distances if described local search is unsuccessful.

3. according to the device of claim 2, wherein said search unit further is configured to revise described search according to the local load balancing Consideration.

4. according to the device of claim 3, wherein said load balancing Consideration obtains from central load balancing Control Node.

5. according to the device of claim 1, wherein said object by segmentation, makes that each segmentation all can be by separate searches and can be by independent transmission before storage.

6. according to the device of claim 1, wherein when revising the object of described segmentation, each segmentation is handled independently.

7. according to the device of claim 5, further be configured to comprise with the variation of segmentation increment as original segmentation.

8. according to the device of claim 1, wherein said recognition function is injection basically.

9. device according to Claim 8, wherein said recognition unit are configured to the recognition function of hash function as described basic injection.

10. according to the device of claim 1, wherein said recognition unit is configured to use the recognition function of complete injection.

11. device according to claim 1, wherein be stored in a plurality of nodes of described network separately through the copy of object, each copy is associated with described unique identifier separately, thereby uses described unique identifier separately to extract described copy from each node.

12. device according to claim 2, wherein the distance of node is determined by the networking parameter, and wherein said networking parameter comprises at least one in one group, and this group comprises: physical distance, bandwidth, two-way time, stand-by period, hop count and economic cost.

13. device according to claim 11, wherein said interception unit is configured to interception process object before buffer memory, this device further comprises the Load Balance Unit that is associated with described search unit, make described Load Balance Unit can extract the different segmentations of described object, the network utilization of relative equilibrium is provided thus from the different nodes of described network.

14. device according to claim 1, further comprise the content control unit, be configured to store unique identifier relevant and the rule that is used for described control with the data object of wanting to control its distribution, thereby can use given unique identifier to search for described control module, to extract the respective rule of the further distribution that will be applied to described object.

15. according to the device of claim 14, wherein said rule is any one that comprises in the group that transmission blocking, object modification and object replace.

16. device according to claim 1, wherein said transmission place further comprises reception place recognition unit, be configured to by send the system unit that ICMP message is identified in place, described reception place to described recipient, and allow by described system unit identification and revise response to described ICMP message, thus described system unit is defined as parts at place, described reception place.

17. device according to claim 1, wherein said transmission place further comprises reception place recognition unit, be configured to discern the system unit nearest with described recipient by send ICMP message to described recipient, and allow the response of described ICMP message is discerned and revised by first system unit of described response process, thereby described system unit is defined as the system unit nearest with described recipient.

18. the method at the transmission over networks data object comprises:

Place, transmission place on described network:

Interception and storage go on the recipient road separately through object; And

Generate described unique identifier through object,

Replace described transmission object so that on described network, transmit with described unique identifier, and

Use described unique identifier to come the described object of mark in described buffer memory; And

Place, reception place on described network:

Use described unique identifier to search for corresponding object at least one network node,, thus described corresponding object is offered separately recipient from an approaching relatively network node, and need not carry out redundant Network Transmission to discern described corresponding object.

19. the method for claim 18, wherein said search is carried out in this locality at first, carry out at the recipient's who approaches described expectation node then, and if still do not find corresponding object, so described search is steadily expanded to the farther reposefully node of distance.

20. the method for claim 18 wherein before storage and the described establishment unique identifier, becomes pre-sizing with described through object fragments in described buffer memory.

21. the method for claim 19, wherein the distance of node uses the networking parameter to be determined, the networking parameter comprises at least one in a group: physical distance, bandwidth, two-way time, stand-by period, hop count and economic cost.

22. the method for claim 20 is wherein carried out described search on a plurality of nodes to described expectation recipient's different paths, guaranteed relative network utilisation equilibrium thus.

23. the node of a communication network is configured to be minimized in object transfer redundant on the described network, this node comprises:

The node dictionary is used to store described data object;

Indexing unit with described node dictionary is associated is used for

A) generate the unique identifier that is stored in each data object in the described node dictionary, provide electronic signature for each described data object thus;

C) in described network service, replace described data object with the triplicate of described identifier, continuing at described network communication, and

Extraction unit with described node dictionary is associated is used for

D) receive inquiry from described network to the object that is identified with signing electronically,

E) signature of storing in more described electronic signature and the described buffer memory,

F) if found coupling, extract object so corresponding to the signature of described coupling, and

G) if do not find coupling, send described inquiry to a neighbor node so.

24. a buffer memory dictionary is associated with the node of communication network, described buffer memory dictionary is configured to be minimized in object transfer redundant on the described network, and this buffer memory dictionary can be stored described data object, and further is associated in:

Indexing unit is used for

A) generation is stored in the unique identifier of each data object of described buffer memory dictionary, provides electronic signature for each described data object thus;

Extraction unit is used for

E) signature of storing in more described electronic signature and the described buffer memory dictionary,

G) if do not find coupling, send described inquiry to a neighbor node so.

25. the method at the transmission over networks data object that comprises a plurality of nodes comprises:

Object is passed through in storage arbitrarily in the node that is bordering on the sender that connects,

Generate the unique identifier of described object;

To store in the described relative near node with the described unique identifier that described object is associated;

Expectation recipient to described object sends described unique identifier;

Be bordering on described expectation recipient's node connecting, use the unique identifier of described transmission to come to compare, to find coupling with the identifier of objects stored wherein;

If found coupling, extract corresponding storage object so, and send to described expectation recipient;

26. the method for claim 25 wherein uses the networking parameter to determine the distance of node, the networking parameter comprises at least one in the group, and this group comprises physical distance, bandwidth, two-way time, stand-by period, hop count and economic cost.

27. one kind is used for the device that on network remote identification approaches the system unit of given position most, this device comprises:

Transmitting element, be used for sending described response request to described given position, thereby described response request is received by described given position, and send it back a response, described response is discernible for described system unit, thereby first system unit receives the described response of giving described device with oneself sign.

28. device according to claim 27, wherein said response request is an ICMP message, comprise first field with first number, with second field with second number, this second number is the predefined function of described first number, and described predefined function is made by described system unit and is used for discerning described response.

29. one kind on network remote identification approach the method for given position most, this method comprises:

Generation is for the discernible response request of system unit separately,

Send described response request to described given position, thereby described response request is received by described given position, and send it back a response, described response is discernible for described system unit, thereby first system unit receives the described response with oneself sign.

30. the method for claim 29, the method that wherein generates described response request is included in to be inserted first number and is that described predefined function can make described system unit can discern described response in second number of predefined function of described first number in the described response request.

31. device that is used to search and receive the immediate adjacent destination node in place, wherein send the place and further comprise reception place recognition unit, be configured to by send the system unit that ICMP message is identified in place, described reception place to described recipient, and allow the response of described ICMP message is discerned and revised by described system unit, thereby described system unit is defined as parts at place, described reception place.

32. device that is used to search and send immediate reception place, place, wherein said transmission place further comprises reception place recognition unit, be configured to discern the system unit that approaches described recipient most by sending ICMP message to described recipient, and allow to the response of described ICMP message by described response process the identification of first system unit and revise, thus described system unit is defined as approaching most described recipient's system unit.

33. a device that is used to search and receive the immediate adjacent destination node in place, this method comprises:

Be identified in the system unit that described reception place is located by send ICMP message to described recipient,

Identification is to the response of described ICMP message, and

34. a method that is used to search and send immediate reception place, place, this method comprises:

Discern the system unit that approaches described take over party most by sending ICMP message to described recipient, and

And make response to described ICMP message by the identification of first system unit of described response process with revise, thereby described system unit is defined as approaching most described recipient's system unit.