EP2289217A1 - Maintaining distributed hash tables in an overlay network - Google Patents

Maintaining distributed hash tables in an overlay network

Info

Publication number
EP2289217A1
EP2289217A1 EP08759976A EP08759976A EP2289217A1 EP 2289217 A1 EP2289217 A1 EP 2289217A1 EP 08759976 A EP08759976 A EP 08759976A EP 08759976 A EP08759976 A EP 08759976A EP 2289217 A1 EP2289217 A1 EP 2289217A1
Authority
EP
European Patent Office
Prior art keywords
node
nodes
routing table
neighbouring
overlay network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP08759976A
Other languages
German (de)
French (fr)
Other versions
EP2289217B1 (en
Inventor
Jouni MÄENPÄÄ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telefonaktiebolaget LM Ericsson AB
Original Assignee
Telefonaktiebolaget LM Ericsson AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget LM Ericsson AB filed Critical Telefonaktiebolaget LM Ericsson AB
Priority to EP13170003.1A priority Critical patent/EP2634997B1/en
Publication of EP2289217A1 publication Critical patent/EP2289217A1/en
Application granted granted Critical
Publication of EP2289217B1 publication Critical patent/EP2289217B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • H04L67/1044Group management mechanisms 
    • H04L67/1048Departure or maintenance mechanisms
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • H04L67/1061Peer-to-peer [P2P] networks using node-based peer discovery mechanisms
    • H04L67/1065Discovery involving distributed pre-established resource-based relationships among peers, e.g. based on distributed hash tables [DHT] 

Definitions

  • the present invention relates to a mechanism for maintaining distributed hash tables in an overlay network.
  • the invention is applicable in particular to an optimised procedure for handling the leaving of a node from an overlay network
  • Peer-to-peer or P2P networks make use of the pooled resources of participating nodes including processing capabilities and communication bandwidth to facilitate a wide variety of services including file sharing and VoIP telephony.
  • particular P2P services may make use of "overlay networks" to optimise resource location.
  • An overlay network comprises nodes connected by virtual links representing paths extending across possibly many physical links in the underlying network (e.g. the Internet). Each node in the overlay network maintains a routing table containing a set of links to certain other nodes within the overlay network. Resource requests are passed between nodes until they arrive at a node which is responsible for that resource.
  • DHT Distributed Hash Tables
  • keys e.g. song titles, SIP URIs, etc
  • a finite value space e.g. 128 bits.
  • the hashing algorithm is chosen to ensure a relatively uniform spread of hash values across the value space. Thus, for example, the hashing of 100 song titles will likely result in 100 hash values that are relatively evenly spaced across the value space.
  • Nodes within an overlay network are identified by usernames, which are themselves hashed into respective hash values. Each node then becomes responsible for a set of hash values within the value space which neighbour its own value.
  • a node will store locations (e.g. IP addresses) from which resources, matching resource names which it "owns", can be obtained.
  • locations e.g. IP addresses
  • the node determines whether or not it owns the corresponding hash value. If so, it returns the location of the resource to the requester (via the overlay network). If it does not own the hash value, it inspects its routing table to identify that node within the table which has a hash value closest to the hash value of the request, and forwards the request to that node. The receiving node repeats the procedure, and so on until the request arrives at the node which does own the hash value corresponding to the request and which therefore knows the resource location.
  • FIG. 1 illustrates an overlay network organised as a ring (only a small number of the nodes within the ring are illustrated).
  • each node maintains a routing table containing the locations and hash values of a small number of succeeding and preceding nodes in the ring, as well as for a small number of more distant nodes.
  • a Node X maintains within its routing table locations for two successor nodes and two predecessor nodes, as well as for three remote nodes. Whilst a larger number of entries within the routing tables can make the network more efficient in terms of routing and more robust against node withdrawal, large tables are difficult to maintain and therefore increase the unreliability of the network.
  • a node within the overlay network ensures that the information in its routing table is up to date by attempting to contact its neighbours periodically.
  • a number of different mechanisms may be used for this purpose:
  • a node can periodically send keep-alive messages to check that the other nodes listed in its routing table have not left the overlay network.
  • DHT approaches such as Pastry [A. Rowstron and P. Druschel: Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. Middleware, 2001], Chord [I. Stoica, R. Morris, D. Karger, M. F. Kaashoek and H. Balakrishnan: Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications. In Proceedings of the ACM SIGCOMM'01 Conference, August 2001, San Diego, California, USA.] and Content Addressable Network (CAN) [S. Ratsanamy, P. Francis, M.
  • a node can periodically send queries to learn about new nodes that could be inserted into the routing table, replacing old entries (e.g. Chord).
  • a node can periodically send queries to its direct neighbours requesting information about the entries in its neighbours' routing tables. This information is used to update the node's own routing table (e.g. Chord).
  • Chord e.g. Chord
  • a node can periodically send its own routing table to its neighbours (e.g. CAN).
  • Kademlia P. Maymounkov and D. Mazieres: Kademlia: A peer-to-peer information system based on the xor metric. In Proceedings of IPTPS02, Cambridge, USA, March 2002]).
  • FIG. 2 shows an example of neighbourhood relations in a DHT.
  • a ring topology is assumed.
  • Node X maintains three successors and three predecessor pointers in its routing table.
  • the reason for maintaining multiple successor and predecessor pointers is to increase robustness. If the probability that a single successors will fail is p, then the probability that all three successors will fail simultaneously isp 3 .
  • this is not sufficient to maintain connectivity in the network; if all three successors (or alternatively, all three predecessors) of a given node leave the network within a sufficiently short period of time, the network fragments.
  • Nodes can leave the network either gracefully or ungracefully.
  • a node informs its neighbours about its intention to leave the network prior to actually leaving. The node does this by sending a Leave message (understood at the application layer). This allows the neighbours to immediately remove the leaving node from their routing tables.
  • a node leaves the network ungracefully, it exits the network without first informing its neighbours. Therefore, the neighbours must detect for themselves that the node has left.
  • Reasons for ungraceful leaving include the following: (i) the node has crashed, (ii) the P2P application has crashed or has been closed down unexpectedly, and (iii) selfish behaviour.
  • a user may choose to leave ungracefully in order to avoid delays inherent in a graceful departure.
  • nodes can learn that a neighbour has left in two different ways:
  • a node When the transport protocol is unreliable (e.g. UDP), a node does not learn that the neighbour has left until it attempts to send the next periodic DHT maintenance message to the neighbour. In addition to waiting for the transmission of the next periodic maintenance message, the node also has to wait until the transaction times out before it can be certain that the neighbour has really left.
  • UDP transport protocol
  • the node In addition to waiting for the transmission of the next periodic maintenance message, the node also has to wait until the transaction times out before it can be certain that the neighbour has really left.
  • each of the leaving node's direct neighbours has one pointer less in its routing table. For instance, if node Sl in Figure 1 leaves the network, Node X only has two successor pointers left, namely S2 and S3. If also S2 and S3 also leave the network before Node X has a chance to find additional successors, the overlay network becomes partitioned since
  • Node X no longer knows any successors. A similar situation can arise in respect of the predecessor nodes of Node X.
  • a problem with existing solutions is that if all of the successors or predecessors of a given node fail within a short period of time, the network can become partitioned, and resource requests cannot bridge the gap.
  • This "short period of time” refers to the time between two consecutive DHT maintenance messages. If such maintenance messages are sent for instance every 60 seconds, then the overlay network is disrupted if all the successors or predecessors of even a single node leave the overlay network within this 60 second period. This is not an unlikely event if the network is experiencing a high "churn" rate. Whilst an intuitive solution to this problem would be to send DHT maintenance messages more frequently, the interval between periodic maintenance messages cannot be made arbitrarily small as the resulting signalling load would overload the network. This problem has been confirmed by S. Rhea, D. Geels, T. Roscoe and J. Kubiatowicz: Handling Churn in a DHT, In Proceedings of the USENIX Annual Technical Conference, June 2004.
  • a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
  • the method comprises, upon or immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node (or one of the neighbouring nodes of the departing node aware of the departure) to each neighbouring node (or each other neighbouring node of the departing node), indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node.
  • Each neighbouring node (or each other neighbouring node) receives the Leave request and uses said mapping(s) to update its routing table.
  • Embodiments of the invention allow neighbours of a withdrawing node to quickly update their routing tables with a replacement predecessor or successor node. In the event of high churn in the network, the risk of a fracture in the network chain is greatly reduced.
  • a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
  • the method comprises, immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node to each neighbouring node of the departing node, indicating the departure and containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node.
  • the node Upon receipt of a Leave request at each neighbouring node, uses the mapping(s) to update its routing table.
  • At least one mapping contained within a Leave request corresponds to a neighbouring node of the departing node which is unknown to the recipient node.
  • a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
  • the method comprises, upon departure of a node from the overlay network, sending a Leave request from one of the neighbouring nodes of the departing node aware of the departure, to other neighbouring node of the departing node, the request indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node.
  • the node Upon receipt of a Leave request at said each other neighbouring node, the node uses said mapping(s) to update its routing table.
  • the node sending the Leave request(s) detects the departure of the departing node as a result of the failure of the departing node to respond to periodically transmitted keep-alive messages. It reacts to the detection by sending the Leave request(s).
  • Said one or more mappings for nodes not contained within the routing table of the recipient node may be mappings contained within the routing table of the node sending the Leave request.
  • a node receiving a Leave request may determine if the recipient node is aware of any neighbouring nodes of the departing node and of which the node sending the Leave request is unaware. If so, the receiving node sends a Leave request to that or those nodes, the Leave request containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node. Upon receipt of the Leave request at the or each further recipient node, the node updates its routing table.
  • said step of updating the routing table at a recipient node may comprise deleting the mapping corresponding to the departing node and adding the new mappings contained within the received Leave request to the routing table.
  • a feature that may be advantageously incorporated into embodiments of the invention is the caching one or more mappings for nodes not included within the respective routing tables.
  • the routing table of at least one neighbouring node of the departing node can be updated to include one or more of the cached mappings.
  • a node for use within an overlay network and comprising a memory for storing a routing table containing, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
  • the node also comprises a processing unit configured to send a Leave request to one or neighbouring nodes of the node upon departure of the node or of a neighbouring node from the network, the Leave request identifying the departing node and containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node.
  • the node may comprise a further processing unit configured to receive a Leave request from a neighbouring node, to delete the mapping corresponding to the departing node identified in the Leave request from the routing table contained within said memory, and to add one or more new mappings contained within the Leave request to the routing table.
  • a still further processing unit may be configured to determine if the Leave request originates from the departing node and, if not, to inspect said routing table to identify any neighbouring nodes of the departing node of which the node sending the Leave request is unaware, and to send a Leave request to any such identified nodes. This Leave request identifies the departing node and contains one or more mappings for nodes not contained within the routing table of the recipient node.
  • the node may comprise a further memory for caching one or more mappings for nodes not included within the routing table and a further processor for updating the routing table to include one or more of the cached mappings, in the event of the departure of a neighbouring node from the network.
  • a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
  • the method comprises periodically exchanging maintenance messages between said nodes in order to provide updated addressing information for nodes.
  • addressing information is received at a given node for a peer node and that peer node is not included within the routing table of the given node, the information is cached at the given node.
  • the peer node is added to the routing table using the cached information.
  • Figure 1 illustrates schematically a DHT -based ring overlay network comprising a number of nodes
  • Figure 2 further illustrates a DHT -based ring overlay network, showing neighbourhood relations between nodes;
  • Figure 3 illustrates schematically a node of the overlay network of Figure 2;
  • Figure 4 is a flow diagram illustrating a mechanism for handling the graceful departure of a node from the overlay network of Figure 2; and
  • Figure 5 is a flow diagram illustrating a mechanism for handling the ungraceful departure of a node from the overlay network of Figure 2;
  • the optimized DHT leave operation described here relies upon a node assisting a neighbouring node to quickly update its routing tables in the event that the neighbouring node's routing table is affected by the departure of a node from the network.
  • the departing node may be the neighbouring node, i.e. in the graceful departure scenario, or another neighbouring node in the case of an ungraceful departure.
  • FIG. 2 shows an example of a Chord DHT -based overlay network using a ring topology.
  • Chord DHT is used here by way of example, the procedures described are equally applicable to other DHT -based overlay networks.
  • each node in the DHT-based overlay network maintains pointers to six neighbours: namely three predecessor nodes and three successor nodes.
  • the proposed mechanism works with any number of successor and predecessor pointers.
  • a Node X has three successors: Sl, S2, and S3. If the overlay network is experiencing heavy "churn", all of the successors of Node X might choose to leave the overlay network within a short time frame. Assuming that the nodes depart gracefully, they will send Leave requests to their neighbours. However, as described above, Node X may not have time to identify any other successor nodes before all three successors have left the network. In order to avoid a partitioning of the network in this scenario, each node that intends to leave the overlay helps its neighbours to fill their routing tables with alternative neighbour nodes, before actually leaving the network.
  • neighbor table is used to refer to a part of the routing table containing pointers to direct neighbours
  • predecessor table is used to refer to the part of the neighbour table containing the predecessor pointers
  • successor table is used to refer to the part of the neighbour table containing the successor pointers.
  • node Sl in Figure 2 chooses to leave the overlay network gracefully. As with all other nodes in the network, Sl maintains pointers to neighbouring nodes in its routing table. In Figure 2 it is assumed that node Sl maintains three predecessor and three successor pointers. The successors of node Sl are
  • node Sl Before leaving the network, node Sl repeats the following procedure for each of its neighbours N: • If node N is a predecessor of node Sl, node Sl constructs a new successor table for N and includes it in the Leave message that is sent to node N. Node Sl should not include itself in the successor table it creates for node N.
  • the successor table can include nodes between S 1 and N as well as successors of S 1.
  • N is not the immediate predecessor of S 1.
  • Some of these intervening nodes may be previously unknown to N. (According to the Chord DHT algorithm, only the successor of a newly joined node is aware of the node: other nodes will learn about the new node when the next periodic
  • node Sl crates a new predecessors table for node N and includes this in the Leave message sent to node N.
  • This table includes any of Si 's successors that are located between node Sl and node N. Node Sl should not of course include itself in the predecessors table sent to node N.
  • node N • Having received the Leave message containing the successors/predecessors table from node Sl, node N first removes node Sl from its routing table. Next, N goes through the list of nodes carried in the Leave message. For each node in the list, node N compares the node to the entries in its successor and predecessor tables, and inserts a node into the correct position in the table if the node is not already present. This procedure ensures that the overlay network remains in a stable state and retains full connectivity even after the departure of node Sl. That is, the departure of node Sl does not in any way hinder the operation of the network.
  • a different situation arises if a node e.g. node A in Figure 2 leaves the network without informing its neighbours. This could happen due to a sudden crash or some other abnormal termination of the P2P application, or due to selfish behaviour on the part of the leaving node.
  • the neighbour B of node A is the first node within the overlay network to detect that node A has failed (e.g. from the fact that a periodic keep-alive message, maintenance message or any other message targeted to node A has failed).
  • Node B is made responsible for informing the other neighbours of node A of node A's departure and for updating the contents of the other neighbours' neighbour tables.
  • node B cannot re-create the full neighbour table of node A (in this example, node B has no knowledge of S 1 which is a neighbour of node A but not of node B), neighbours other than B must participate in the procedure by updating the neighbour tables of those neighbours of which node B is unaware.
  • the neighbour which first detects the departure of node A is the most distant successor or the most distant predecessor of node A, then that neighbour does not have any information available which the other neighbours do not already know.
  • the most distant neighbours cannot update the contents of the closer neighbours' neighbour tables.
  • the most distant neighbours can still send an empty Leave message on behalf of node A which will trigger the closer neighbours to send Leave requests with neighbour tables to other neighbours (the closer neighbours have information which benefits the other nodes).
  • node Sl is the first node to detect that node X has left, for example due to the failure of node X to respond to a keep-alive message from node Sl.
  • node Sl will create an as-accurate-as-possible representation of the contents of the neighbour table of node X.
  • Sl can recreate information in respect of five of the six neighbours of node X. This is because Sl knows all the successors of node X and two of the predecessors of node X.
  • node Sl performs the following procedure for each neighbour N of node X that it is aware of:
  • Node Sl constructs the contents of the neighbour table of node N.
  • Node S 1 removes node X from the neighbour table it constructed for node N, chooses appropriate nodes from the recovered neighbour table of node X and inserts these into the new neighbour table for node N. More specifically, if node N is a predecessor of node X, node Sl creates a new successor table for node N. Alternatively, if node N is a successor of node X, node Sl creates a new predecessor table for node N.
  • Node Sl sends a Leave request on behalf of node X to node N and includes in this the new successor or predecessor table it has created.
  • Node N updates its neighbour table based on the Leave request it receives from node S 1.
  • node S 1 When determining the contents of the routing table of node X, node S 1 does not know the identity of the most distant predecessor of node X, namely P3. However, both Pl and P2 know the identity of P3. Therefore, when a neighbour N of node X receives a Leave request sent by another node on behalf of node X (it can detect this as the source address of the originator of the Leave request does not match the address of node X), neighbour N carries out the following actions:
  • Node N recreates a representation of the contents of the neighbour table of node X. More specifically, if node N is a predecessor of node X, node N recreates the predecessor table of X. On the other hand, if node N is a successor of node X, node N recreates a representation of the successor table of node X.
  • node N checks if it is aware of any neighbours of node X that the sender of the Leave request (i.e. node Sl) is not aware of. Assuming that node N is node P2 in Figure 2, P2 detects that the sender of the Leave request, node Sl, is not aware of node P3 (i.e. the third predecessor of node X) .
  • Node N sends a Leave request on behalf of node X to each of the neighbours of node X that the sender of the Leave request is not aware of. Assuming again that node N is node P2 of Figure 2 and the sender is Sl, P2 would send a Leave request to node P3.
  • node N returns a list of the neighbours of node X of which the sender of the Leave request (i.e. node Sl) is unaware of (in the response that node N generates to the Leave request).
  • the sender of Leave request i.e. node Sl
  • nodes can then insert these nodes into its predecessor table.
  • the first node to detect that node X has left the overlay network was assumed to be a successor (Sl). If the first node to detect that node X has left the overlay network is a predecessor, the predecessor (e.g. Pl) has no knowledge of the most distant successor of node X, i.e. S3 in this example. In this case, one of the successors of node X, e.g. node Sl, carries out the above-mentioned steps.
  • Maintenance operations are carried out in a DHT network and involve periodic message exchanges between participating nodes in order to learn about new nodes and to check the states of neighbouring nodes.
  • nodes in a DHT -based overlay conventionally use only a part of the information which they receive from their neighbours to update their own routing tables, and discard the rest of the information. For instance, in the example network of Figure 2, although node X learns about the existence of node E from its direct neighbour Pl, node X normally discards this information, since it maintains only three predecessor pointers, predecessors Pl, P2 and P3, in its predecessor table.
  • node X caches node E's contact information, it can more quickly recover from the possible ungraceful departure of node Pl (and also of P2 and P3). In other words, although node X does not insert node E into its neighbour table, node can X store the contact information of node E into another data structure implementing a neighbour cache. If node Pl suddenly leaves the network, node X can check the cache and contact for example node E to see if it is alive and can be added to the routing table.
  • the node 1 comprises a memory 2 configured to store the routing table for the node.
  • the node is provided with an interface 3 to other nodes in the overlay network (typically an interface to an IP network, e.g. the Internet).
  • a first processing unit 4 is configured to receive Leave requests from neighbouring nodes via the interface 3. In the event that a Leave request is received, the first processing unit causes the routing table to be updated as described above.
  • the Leave request is also received by a second processing unit 5 which determines if the departing node is the node that sent the request. If this is not the case, the second processing unit 5 inspects its routing table to identify any neighbouring nodes of the departing node of which the node that sent the request is unaware. It then constructs one or more further Leave requests and sends these to the identified nodes.
  • a third processing unit 6 is configured to handle the graceful departure of the node from the network. It does this by sending a Leave request to the node's neighbours, identifying one or more substitute mappings for each neighbouring node.
  • This third processing unit is also configured to be notified by a connectivity detector 7, responsible for periodically sending keep-alive messages to neighbouring nodes, when a neighbour leaves the overlay network ungracefully, and to send appropriate Leave requests to neighbours of the departed node.
  • a fourth processing unit 8 maintains a cache within a memory 9, containing mappings for nodes not contained within the current routing table. In the event that the fourth processing unit receives a Leave request from a neighbour, the unit may extract a mapping from the cache memory and add this to the routing table.
  • this illustrates the mechanism applied in the graceful departure scenario.
  • the departing node sends Leave requests to its neighbouring nodes (only three of which are illustrated).
  • each of the neighbouring nodes updates its routing tables.
  • Figure 5 illustrates the mechanism applied in the case of the ungraceful departure scenario.
  • a first neighbouring node of a departing node sends a periodic keep-alive message to the departing node.
  • the first neighbour updates its routing table at step T3 to delete the entry for the departed node.
  • the first neighbour sends Leave requests to neighbours of the departed node, on behalf of the departed node.
  • the neighbours update their routing tables.
  • One of the neighbours in this case the third neighbour, is aware of a further neighbour of the departed node which the first neighbour is unaware of.
  • the neighbour sends a further leave request to that further neighbour.
  • the further neighbour updates its routing table.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer And Data Communications (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

A method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node. The method comprises, upon or immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node (or one of the neighbouring nodes of the departing node aware of the departure) to each neighbouring node (or each other neighbouring node of the departing node), indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node. Each neighbouring node (or each other neighbouring node) receives the Leave request and uses said mapping(s) to update its routing table.

Description

MAINTAINING DISTRIBUTED HASH TABLES IN AN OVERLAY NETWORK
Technical Field
The present invention relates to a mechanism for maintaining distributed hash tables in an overlay network. The invention is applicable in particular to an optimised procedure for handling the leaving of a node from an overlay network
Background
Peer-to-peer or P2P networks make use of the pooled resources of participating nodes including processing capabilities and communication bandwidth to facilitate a wide variety of services including file sharing and VoIP telephony. In the absence of central servers, particular P2P services may make use of "overlay networks" to optimise resource location. An overlay network comprises nodes connected by virtual links representing paths extending across possibly many physical links in the underlying network (e.g. the Internet). Each node in the overlay network maintains a routing table containing a set of links to certain other nodes within the overlay network. Resource requests are passed between nodes until they arrive at a node which is responsible for that resource.
Distributed Hash Tables (DHT) provide an efficient means for mapping resource names ("keys") to locations within an overlay network. DHT makes use of a hashing algorithm to map keys, e.g. song titles, SIP URIs, etc, to a finite value space, e.g. 128 bits. The hashing algorithm is chosen to ensure a relatively uniform spread of hash values across the value space. Thus, for example, the hashing of 100 song titles will likely result in 100 hash values that are relatively evenly spaced across the value space. Nodes within an overlay network are identified by usernames, which are themselves hashed into respective hash values. Each node then becomes responsible for a set of hash values within the value space which neighbour its own value. In practice, a node will store locations (e.g. IP addresses) from which resources, matching resource names which it "owns", can be obtained. When a node in the overlay network receives a request for a resource, the node determines whether or not it owns the corresponding hash value. If so, it returns the location of the resource to the requester (via the overlay network). If it does not own the hash value, it inspects its routing table to identify that node within the table which has a hash value closest to the hash value of the request, and forwards the request to that node. The receiving node repeats the procedure, and so on until the request arrives at the node which does own the hash value corresponding to the request and which therefore knows the resource location.
Figure 1 illustrates an overlay network organised as a ring (only a small number of the nodes within the ring are illustrated). In this example, each node maintains a routing table containing the locations and hash values of a small number of succeeding and preceding nodes in the ring, as well as for a small number of more distant nodes. In the illustrated network, a Node X maintains within its routing table locations for two successor nodes and two predecessor nodes, as well as for three remote nodes. Whilst a larger number of entries within the routing tables can make the network more efficient in terms of routing and more robust against node withdrawal, large tables are difficult to maintain and therefore increase the unreliability of the network.
A node within the overlay network ensures that the information in its routing table is up to date by attempting to contact its neighbours periodically. A number of different mechanisms may be used for this purpose:
1) A node can periodically send keep-alive messages to check that the other nodes listed in its routing table have not left the overlay network. This mechanism is used by DHT approaches such as Pastry [A. Rowstron and P. Druschel: Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. Middleware, 2001], Chord [I. Stoica, R. Morris, D. Karger, M. F. Kaashoek and H. Balakrishnan: Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications. In Proceedings of the ACM SIGCOMM'01 Conference, August 2001, San Diego, California, USA.] and Content Addressable Network (CAN) [S. Ratsanamy, P. Francis, M. Handley, R. Karp and S. Shenker: A scalable content-addressable network. In Proceedings of ACM SIGCOMM 2001, August 2001]. 2) A node can periodically send queries to learn about new nodes that could be inserted into the routing table, replacing old entries (e.g. Chord).
3) A node can periodically send queries to its direct neighbours requesting information about the entries in its neighbours' routing tables. This information is used to update the node's own routing table (e.g. Chord).
4) A node can periodically send its own routing table to its neighbours (e.g. CAN).
Another (additional) approach to maintaining the routing tables involves a node checking whether the originator of a resource request could be inserted into its routing table (e.g. Kademlia [P. Maymounkov and D. Mazieres: Kademlia: A peer-to-peer information system based on the xor metric. In Proceedings of IPTPS02, Cambridge, USA, March 2002]).
Consider Figure 2 which shows an example of neighbourhood relations in a DHT. In the Figure, a ring topology is assumed. Node X maintains three successors and three predecessor pointers in its routing table. It should already be clear that the reason for maintaining multiple successor and predecessor pointers is to increase robustness. If the probability that a single successors will fail is p, then the probability that all three successors will fail simultaneously isp3. However, in extremely large real- world DHT- based overlay networks, this is not sufficient to maintain connectivity in the network; if all three successors (or alternatively, all three predecessors) of a given node leave the network within a sufficiently short period of time, the network fragments.
Nodes can leave the network either gracefully or ungracefully. When leaving gracefully, a node informs its neighbours about its intention to leave the network prior to actually leaving. The node does this by sending a Leave message (understood at the application layer). This allows the neighbours to immediately remove the leaving node from their routing tables. When a node leaves the network ungracefully, it exits the network without first informing its neighbours. Therefore, the neighbours must detect for themselves that the node has left. Reasons for ungraceful leaving include the following: (i) the node has crashed, (ii) the P2P application has crashed or has been closed down unexpectedly, and (iii) selfish behaviour. In alternative (iii), a user may choose to leave ungracefully in order to avoid delays inherent in a graceful departure. In the case of an ungraceful departure from the overlay network, nodes can learn that a neighbour has left in two different ways:
1) When the underlying transport protocol is reliable (e.g. TCP), the departure of a neighbour is detected quickly from the fact that the transport layer connection goes down.
2) When the transport protocol is unreliable (e.g. UDP), a node does not learn that the neighbour has left until it attempts to send the next periodic DHT maintenance message to the neighbour. In addition to waiting for the transmission of the next periodic maintenance message, the node also has to wait until the transaction times out before it can be certain that the neighbour has really left.
In the case of both graceful and ungraceful departure, the end result is that each of the leaving node's direct neighbours has one pointer less in its routing table. For instance, if node Sl in Figure 1 leaves the network, Node X only has two successor pointers left, namely S2 and S3. If also S2 and S3 also leave the network before Node X has a chance to find additional successors, the overlay network becomes partitioned since
Node X no longer knows any successors. A similar situation can arise in respect of the predecessor nodes of Node X.
Thus, a problem with existing solutions is that if all of the successors or predecessors of a given node fail within a short period of time, the network can become partitioned, and resource requests cannot bridge the gap. This "short period of time" refers to the time between two consecutive DHT maintenance messages. If such maintenance messages are sent for instance every 60 seconds, then the overlay network is disrupted if all the successors or predecessors of even a single node leave the overlay network within this 60 second period. This is not an unlikely event if the network is experiencing a high "churn" rate. Whilst an intuitive solution to this problem would be to send DHT maintenance messages more frequently, the interval between periodic maintenance messages cannot be made arbitrarily small as the resulting signalling load would overload the network. This problem has been confirmed by S. Rhea, D. Geels, T. Roscoe and J. Kubiatowicz: Handling Churn in a DHT, In Proceedings of the USENIX Annual Technical Conference, June 2004.
Summary
It is an object of the present invention to minimise disruption to an overlay network when a node fails or otherwise withdraws from the network. At least certain embodiments of the invention achieve this object by allowing the withdrawing node, or a neighbouring node of the withdrawing node, to update the routing tables of other neighbours.
According to a first aspect of the invention there is provided a method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
The method comprises, upon or immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node (or one of the neighbouring nodes of the departing node aware of the departure) to each neighbouring node (or each other neighbouring node of the departing node), indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node. Each neighbouring node (or each other neighbouring node) receives the Leave request and uses said mapping(s) to update its routing table.
Embodiments of the invention allow neighbours of a withdrawing node to quickly update their routing tables with a replacement predecessor or successor node. In the event of high churn in the network, the risk of a fracture in the network chain is greatly reduced.
According to a second aspect of the invention there is provided a method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node. The method comprises, immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node to each neighbouring node of the departing node, indicating the departure and containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node. Upon receipt of a Leave request at each neighbouring node, the node uses the mapping(s) to update its routing table.
In an embodiment of this aspect of the invention, at least one mapping contained within a Leave request corresponds to a neighbouring node of the departing node which is unknown to the recipient node.
According to a third aspect of the invention there is provided a method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
The method comprises, upon departure of a node from the overlay network, sending a Leave request from one of the neighbouring nodes of the departing node aware of the departure, to other neighbouring node of the departing node, the request indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node. Upon receipt of a Leave request at said each other neighbouring node, the node uses said mapping(s) to update its routing table.
In an embodiment of this aspect of the invention, the node sending the Leave request(s) detects the departure of the departing node as a result of the failure of the departing node to respond to periodically transmitted keep-alive messages. It reacts to the detection by sending the Leave request(s). Said one or more mappings for nodes not contained within the routing table of the recipient node may be mappings contained within the routing table of the node sending the Leave request.
A node receiving a Leave request may determine if the recipient node is aware of any neighbouring nodes of the departing node and of which the node sending the Leave request is unaware. If so, the receiving node sends a Leave request to that or those nodes, the Leave request containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node. Upon receipt of the Leave request at the or each further recipient node, the node updates its routing table.
It will be understood that said step of updating the routing table at a recipient node may comprise deleting the mapping corresponding to the departing node and adding the new mappings contained within the received Leave request to the routing table.
A feature that may be advantageously incorporated into embodiments of the invention is the caching one or more mappings for nodes not included within the respective routing tables. In the event of the departure of a node from the network, the routing table of at least one neighbouring node of the departing node can be updated to include one or more of the cached mappings.
According to a fourth aspect of the present invention there is provided a node for use within an overlay network and comprising a memory for storing a routing table containing, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node. The node also comprises a processing unit configured to send a Leave request to one or neighbouring nodes of the node upon departure of the node or of a neighbouring node from the network, the Leave request identifying the departing node and containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node.
The node may comprise a further processing unit configured to receive a Leave request from a neighbouring node, to delete the mapping corresponding to the departing node identified in the Leave request from the routing table contained within said memory, and to add one or more new mappings contained within the Leave request to the routing table. A still further processing unit may be configured to determine if the Leave request originates from the departing node and, if not, to inspect said routing table to identify any neighbouring nodes of the departing node of which the node sending the Leave request is unaware, and to send a Leave request to any such identified nodes. This Leave request identifies the departing node and contains one or more mappings for nodes not contained within the routing table of the recipient node.
The node may comprise a further memory for caching one or more mappings for nodes not included within the routing table and a further processor for updating the routing table to include one or more of the cached mappings, in the event of the departure of a neighbouring node from the network.
According to a fifth aspect of the present invention there is provided a method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node.
The method comprises periodically exchanging maintenance messages between said nodes in order to provide updated addressing information for nodes. When addressing information is received at a given node for a peer node and that peer node is not included within the routing table of the given node, the information is cached at the given node. In the event that a node contained within the routing table of the given node withdraws from the network, the peer node is added to the routing table using the cached information.
Brief Description of the Drawings
Figure 1 illustrates schematically a DHT -based ring overlay network comprising a number of nodes;
Figure 2 further illustrates a DHT -based ring overlay network, showing neighbourhood relations between nodes; Figure 3 illustrates schematically a node of the overlay network of Figure 2;
Figure 4 is a flow diagram illustrating a mechanism for handling the graceful departure of a node from the overlay network of Figure 2; and Figure 5 is a flow diagram illustrating a mechanism for handling the ungraceful departure of a node from the overlay network of Figure 2; and
Detailed Description
The optimized DHT leave operation described here relies upon a node assisting a neighbouring node to quickly update its routing tables in the event that the neighbouring node's routing table is affected by the departure of a node from the network. The departing node may be the neighbouring node, i.e. in the graceful departure scenario, or another neighbouring node in the case of an ungraceful departure. These two scenarios will now be considered in detail.
Reference is made again to Figure 2 which shows an example of a Chord DHT -based overlay network using a ring topology. Whilst Chord DHT is used here by way of example, the procedures described are equally applicable to other DHT -based overlay networks. In the example illustrated, it is assumed that each node in the DHT-based overlay network maintains pointers to six neighbours: namely three predecessor nodes and three successor nodes. Of course, the proposed mechanism works with any number of successor and predecessor pointers.
In Figure 2, a Node X has three successors: Sl, S2, and S3. If the overlay network is experiencing heavy "churn", all of the successors of Node X might choose to leave the overlay network within a short time frame. Assuming that the nodes depart gracefully, they will send Leave requests to their neighbours. However, as described above, Node X may not have time to identify any other successor nodes before all three successors have left the network. In order to avoid a partitioning of the network in this scenario, each node that intends to leave the overlay helps its neighbours to fill their routing tables with alternative neighbour nodes, before actually leaving the network.
In the text below, the term "neighbour table" is used to refer to a part of the routing table containing pointers to direct neighbours, the term "predecessor table" is used to refer to the part of the neighbour table containing the predecessor pointers, and the term "successor table" is used to refer to the part of the neighbour table containing the successor pointers.
Consider the case where node Sl in Figure 2 chooses to leave the overlay network gracefully. As with all other nodes in the network, Sl maintains pointers to neighbouring nodes in its routing table. In Figure 2 it is assumed that node Sl maintains three predecessor and three successor pointers. The successors of node Sl are
S2, S3 and A. The predecessors of node Sl include X, Pl and P2. Before leaving the network, node Sl repeats the following procedure for each of its neighbours N: • If node N is a predecessor of node Sl, node Sl constructs a new successor table for N and includes it in the Leave message that is sent to node N. Node Sl should not include itself in the successor table it creates for node N. The successor table can include nodes between S 1 and N as well as successors of S 1.
Indeed, this will be the case when N is not the immediate predecessor of S 1. Some of these intervening nodes may be previously unknown to N. (According to the Chord DHT algorithm, only the successor of a newly joined node is aware of the node: other nodes will learn about the new node when the next periodic
DHT maintenance message is scheduled).
• If node N is a successor of node Sl, node Sl crates a new predecessors table for node N and includes this in the Leave message sent to node N. This table includes any of Si 's successors that are located between node Sl and node N. Node Sl should not of course include itself in the predecessors table sent to node N.
• Having received the Leave message containing the successors/predecessors table from node Sl, node N first removes node Sl from its routing table. Next, N goes through the list of nodes carried in the Leave message. For each node in the list, node N compares the node to the entries in its successor and predecessor tables, and inserts a node into the correct position in the table if the node is not already present. This procedure ensures that the overlay network remains in a stable state and retains full connectivity even after the departure of node Sl. That is, the departure of node Sl does not in any way hinder the operation of the network.
A different situation arises if a node (e.g. node A in Figure 2) leaves the network without informing its neighbours. This could happen due to a sudden crash or some other abnormal termination of the P2P application, or due to selfish behaviour on the part of the leaving node. Assume that the neighbour B of node A is the first node within the overlay network to detect that node A has failed (e.g. from the fact that a periodic keep-alive message, maintenance message or any other message targeted to node A has failed). Node B is made responsible for informing the other neighbours of node A of node A's departure and for updating the contents of the other neighbours' neighbour tables. However, as node B cannot re-create the full neighbour table of node A (in this example, node B has no knowledge of S 1 which is a neighbour of node A but not of node B), neighbours other than B must participate in the procedure by updating the neighbour tables of those neighbours of which node B is unaware.
In the exceptional case that the neighbour which first detects the departure of node A is the most distant successor or the most distant predecessor of node A, then that neighbour does not have any information available which the other neighbours do not already know. Thus, the most distant neighbours cannot update the contents of the closer neighbours' neighbour tables. However, the most distant neighbours can still send an empty Leave message on behalf of node A which will trigger the closer neighbours to send Leave requests with neighbour tables to other neighbours (the closer neighbours have information which benefits the other nodes).
By way of example, assume that node X in Figure 2 leaves the network without informing its neighbours, i.e. nodes Sl, S2, S3, Pl, P2 and P3. Assume further that node Sl is the first node to detect that node X has left, for example due to the failure of node X to respond to a keep-alive message from node Sl. Immediately after having detected the failure of node X, node Sl will create an as-accurate-as-possible representation of the contents of the neighbour table of node X. In this case, Sl can recreate information in respect of five of the six neighbours of node X. This is because Sl knows all the successors of node X and two of the predecessors of node X. Next, node Sl performs the following procedure for each neighbour N of node X that it is aware of:
• Node Sl constructs the contents of the neighbour table of node N. • Node S 1 removes node X from the neighbour table it constructed for node N, chooses appropriate nodes from the recovered neighbour table of node X and inserts these into the new neighbour table for node N. More specifically, if node N is a predecessor of node X, node Sl creates a new successor table for node N. Alternatively, if node N is a successor of node X, node Sl creates a new predecessor table for node N.
• Node Sl sends a Leave request on behalf of node X to node N and includes in this the new successor or predecessor table it has created.
• Node N updates its neighbour table based on the Leave request it receives from node S 1.
When determining the contents of the routing table of node X, node S 1 does not know the identity of the most distant predecessor of node X, namely P3. However, both Pl and P2 know the identity of P3. Therefore, when a neighbour N of node X receives a Leave request sent by another node on behalf of node X (it can detect this as the source address of the originator of the Leave request does not match the address of node X), neighbour N carries out the following actions:
• Node N recreates a representation of the contents of the neighbour table of node X. More specifically, if node N is a predecessor of node X, node N recreates the predecessor table of X. On the other hand, if node N is a successor of node X, node N recreates a representation of the successor table of node X.
• Based on the neighbour table it constructed, node N checks if it is aware of any neighbours of node X that the sender of the Leave request (i.e. node Sl) is not aware of. Assuming that node N is node P2 in Figure 2, P2 detects that the sender of the Leave request, node Sl, is not aware of node P3 (i.e. the third predecessor of node X) .
• Node N sends a Leave request on behalf of node X to each of the neighbours of node X that the sender of the Leave request is not aware of. Assuming again that node N is node P2 of Figure 2 and the sender is Sl, P2 would send a Leave request to node P3.
• Finally, node N returns a list of the neighbours of node X of which the sender of the Leave request (i.e. node Sl) is unaware of (in the response that node N generates to the Leave request). The sender of Leave request (i.e. node Sl) can then insert these nodes into its predecessor table.
In the example above, the first node to detect that node X has left the overlay network was assumed to be a successor (Sl). If the first node to detect that node X has left the overlay network is a predecessor, the predecessor (e.g. Pl) has no knowledge of the most distant successor of node X, i.e. S3 in this example. In this case, one of the successors of node X, e.g. node Sl, carries out the above-mentioned steps.
Maintenance operations are carried out in a DHT network and involve periodic message exchanges between participating nodes in order to learn about new nodes and to check the states of neighbouring nodes. However, nodes in a DHT -based overlay conventionally use only a part of the information which they receive from their neighbours to update their own routing tables, and discard the rest of the information. For instance, in the example network of Figure 2, although node X learns about the existence of node E from its direct neighbour Pl, node X normally discards this information, since it maintains only three predecessor pointers, predecessors Pl, P2 and P3, in its predecessor table. However, if node X caches node E's contact information, it can more quickly recover from the possible ungraceful departure of node Pl (and also of P2 and P3). In other words, although node X does not insert node E into its neighbour table, node can X store the contact information of node E into another data structure implementing a neighbour cache. If node Pl suddenly leaves the network, node X can check the cache and contact for example node E to see if it is alive and can be added to the routing table.
Referring now to Figure 3, there is illustrated schematically a node 1 of a DHT-based overlay network suitable for implementing the described mechanism. The node 1 comprises a memory 2 configured to store the routing table for the node. The node is provided with an interface 3 to other nodes in the overlay network (typically an interface to an IP network, e.g. the Internet). A first processing unit 4 is configured to receive Leave requests from neighbouring nodes via the interface 3. In the event that a Leave request is received, the first processing unit causes the routing table to be updated as described above. The Leave request is also received by a second processing unit 5 which determines if the departing node is the node that sent the request. If this is not the case, the second processing unit 5 inspects its routing table to identify any neighbouring nodes of the departing node of which the node that sent the request is unaware. It then constructs one or more further Leave requests and sends these to the identified nodes.
A third processing unit 6 is configured to handle the graceful departure of the node from the network. It does this by sending a Leave request to the node's neighbours, identifying one or more substitute mappings for each neighbouring node. This third processing unit is also configured to be notified by a connectivity detector 7, responsible for periodically sending keep-alive messages to neighbouring nodes, when a neighbour leaves the overlay network ungracefully, and to send appropriate Leave requests to neighbours of the departed node.
A fourth processing unit 8 maintains a cache within a memory 9, containing mappings for nodes not contained within the current routing table. In the event that the fourth processing unit receives a Leave request from a neighbour, the unit may extract a mapping from the cache memory and add this to the routing table.
With reference now to Figure 4, this illustrates the mechanism applied in the graceful departure scenario. At steps Sl to S3, the departing node sends Leave requests to its neighbouring nodes (only three of which are illustrated). At steps S4 to S6, each of the neighbouring nodes updates its routing tables.
Figure 5 illustrates the mechanism applied in the case of the ungraceful departure scenario. At step Tl, a first neighbouring node of a departing node sends a periodic keep-alive message to the departing node. After a timeout T2 at the first neighbour during which no response to the keep-alive is received, the first neighbour updates its routing table at step T3 to delete the entry for the departed node. At steps T4 and T5, the first neighbour sends Leave requests to neighbours of the departed node, on behalf of the departed node. The contain new mappings as described above. Upon receipt of the Leave requests, at steps T6 and T7 the neighbours update their routing tables. One of the neighbours, in this case the third neighbour, is aware of a further neighbour of the departed node which the first neighbour is unaware of. At step T8 the neighbour sends a further leave request to that further neighbour. At step T9 the further neighbour updates its routing table.
As the mechanism described here improves the robustness of a DHT -based overlay network, it is especially useful for critical DHT-based systems such P2PSIP telephony networks such as are being standardized by the P2PSIP working group of the Internet Engineering Task Force (IETF).
It will be appreciated by those of skill in the art that various modifications may be made to the above described embodiment without departing from the scope of the present invention.

Claims

CLAIMS:
1. A method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node, the method comprising: upon or immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node or one of the neighbouring nodes of the departing node aware of the departure, to each neighbouring node or each other neighbouring node of the departing node, indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node; and receiving a Leave request at each neighbouring node or at each other neighbouring node and using said mapping(s) to update the routing table at the recipient node.
2. A method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node, the method comprising: immediately prior to departure of a node from the overlay network, sending a Leave request from the departing node to each neighbouring node of the departing node, indicating the departure and containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node; and receiving a Leave request at each neighbouring node and using said mapping(s) to update the routing table at the recipient node.
3. A method according to claim 2, wherein at least one mapping contained within a Leave request corresponds to a neighbouring node of the departing node which is unknown to the recipient node.
4. A method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node, the method comprising: upon departure of a node from the overlay network, sending a Leave request from one of the neighbouring nodes of the departing node aware of the departure, to other neighbouring node of the departing node, the request indicating the departure and containing one or more mappings for nodes not contained within the routing table of the recipient node; and receiving a Leave request at said each other neighbouring node and using said mapping(s) to update the routing table at the recipient node.
5. A method according to claim 4, wherein the node sending the Leave request(s) detects the departure of the departing node as a result of the failure of the departing node to respond to periodically transmitted keep-alive messages, and reacts to the detection by sending the Leave request(s).
6. A method according to claim 4 or 5, wherein said one or more mappings for nodes not contained within the routing table of the recipient node are mappings contained within the routing table of the node sending the Leave request.
7. A method according to any one of claims 4 to 6 and comprising, at a node receiving a Leave request, determining if the recipient node is aware of any neighbouring nodes of the departing node and of which the node sending the Leave request is unaware and, if so, sending a Leave request to that or those nodes, the Leave request containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node, and, upon receipt of the Leave request at the or each further recipient node, updating the routing table of the node.
8. A method according to any one of the preceding claims, wherein the Leave request contains one of a new successor and predecessor routing table part for the recipient node.
9. A method according to any one of the preceding claims, wherein said overlay network is a Distributed Hash Table network, and said overlay network addresses are hash values.
10. A method according to any one of the preceding claims, said step of updating the routing table at a recipient node comprising deleting the mapping corresponding to the departing node and adding the new mappings contained within the received Leave request to the routing table.
11. A method according to any one of the preceding claims and comprising, at nodes of the overlay network, caching one or more mappings for nodes not included within the respective routing tables and, in the event of the departure of a node from the network, updating the routing table of at least one neighbouring node of the departing node to include one or more of the cached mappings.
12. A node for use within an overlay network and comprising: a memory for storing a routing table containing, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node; a processing unit configured to send a Leave request to one or neighbouring nodes of the node upon departure of the node or of a neighbouring node from the network, the Leave request identifying the departing node and containing one or more overlay network address to physical locator mappings for nodes not contained within the routing table of the recipient node.
13. A node according to claim 12 and comprising a further processing unit configured to receive a Leave request from a neighbouring node, to delete the mapping corresponding to the departing node identified in the Leave request from the routing table contained within said memory, and to add one or more new mappings contained within the Leave request to the routing table.
14. A node according to claim 13 and comprising a still further processing unit configured to determine if the Leave request originates from the departing node and, if not, to inspect said routing table to identify any neighbouring nodes of the departing node of which the node sending the Leave request is unaware, and to send a Leave request to any such identified nodes, the Leave request identifying the departing node and containing one or more mappings for nodes not contained within the routing table of the recipient node.
15. A node according to any one of claims 12 to 14 and comprising a further memory for caching one or more mappings for nodes not included within the routing table and a further processor for updating the routing table to include one or more of the cached mappings, in the event of the departure of a neighbouring node from the network.
16. A method of maintaining routing tables at nodes of an overlay network, where a routing table of a given node contains, for each of a set of neighbouring successor and predecessor nodes, a mapping between an overlay network address of the node and a physical locator of the node, the method comprising: periodically exchanging maintenance messages between said nodes in order to provide updated addressing information for nodes; when addressing information is received at a given node for a peer node and that peer node is not included within the routing table of the given node, caching the information at the given node; and in the event that a node contained within the routing table of the given node withdraws from the network, adding the peer node to the routing table using the cached information.
EP08759976.7A 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network Not-in-force EP2289217B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP13170003.1A EP2634997B1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2008/056376 WO2009141013A1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network

Related Child Applications (2)

Application Number Title Priority Date Filing Date
EP13170003.1A Division-Into EP2634997B1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network
EP13170003.1A Division EP2634997B1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network

Publications (2)

Publication Number Publication Date
EP2289217A1 true EP2289217A1 (en) 2011-03-02
EP2289217B1 EP2289217B1 (en) 2017-01-04

Family

ID=40427876

Family Applications (2)

Application Number Title Priority Date Filing Date
EP13170003.1A Not-in-force EP2634997B1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network
EP08759976.7A Not-in-force EP2289217B1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP13170003.1A Not-in-force EP2634997B1 (en) 2008-05-23 2008-05-23 Maintaining distributed hash tables in an overlay network

Country Status (7)

Country Link
US (1) US8526331B2 (en)
EP (2) EP2634997B1 (en)
JP (1) JP5684110B2 (en)
CN (1) CN102037704B (en)
CA (1) CA2724406A1 (en)
HK (1) HK1157096A1 (en)
WO (1) WO2009141013A1 (en)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2139202B1 (en) * 2008-06-27 2012-03-28 Alcatel Lucent Method of providing a successor list
CN101616171B (en) * 2008-06-27 2013-08-28 华为技术有限公司 Method, device and system for retaining node guiding state
CN101997901B (en) * 2009-08-28 2014-04-30 国际商业机器公司 P2P file transfer management method and system
CN102148740B (en) * 2010-02-05 2013-09-18 中国移动通信集团公司 Neighbor cell routing table updating method and system
CN101826986B (en) * 2010-05-04 2011-12-28 中国人民解放军国防科学技术大学 Physical network perceived overlay network constructing method
CN103095744B (en) * 2011-10-28 2016-05-25 中国移动通信集团公司 Peer network node down method and system, peer network node
CN103139081B (en) * 2011-11-28 2017-08-11 中兴通讯股份有限公司 Distributed hashtable routing table update method and node
US9032251B2 (en) * 2013-03-12 2015-05-12 Cray Inc. Re-forming an application control tree without terminating the application
US20140330901A1 (en) * 2013-05-03 2014-11-06 Htc Corporation Method and device for sharing digital object over mesh network
GB2530722B (en) * 2014-09-19 2016-08-17 Imagination Tech Ltd Cache hashing
KR101928529B1 (en) 2015-06-10 2018-12-13 울산과학기술원 Code Distributed Hash Table based MapReduce System and Method
CN108075991B (en) * 2016-11-18 2020-09-08 新华三技术有限公司 Message forwarding method and device
CN110324248B (en) * 2018-03-30 2021-07-30 中移(苏州)软件技术有限公司 Bare metal server route updating method and device, electronic equipment and medium

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7725600B2 (en) * 2004-02-03 2010-05-25 Nokia Corporation Method and apparatus providing address management in a flat structure mobile network
US7640339B1 (en) * 2005-02-14 2009-12-29 Sun Microsystems, Inc. Method and apparatus for monitoring a node in a distributed system
EP1748604B1 (en) 2005-07-28 2010-03-03 Technische Universität Berlin A communication network, a method of routing data packets in such communication network and a method of locating and securing data of a desired resource in such communication network
US7881223B2 (en) * 2006-03-31 2011-02-01 Panasonic Corporation Method for on demand distributed hash table update
US7684352B2 (en) * 2006-11-02 2010-03-23 Nortel Networks Ltd Distributed storage of routing information in a link state protocol controlled network
EP2087667A4 (en) * 2006-11-27 2015-03-04 Ericsson Telefon Ab L M A method and system for providing arouting architecture for overlay networks
US20080225780A1 (en) * 2007-03-13 2008-09-18 Nortel Networks Limited Use of distributed hashtables for wireless access mobility management
CN101499997B (en) * 2008-01-29 2013-06-05 中兴通讯股份有限公司 Apparatus for multi-path low speed service multiplexing and demultiplexing, and method therefor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2009141013A1 *

Also Published As

Publication number Publication date
EP2634997A1 (en) 2013-09-04
WO2009141013A1 (en) 2009-11-26
CN102037704B (en) 2014-08-27
JP5684110B2 (en) 2015-03-11
CN102037704A (en) 2011-04-27
JP2011521573A (en) 2011-07-21
CA2724406A1 (en) 2009-11-26
US8526331B2 (en) 2013-09-03
HK1157096A1 (en) 2012-06-22
EP2289217B1 (en) 2017-01-04
EP2634997B1 (en) 2018-03-07
US20110110269A1 (en) 2011-05-12

Similar Documents

Publication Publication Date Title
EP2289217B1 (en) Maintaining distributed hash tables in an overlay network
EP2308213B1 (en) Maintenance of overlay networks
US8792331B2 (en) Method of providing a successor list
EP2139205B1 (en) Method of redundant data storage
US20060036747A1 (en) System and method for resource handling of SIP messaging
Joung et al. Chord2: A two-layer Chord for reducing maintenance overhead via heterogeneity
EP2234332A1 (en) Method, device and system for diagnosing whether nodes are abnormal or not in p2p network
US9686353B2 (en) Method and node for locating objects in a peer-to-peer network
US20120158756A1 (en) Searching in Peer to Peer Networks
US8817595B2 (en) Overlay network node and overlay networks
Maenpaa et al. Self-Tuning Distributed Hash Table (DHT) for REsource LOcation And Discovery (RELOAD)
El-Ansary et al. An overview of structured overlay networks
JP4947663B2 (en) Delay time determination method, peer node, and program in overlay network
Kunzmann Recursive or iterative routing? Hybrid!
Antonopoulos et al. A Multi-Ring Method for Efficient Multi-Dimensional Data Lookup in P2P Networks.
JP2006221457A (en) Servant for replication control in pure p2p communication, replication control method for the same, and program
Yu et al. BGKR: A novel P2P network based on generalized Kautz and ring with constant congestion
Cai et al. Caching routing indices in structured P2P overlays
Glynn et al. Maintaining a structured overlay network in a hostile environment
Khavari et al. Structured peer-to-peer control plane
Takabatake et al. A p2p file sharing technique by indexed-priority metric
Wauthy P2PSIP: towards a decentralized, SIP-based VoIP system
Vollmann Peer-To-Peer Information Systems
Maenpaa et al. RFC 7363: Self-Tuning Distributed Hash Table (DHT) for REsource LOcation And Discovery (RELOAD)
Mwansa Generic Network Location Service

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20101028

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17Q First examination report despatched

Effective date: 20110607

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: TELEFONAKTIEBOLAGET L M ERICSSON (PUBL)

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602008048234

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04L0029060000

Ipc: H04L0029080000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160822

RIC1 Information provided on ipc code assigned before grant

Ipc: H04L 29/08 20060101AFI20160808BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 860234

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170115

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008048234

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

Ref country code: NL

Ref legal event code: MP

Effective date: 20170104

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 860234

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170104

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170504

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170404

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170531

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170504

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170404

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008048234

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

26N No opposition filed

Effective date: 20171005

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170531

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170523

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20170531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170523

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170523

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20080523

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170104

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170104

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602008048234

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04L0029080000

Ipc: H04L0065000000

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20220527

Year of fee payment: 15

Ref country code: FR

Payment date: 20220525

Year of fee payment: 15

Ref country code: DE

Payment date: 20220527

Year of fee payment: 15

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602008048234

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20230523

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231201

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230523

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230531