WO2011026508A1 - Method for distributing data traffic over radio links - Google Patents
Method for distributing data traffic over radio links Download PDFInfo
- Publication number
- WO2011026508A1 WO2011026508A1 PCT/EP2009/061265 EP2009061265W WO2011026508A1 WO 2011026508 A1 WO2011026508 A1 WO 2011026508A1 EP 2009061265 W EP2009061265 W EP 2009061265W WO 2011026508 A1 WO2011026508 A1 WO 2011026508A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- transmitter
- receiver
- link
- segments
- links
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W40/00—Communication routing or communication path finding
- H04W40/02—Communication route or path selection, e.g. power-based or shortest path routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W92/00—Interfaces specially adapted for wireless communication networks
- H04W92/04—Interfaces between hierarchically different network devices
- H04W92/10—Interfaces between hierarchically different network devices between terminal device and access point, i.e. wireless air interface
Definitions
- the present invention relates to a method for distributing a packet data stream over a plurality of radio links, wherein the stream received by a transmitter is divided by the transmitter and distributed via the radio links to a receiver. It also relates to a transmitter, a receiver and a system being adapted for the same purpose.
- the desired bandwidth for transport of Ethernet traffic over a microwave radio hop is larger than the speed provided by a single radio link.
- several radio links needs to be used in parallel to achieve a larger bandwidth.
- Other type of links such as optical links, where data flowing on a high speed link needs to be transported over several links.
- Existing solutions implements link aggregation, by which Ethernet packets are distributed over the available links.
- the link aggregation solution must send all Ethernet packets belonging to one stream over the same link.
- a hashing algorithm therefore may be applied to certain information that is static in the Ethernet packet for a stream.
- WO2006/065142 relates to inverse multiplexing within telecommunication, and in particular to a versatile inverse multiplex protocol comprising associated state machines. It also relates to a method for using this protocol allowing a high speed link to be distributed via an inverse multiplexer into a number of lower speed connections in pursuance to G.704 recommendations and using Sa bits. With this solution, the packet data stream is divided into bits which are distributed on a plurality of links. The problem with this solution is that it can only be used on low speed links having the same bandwidth.
- the object of the present invention is therefore to effectively distribute high speed data traffic over a number of low speed links with different bandwidth.
- the object is solved by means of a method for distributing a packet data stream over a plurality of radio links.
- the stream received by a transmitter is divided by the transmitter and distributed via the links to a receiver.
- the receiver aggregates the distributed stream.
- the method comprises the steps of:
- the transmitter divides the packets in the stream into segments. It further uses an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links.,
- the receiver identifies the distributed segments using the IM. It further reassembles the segments into packets in order to aggregate the stream.
- the object of the present invention is also solved by means of a transmitter adapted for distributing a packet data stream over a plurality of links.
- the transmitter is adapted to distribute the stream via the links to the receiver according to claim 14.
- the transmitter is particularly characterized in that it is adapted to divide the packets in the stream into segments. It is further adapted to use an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links.
- IM inverse multiplexer protocol
- the object of the present invention is also solved by means of a receiver adapted for collecting a packet data stream over a plurality of links.
- the receiver is adapted to aggregate the stream distributed by the transmitter according to claim 13.
- the receiver is particularly characterized in that it is adapted to identify the segments distributed by the transmitter using an inverse multiplexer protocol, IM. It is further adapted to reassemble the segments into packets in order to aggregate the stream.
- the object of the present invention is finally solved by means of a system being adapted for distributing a packet data stream over a plurality of links.
- the stream received by a transmitter in the system is divided by the transmitter. It is further distributed via the links to a receiver in the system which aggregates the distributed stream.
- the system is particularly characterized in that:
- the transmitter is adapted to divide the packets in the stream into segments. It is further adapted to use an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links.
- IM inverse multiplexer protocol
- the receiver is adapted to identify the distributed segments using the IM. It is further adapted to reassemble the segments into packets in order to aggregate the stream .
- the present invention has the advantage that it allows for better traffic distribution amongst links.
- the traffic is balanced between the links due to the division of the packets and the dynamic distribution over several links.
- Links can also be gracefully removed or added with the present solution.
- the invention allows for a dynamic change in link bandwidth that will not introduce transmission errors. Error avoiding and fault handling is improved.
- a non-identical bandwidth of the participating links does not complicate the distribution of traffic over the links. This means that each link will automatically adapt to a change in speed without loss of data (which requires that the layer below supports error free bandwidth change).
- the links may have a different transmission delay.
- each link 26 may be operated at a bandwidth different from the other links (which means automatic adaptation).
- An additional advantage is that the present invention can be simply realized. Additional advantages are achieved by implementing one or several of the features of the dependent claims not mentioned above. This will be further explained below.
- Figure 1 illustrates high speed Ethernet traffic distributed over links with lower speed.
- Figure 2 illustrates the basic operation of segment interleaving according to the present invention.
- Figure 3 illustrates iGFP framing for the link inverse multiplexing
- Ethernet link layer
- Figure 4 illustrates definition of the iGFP payload length indication field.
- Figure 5 illustrates the iGFP control frames.
- Figure 6 illustrates the segment header for in-band signalling.
- Figure 7 illustrates the iGFP signalling frames with definition of tag data.
- Figure 8 illustrates segment header with in-band signalling ACK.
- Figure 9 illustrates iGFP signalling frames with definition of ACK-tag content.
- Figure 10 illustrates a simple exchange of messages via in-band signalling.
- Figure 1 1 illustrates iGFP signalling messages.
- Figure 12 illustrates message exchange after restart of control software.
- Figure 13 illustrates message exchange after receiving loss of framing.
- Figure 14 illustrates message exchange after a node has changed its ID.
- Figure 15 illustrates fault propagation of client layer down.
- Figure 16 illustrates loss of framing generating a remote defect indication.
- Figure 17 illustrates iGFP transports the segment header and segment data.
- Figure 18 illustrates segment header with segment identification.
- Figure 19 illustrates segment header field definitions.
- Figure 20 illustrates shutdown of a link initiated from one side.
- Figure 21 illustrates a block diagram for the link inverse multiplexing for
- Ethernet transmitter
- Figure 22 illustrates a block diagram for the link inverse multiplexing for
- FIG. 23 illustrates the method according to the present invention. DETAILED DESCRIPTION
- the present invention relates to a method for distributing a packet data stream over a plurality of links, wherein the stream received by a transmitter is divided by the transmitter and distributed via the links to a receiver. It also relates to a transmitter, a receiver and a system being adapted for the same purpose.
- Link in the following is used to represent radio links, such as a microwave radio links, optical links or copper based links.
- a link then corresponds to a physical channel or carrier of some kind.
- a transmitter and receiver adapted to perform said method described below are also disclosed in the following. In the following, the inventive method for distributing a packet data stream over a plurality of links will be described.
- FIG. 1 illustrates a system where high speed Ethernet traffic may be distributed over links with lower speed.
- the system comprises a high speed Ethernet link 20 (incoming and outgoing) fed into a transmitter 21 .
- the transmitter is connected via a plurality of links 26 (for instance microwave or optical) to a receiver 23.
- the transmitter and the receiver enable distribution of a packet data stream from the high speed Ethernet link. If radio links are used, they may operate on different speed.
- the packet data stream is distributed over the plurality of links 26, wherein the stream received by the transmitter 21 is divided by the transmitter and distributed via the links to the receiver 23.
- the receiver then aggregates the distributed stream.
- the method according to the present invention particularly comprises two steps, see figure 23, where:
- the transmitter divides 27 the packets 29,30,31 in the stream into segments 50 and uses an inverse multiplexer protocol, IM, to distribute these segments 50 dynamically in parallel over the links
- the receiver identifies 28 the distributed segments using the IM and reassembles the segments into packets in order to aggregate the stream.
- the method according to the present invention defines an inverse multiplexer (IM) protocol to effectively distribute high speed data traffic over the plurality of low speed links 26.
- the packets are treated as one flow.
- the Ethernet traffic is divided into segments and the segments are distributed over the links in a dynamic fashion. This means that the packets are segmented in fixed size segments which are being sent in parallel over the links to reduce latency.
- the IM links may provide different bit rates and the bit rate is also allowed to vary over time (e.g. radio links with adaptive modulation may change the rate depending on weather conditions).
- the transmitter 21 and receiver 23 are equipped with means 24,25 for enabling said division and reassembling of the packets in the stream.
- the means are named RL-IME group (Radio Link - Inverse Multiplexing for Ethernet), in which the packets are divided and reassembled. The divided packets are then distributed to the different RL-IME links 26, which connect the transmitter to the receiver.
- RL-IME group Radio Link - Inverse Multiplexing for Ethernet
- Figure 2 shows how packets are divided and reassembled in the system according to figure 1 .
- Packets a, b and c 29 have been completed and forwarded to higher layer.
- Packets d and e 30 are still under transmission, which means that they are partly re-assembled, partly under transmission and partly awaiting transmission while packets f, g, and h 31 are waiting to be selected for transmission.
- the figure also shows how segments may be interleaved across the links.
- the traffic distribution over the participating links should not be depending on the payload content (e.g. like the hashing algorithm(s) for the link aggregation method).
- the payload content e.g. like the hashing algorithm(s) for the link aggregation method.
- By splitting each Ethernet packet into segments 50 and then transmit several segments in parallel over the participating links will ensure that no link will be left idle while other links are saturated with data.
- a protocol is needed to be able to reassemble the segments in the receiver 23 correctly.
- the link level part handles link level in-band communication (such as in-band signalling) and synchronisation so that segment 50 boundaries can be identified as well as being responsible for detecting that a link is down.
- the group level part handles the identification of the segments as well as the start and end segment for the Ethernet packets.
- the IM may as an example use a compact framing procedure, iGFP, with fault propagation information. This may for instance be used for framing and synchronization during the packet distribution.
- iGFP is a compact GFP with fault propagation information included in the length field, which gives small framing overhead and better utilization.
- the RL-IME Link layer may be based on a proprietary iGFP (Generic Framing Procedure). iGFP is based on the standard ITU standardized GFP framing, see ITU-T G.7041/Y.1303.
- iGFP framing is illustrated in 3 while the definition of the iGFP payload length indication field (PLI) is shown in figure 4.
- PLI payload length indication field
- a total of 14 bits are reserved for length indication. This restricts the maximum possible data payload size to be 16383 bytes.
- PLI [15] is used for indication of "Client Layer Down (CLD)”.
- PLI [14] is used to indicate Remote Defect Indication (RDI), see figure 4.
- the cHEC[l5:0] is a CRC-16 according to the definition in ITU-T G.7041/Y.1303.
- the frame alignment for the iGFP is done according to ITU-T G.7041/Y.1303, with the option for correction of single bit errors in the header enabled.
- the iGFP header is scrambled according to ITU-T G.7041/Y.1303.
- the iGFP payload is scrambled according to ITU-T G.7041/Y.1303. Note that this also includes the signalling channel.
- the transmitter 21 and the receiver 23 signals may exchange information via in-band signalling on at least one link 26.
- Such in-band signalling may for instance also be used by the transmitter, where it informs the receiver on each link via in-band signalling that it will or will not accept segments 50 sent on that link.
- a solution for in-band signalling will be described in the following. The person skilled in the art however realizes that other solutions for in-band signalling may also be used.
- FIG. 5 illustrates the iGFP control frames (see also figure 3) while figure 6 illustrates the segment header for in-band signalling.
- the interpretation of Control B1 and Control B2 is shown in figure 6.
- the CRC-4 field is a checksum protection of the other 12 bits in the segment header. The CRC-4 detects all single bit errors in the control words.
- FIG 7 illustrates the iGFP signalling frames with definition of tag data. It is allowed to repeatedly send the same in band element an unlimited number of times without violating the protocol. The receiver should consider identical elements as duplicates. A sequence of duplicates is considered to be terminated once a different non error element has been received (an element with CRC-4 error may very well be the same element). The iGFP in band signalling is only sent in the absence of data.
- the 'control data' field of an acknowledge tag is shown in figure 8.
- Each ACK identifies the specific tag being acknowledged in addition to providing specific information for each type of tag being acknowledged.
- Figure 9 defines the B1 data field for the acknowledge tag.
- an in-band signalling example is shown. In-band signalling messages are exchanged through a simple protocol.
- Not receiving an ACK within a reasonable time may be due to that the ACK has been lost / corrupted or that the element to be acknowledged has been lost / corrupted. Since the in-band signalling has less priority than Ethernet data it may be that the data or ACK does not get the chance to be transmitted.
- the recommended action of a missing ACK is to retransmit the data (duplicates are allowed) in hope that the far end will send / re send the missing ACK. Due to the dependency of the "User Data" traffic load the in-band protocol definition does not specify a final timeout and leaves this to the implementation.
- the implementation should also have some means to identify the transmission status of an element to be sent (it makes no sense to expect a reply to an unsent element).
- the message is protected by a CRC-16 while the CRC-4 of the control bytes is needed in case the control bytes are used for hardware based control functions (e.g. synchronized switching between definitions of RL-IMEs).
- the CRC-4 is also the same for the segment header and the implementation for the termination of the segments 50 can then be shared with the termination of the control bytes.
- FIG 1 1 The definition of the messages, see figure 10, is shown in figure 1 1 .
- the messages defined in the figure can be sent at any time. As long as a participating link stays up it may normally be assumed that there is no need to re exchange the information.
- the message sender refers to the transmitter 21 while the message receiver refers to the receiver 23.
- Three cases where the information should be exchanged via in-band signalling are shown in figures 12,13 and 14.
- Figure 12 shows the case where Node #1 (the transmitter) restarts its controlling SW while the RL-IME hardware continues to operate.
- the control SW on Node #1 When the control SW on Node #1 is up it sends a request to Node #2 (the receiver) for its "Far End Id" followed by pushing its own “Far End Id” to Node #2. This message exchange will occur on all links.
- Figure 13 shows the case where one node gets a LOF (Loss of Framing) on a link 26. It must then refresh the "Far End Id". Since the LOF may be experienced only at one end it is no need to push the local "Far End Id”.
- Figure 14 shows a case where a node's Id for some reason is changed. It is then responsible for pushing a "Far End Id" to the other node.
- iGFP uses the standard GFP frame alignment, see ITU-T G.7041 Y.1303. This is a variation of the HEC-based (Header Error Checksum) frame delineation mechanism defined for Asynchronous Transfer Mode.
- HEC-based Header Error Checksum
- the RL- IME Link layer is in the Loss Of Frame (LOF) state whenever the state machine is in state HUNT (State in the framer where the alignment is not secured-searching for alignment) or PRESYNC (see ITU-T G.7041/Y.1303).
- LEF Loss Of Frame
- HUNT State in the framer where the alignment is not secured-searching for alignment
- PRESYNC see ITU-T G.7041/Y.1303
- the frames differ from the standardized frames in that they have the CLO (Client Layer Down) and RDI (Remote Defect Indication) bit encoding in the header. This precludes the predefinition of idle frames.
- the idle frames are inserted on the transmit side and discarded on the receive side and are thus invisible on the outside of the RL-IME Link layer.
- the RL-IME layer can propagate a client signal failure by setting the CLD bit in the iGFP header. This is a source adaptation to sink adaptation signalling scheme and the filtering criteria must be defined by the adaptations.
- RDI When the iGFP sink is in the LOF state, RDI shall be set in iGFP source. There is no filtering time. There is only a 1 :1 correspondence so that RDI at all times reflect the LOF state.
- the declaration of RDI from LOF is declared until RDI is received depends on the speed of the link. For a 2Mbit link it may take up to 8 msec. For a 100 Mbit link it may take up to 0.16 msec.
- LOF For an RDI to actually be transmitted, LOF must be present at the time when an iGFP header is generated. If the LOF is absent at this short time it will not be sampled before the next iGFP header is transmitted.
- Figure 16 illustrates loss of framing generating a RDI.
- the iGFP frames are sent according to the following priorities:
- the RL-IME group layer takes the incoming Ethernet frames, multiplexes them over a set of links 26 and reassembles them on the receive side into a stream of Ethernet frames.
- the multiplexing and de-multiplexing preserves the frame sequence, see figure 2.
- the inverse multiplexing is performed by dividing the incoming frame into segments 50. The segments are numbered and then transmitted on the links with spare capacity. On the receiving side the frame is reassembled by looking at the sequence number and when all segments are available the frame is forwarded.
- the Group layer is an extension of the iGPF protocol implemented by the addition of a segment header.
- the segment header is
- FIG. 16 The location of the segment header in the iGFP frame and the definition of the segment header are shown in figure 17.
- Figure 18 illustrates the segment header with segment identification and figure 19 illustrates segment header field definitions.
- the RL-IME Group protocol itself is not sensitive to segment 50 sizes, but a constant segment size eases the implementation of the reassembly function. A fixed size of 128 byte data in each segment has thus been chosen. This means that the iGFP frame including the iGFP segment header will be 128+6 bytes.
- the size of the distributed segments 50 of each packet 29,30,31 is fixed except for the last segment of said packet where the size may vary. This means that an exception to the segment size rule is done for the "LS" marked segment (see figures 18-19, which may be smaller so as to improve the link utilization. Thus the segment sizes will be 128 for all but the last segment, which may be 1 ...128. Segment size 0 is not allowed and will be interpreted as a control frame as PLI will be 2. Since there may be a dynamic variation of the link speed for the participating links it is necessary to not have a fixed multiplexing scheme at the transmitting side, but to rather send more data on links with higher speed than on the links with lower speed. The receiving side is for this reason able to pick up segments 50 and reassemble them according to the information in the segment header independently of how the far end transmitter implements its multiplexing scheme.
- the multiplexing is easiest implemented by selecting the transmitter with the lowest Tx FIFO fill-level (or below a threshold fill-level) for the next segment 50 (note: the Tx FIFO must be able to hold the entire segment to obtain parallelism). Once a segment has been assigned to a link, that segment will keep the Tx FIFO above the threshold level for some time while the transmitter empties the Tx FIFO. Once the fill-level goes below the threshold the link is once more competing for new segments.
- the transmitter 21 may, when a link 26 is added or removed, send a message via the in-band signalling to instruct the receiver to interrupt transmission of any new segments 50 via said link.
- a disable command will cause the local side to stop sending data segments in addition to sending RDI towards the far end to instruct it to stop sending data as well.
- This configuration management improves error avoiding and fault handling when a link is added or removed. Adding or removing of links can be thereby be done without loss of traffic.
- the configuration shown in figure 1 it will be possible to add or remove RL-IME links without traffic disturbance, with the prerequisite that the control system grants some time (implementation specific) between a link is disabled and the underlying communication channel is removed.
- the transmitter 21 and the receiver 23 may operate almost independently of each other.
- the only dependency is the LOF (Loss of Framing) to RDI (Remote Defect Indication) feedback as shown in figure 16.
- LOF Loss of Framing
- RDI Remote Defect Indication
- the IM uses buffering resources in the transmitter 21 and the receiver 23 to optimize the division and distribution of the packet data stream, which will be described in the following.
- Figure 21 shows a block diagram of an example of a RL-IME transmitter 21 . A person skilled in the art realizes that the transmitter may be realised in different ways.
- An iGFP function 32 provides the link layer and handles the in-band signalling.
- the in-band channels are accessed via the control and status interface.
- the iGFP function will automatically send in-band cells or idle cells when there are no segments 50 to be transmitted.
- the iGFP function Upon under-run of segment data, the iGFP function will complete the current segment by generating a zero data pattern. This is done to keep the far end in sync at the link level.
- Each of the iGFP instances reports its status to the Tx Scheduler. The status consists of: enabled/disabled, buffer fill-level, link status (LOF from iGFP Rx).
- the transmitter may fill the segments 50 to be distributed at a high speed into a transmission buffer 33 per link and wherein the link 26 empties the transmission buffers at a lower speed. This allows filling all the buffers in sequence, while the emptying is done in parallel.
- the transmitter may select a transmission buffer for filling based on the fill level of the transmission buffer. Once the level is below a certain threshold a complete segment is written to that link buffer.
- the size of the iGFP transmission buffer should be as little as possible, but it must be large enough to keep all links busy at all times during high traffic load.
- a Tx Scheduler 34 waits for a Tx Segmentation 35 to request a link 26 for transmission of a new segment 29,30,31 . Once such a request is detected, the Tx Scheduler uses the fill level of the links that are both enabled and that does not experience a LOF condition. In case no link is ready for the new segment 50 the Tx Scheduler waits until one is ready. Once a link is ready to accept a new segment (the entire segment) the Tx Scheduler connects this link to the Tx Segmentation via a multiplexer 51 and signals back to the Tx Segmentation that it may start its transfer.
- the Tx Segmentation 35 waits for the packet buffer 37 to indicate that it has selected a new packet for transmission (pkt ready).
- the Tx Segmentation requests the Tx Scheduler to find a suitable link and writes a segment 50 worth of data from the packet to the selected link.
- the "request link” and "segment write” is repeated until the entire packet has been sent.
- the transmitter 21 buffers packets to be distributed into the packet buffer 37.
- the packet buffer 37 operates as a store and forward buffer for the Ethernet packets received on an ETH interface 49. Packets may be picked from this buffer according to a priority selection algorithm.
- the packet buffer 37 implements flow control. The internals of this buffer functionality is not described here, but it may implement different algorithms for selecting packets for transmission (plain first-in-first-out, dropping old packets, selection based on priority, etc.)
- the receiver 23 is a little more advanced than the transmitter. This is due to that the variation in link speed and the variation in the link transmission delay must be handled here.
- Figure 22 shows a block diagram of the realisation of the RL-IME receiver. A person skilled in the art realizes that the receiver may be realised in different ways.
- An iGFP 38 function provides the link layer and handles the in-band signalling.
- the in-band channels are accessed via the control and status interface.
- the iGFP inspects the incoming traffic and terminates in-band signalling as well as idle cells.
- Other incoming iGFP frames are stripped for the iGFP header and the payload is forwarded to the link buffer together with start of frame (SOF), end of frame (EOF) and CRC-4 status for the segment 50 header.
- SOF start of frame
- EEF end of frame
- CRC-4 status for the segment 50 header.
- the data interface towards the link buffer is 8 bits wide and there is no flow control. This means that the iGFP may retain its synchronisation even if there is a "traffic jam" along the data path.
- Each of the iGFP instances reports its status both via registers and to the link buffer.
- the status consists of: LOF, RDI and CLD.
- the receiver fills the distributed segments 50 into a link buffer 39 per link 26.
- the size of this buffer is given by the maximum relative speed variation amongst the links and by the maximum delay variation amongst the links.
- the link buffer in the receiver consists of three parts: Store 40, data buffer storage 41 and retrieve 42.
- the storing part 40 writes two and two 8 bit data as an 18 bit value into the data buffer. The two added bits are to be interpreted as a tag to identify the corresponding 16 bits:
- the reason for using 16 bit data is to double the internal bandwidth when handling the data through the data path. Segments 50 with a CRC-4 error will not be stored.
- the data buffer storage 41 is an 18 bit wide dual port RAM. The size required is depending on the speed and delay variation amongst the participating links.
- An Rx Scheduler 43 consists of four parts: SNO hunt 44, Collect 45, data buffer 46 and Serialize 47.
- the SNO hunt 44 is responsible for finding the link that can provide the next segment 50 to be processed. When it finds a SNO that is less than the expected SNO it will issue a drop command to that link to reveal the next segment in that link's buffer. Once a link has been selected it waits until the "Collect" signals that the segment has been retrieved. Then a new search for SNO+1 is started. In case a SNO is missing the SNO hunt will go for the next one (see “reassembly buffer” for consequence).
- the collect part 45 reads the entire segment 50 from the selected link (still in 16 bits format) and stores it in the (small) data buffer. Together with the data (and tag) it is also stored information extracted from the FS and LS. This is done to be able to identify the start and the end of the packet to be reassembled later on. The filling of the data buffer is subject to flow control. Once a segment 50 has been completed a new request to the SNO hunt is being issued.
- the data buffer 46 is a very small buffer that may be filled at a rate twice the rate that it is being emptied. This gives the producer additional time to switch to another link and resume writing to the buffer before it is empty.
- the serialize part 47 converts the data back from 16 bit to 8 bit, adds start of frame (SOF) and end of frame (EOF) related to the Ethernet frame. Any missing segments 50 may cause one or more SOF/EOF to be missing.
- the receiver 21 collects the content of the distributed segments 50 in correct order and reassembles these into a reassembly buffer 48 as complete packets.
- the reassembly buffer identifies Ethernet frames to be the data between an SOF/EOF pair. However, transmission errors may cause frames to be shorter due to missing segments. In case the missing segments contain the FS/LS indication, this will result in missing SOF/EOF.
- the following SOF Start of Frame
- For a missing SOF the presence of data after an EOF indicates the start of a new Ethernet frame. A missing EOF followed by a missing SOF will concatenate Ethernet frames.
- the invention disclosed in this document describes a system (protocols and apparatus) to efficiently make use of the aggregated speed of several independent low speed links to transport high speed data.
- This document uses Ethernet and microwave radio links as specific examples.
- the theory applies in general to all cases where data flowing on a high speed link is to be transported over several independent links.
- the present invention has the advantage that it allows for better traffic distribution amongst links.
- the traffic is balanced between the links due to the division of the packets and the dynamic distribution over several links. Links can also be gracefully removed or added with the present solution.
- the invention allows for a dynamic change in link bandwidth that will not introduce transmission errors. Error avoiding and fault handling is improved. A non-identical bandwidth of the participating links does not complicate the distribution of traffic over the links. This means that each link will automatically adapt to a change in speed without loss of data (which requires that the layer below supports error free bandwidth change).
- the links may have a different transmission delay.
- each link 26 may be operated at a bandwidth different from the other links (which means automatic adaptation).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention relates to a method for distributing a packet data stream over a plurality of radio links, wherein the stream received by a transmitter is divided by the transmitter and distributed via the links to a receiver, which receiver aggregates the distributed stream. The method comprises the steps of: - the transmitter dividing (27) the packets in the stream into segments and using an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links, - the receiver identifying (28) the distributed segments using the IM and reassembles the segments into packets in order to aggregate the stream.
Description
METHOD FOR DISTRIBUTING DATA TRAFFIC OVER RADIO LINKS
TECHNICAL FIELD
The present invention relates to a method for distributing a packet data stream over a plurality of radio links, wherein the stream received by a transmitter is divided by the transmitter and distributed via the radio links to a receiver. It also relates to a transmitter, a receiver and a system being adapted for the same purpose.
BACKGROUND
The desired bandwidth for transport of Ethernet traffic over a microwave radio hop is larger than the speed provided by a single radio link. Hence, several radio links needs to be used in parallel to achieve a larger bandwidth. The same situation occurs also for other type of links, such as optical links, where data flowing on a high speed link needs to be transported over several links. Existing solutions implements link aggregation, by which Ethernet packets are distributed over the available links.
However, it is not allowed to re-order Ethernet packets belonging to a packet data stream. Hence, the link aggregation solution must send all Ethernet packets belonging to one stream over the same link. To select a link, a hashing algorithm therefore may be applied to certain information that is static in the Ethernet packet for a stream.
This result in the problem that, depending on the nature of the overall traffic, some links may be saturated with traffic while others may have idle time. To be able to alter the utilization profile of the links the hashing algorithm may have to be changed dynamically as the nature of the traffic changes over time.
WO2006/065142 relates to inverse multiplexing within telecommunication, and in particular to a versatile inverse multiplex protocol comprising associated state machines. It also relates to a method for using this protocol allowing a high speed link to be distributed via an inverse multiplexer into a
number of lower speed connections in pursuance to G.704 recommendations and using Sa bits. With this solution, the packet data stream is divided into bits which are distributed on a plurality of links. The problem with this solution is that it can only be used on low speed links having the same bandwidth. SUMMARY
The object of the present invention is therefore to effectively distribute high speed data traffic over a number of low speed links with different bandwidth.
The object is solved by means of a method for distributing a packet data stream over a plurality of radio links. The stream received by a transmitter is divided by the transmitter and distributed via the links to a receiver. The receiver aggregates the distributed stream. The method comprises the steps of:
1 . The transmitter divides the packets in the stream into segments. It further uses an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links.,
2. The receiver identifies the distributed segments using the IM. It further reassembles the segments into packets in order to aggregate the stream.
The object of the present invention is also solved by means of a transmitter adapted for distributing a packet data stream over a plurality of links. The transmitter is adapted to distribute the stream via the links to the receiver according to claim 14. The transmitter is particularly characterized in that it is adapted to divide the packets in the stream into segments. It is further adapted to use an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links. The object of the present invention is also solved by means of a receiver adapted for collecting a packet data stream over a plurality of links. The receiver is adapted to aggregate the stream distributed by the transmitter according to claim 13. The receiver is particularly characterized in that it is
adapted to identify the segments distributed by the transmitter using an inverse multiplexer protocol, IM. It is further adapted to reassemble the segments into packets in order to aggregate the stream.
The object of the present invention is finally solved by means of a system being adapted for distributing a packet data stream over a plurality of links. The stream received by a transmitter in the system is divided by the transmitter. It is further distributed via the links to a receiver in the system which aggregates the distributed stream. The system is particularly characterized in that: The transmitter is adapted to divide the packets in the stream into segments. It is further adapted to use an inverse multiplexer protocol, IM, to distribute these segments dynamically in parallel over the links.
The receiver is adapted to identify the distributed segments using the IM. It is further adapted to reassemble the segments into packets in order to aggregate the stream .
The present invention has the advantage that it allows for better traffic distribution amongst links. The traffic is balanced between the links due to the division of the packets and the dynamic distribution over several links. Links can also be gracefully removed or added with the present solution. Moreover, the invention allows for a dynamic change in link bandwidth that will not introduce transmission errors. Error avoiding and fault handling is improved. A non-identical bandwidth of the participating links does not complicate the distribution of traffic over the links. This means that each link will automatically adapt to a change in speed without loss of data (which requires that the layer below supports error free bandwidth change). The links may have a different transmission delay. Moreover, with the present invention each link 26 may be operated at a bandwidth different from the other links (which means automatic adaptation). An additional advantage is that the present invention can be simply realized.
Additional advantages are achieved by implementing one or several of the features of the dependent claims not mentioned above. This will be further explained below.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will be described in greater detail in the following, with reference to the embodiments that are shown in the attached drawings, in which:
Figure 1 illustrates high speed Ethernet traffic distributed over links with lower speed. Figure 2 illustrates the basic operation of segment interleaving according to the present invention.
Figure 3 illustrates iGFP framing for the link inverse multiplexing for
Ethernet link layer.
Figure 4 illustrates definition of the iGFP payload length indication field. Figure 5 illustrates the iGFP control frames.
Figure 6 illustrates the segment header for in-band signalling.
Figure 7 illustrates the iGFP signalling frames with definition of tag data.
Figure 8 illustrates segment header with in-band signalling ACK.
Figure 9 illustrates iGFP signalling frames with definition of ACK-tag content.
Figure 10 illustrates a simple exchange of messages via in-band signalling.
Figure 1 1 illustrates iGFP signalling messages.
Figure 12 illustrates message exchange after restart of control software.
Figure 13 illustrates message exchange after receiving loss of framing.
Figure 14 illustrates message exchange after a node has changed its ID.
Figure 15 illustrates fault propagation of client layer down.
Figure 16 illustrates loss of framing generating a remote defect indication. Figure 17 illustrates iGFP transports the segment header and segment data.
Figure 18 illustrates segment header with segment identification.
Figure 19 illustrates segment header field definitions.
Figure 20 illustrates shutdown of a link initiated from one side. Figure 21 illustrates a block diagram for the link inverse multiplexing for
Ethernet transmitter.
Figure 22 illustrates a block diagram for the link inverse multiplexing for
Ethernet receiver.
Figure 23 illustrates the method according to the present invention. DETAILED DESCRIPTION
The present invention relates to a method for distributing a packet data stream over a plurality of links, wherein the stream received by a transmitter is divided by the transmitter and distributed via the links to a receiver. It also relates to a transmitter, a receiver and a system being adapted for the same purpose. Link in the following is used to represent radio links, such as a microwave radio links, optical links or copper based links. A link then corresponds to a physical channel or carrier of some kind. A person skilled in the art would realize that a transmitter and receiver adapted to perform said method described below are also disclosed in the following. In the following, the inventive method for distributing a packet data stream
over a plurality of links will be described. The method is intended to solve the problem that for Ethernet traffic, the packets are not allowed to re-order, which results in an unbalanced distribution over a plurality of links with lower speed and different bandwidth. Figure 1 illustrates a system where high speed Ethernet traffic may be distributed over links with lower speed. The system comprises a high speed Ethernet link 20 (incoming and outgoing) fed into a transmitter 21 . The transmitter is connected via a plurality of links 26 (for instance microwave or optical) to a receiver 23. The transmitter and the receiver enable distribution of a packet data stream from the high speed Ethernet link. If radio links are used, they may operate on different speed.
In the present invention, the packet data stream is distributed over the plurality of links 26, wherein the stream received by the transmitter 21 is divided by the transmitter and distributed via the links to the receiver 23. The receiver then aggregates the distributed stream.
The method according to the present invention particularly comprises two steps, see figure 23, where:
1 . 1 . The transmitter divides 27 the packets 29,30,31 in the stream into segments 50 and uses an inverse multiplexer protocol, IM, to distribute these segments 50 dynamically in parallel over the links
26.
2. The receiver identifies 28 the distributed segments using the IM and reassembles the segments into packets in order to aggregate the stream. The method according to the present invention defines an inverse multiplexer (IM) protocol to effectively distribute high speed data traffic over the plurality of low speed links 26. The packets are treated as one flow. The Ethernet traffic is divided into segments and the segments are distributed over the
links in a dynamic fashion. This means that the packets are segmented in fixed size segments which are being sent in parallel over the links to reduce latency. The IM links may provide different bit rates and the bit rate is also allowed to vary over time (e.g. radio links with adaptive modulation may change the rate depending on weather conditions).
In order to enable the procedure according to the present invention, the transmitter 21 and receiver 23 are equipped with means 24,25 for enabling said division and reassembling of the packets in the stream. The means are named RL-IME group (Radio Link - Inverse Multiplexing for Ethernet), in which the packets are divided and reassembled. The divided packets are then distributed to the different RL-IME links 26, which connect the transmitter to the receiver. The details of the transmitter and the receiver will be described in the following.
Figure 2 shows how packets are divided and reassembled in the system according to figure 1 . Packets a, b and c 29 have been completed and forwarded to higher layer. Packets d and e 30 are still under transmission, which means that they are partly re-assembled, partly under transmission and partly awaiting transmission while packets f, g, and h 31 are waiting to be selected for transmission. The figure also shows how segments may be interleaved across the links.
To be able to utilise all links 26 equally, the traffic distribution over the participating links should not be depending on the payload content (e.g. like the hashing algorithm(s) for the link aggregation method). By splitting each Ethernet packet into segments 50 and then transmit several segments in parallel over the participating links will ensure that no link will be left idle while other links are saturated with data. For such segmentation a protocol is needed to be able to reassemble the segments in the receiver 23 correctly.
In the following, an example protocol for both the link level and the group level will be described. The link level part handles link level in-band communication (such as in-band signalling) and synchronisation so that segment 50 boundaries can be identified as well as being responsible for detecting that a link is down. The group level part handles the identification of the segments as well as the start and end segment for the Ethernet packets.
A person skilled in the art would realize that other protocols may also be used to handle the RL-IME (Radio Link Inverse Multiplexing for Ethernet) division and reassembling according to the present invention. The IM may as an example use a compact framing procedure, iGFP, with fault propagation information. This may for instance be used for framing and synchronization during the packet distribution. iGFP is a compact GFP with fault propagation information included in the length field, which gives small framing overhead and better utilization. The RL-IME Link layer may be based on a proprietary iGFP (Generic Framing Procedure). iGFP is based on the standard ITU standardized GFP framing, see ITU-T G.7041/Y.1303. iGFP framing is illustrated in 3 while the definition of the iGFP payload length indication field (PLI) is shown in figure 4. A total of 14 bits are reserved for length indication. This restricts the maximum possible data payload size to be 16383 bytes. Of the two remaining bits, PLI [15] is used for indication of "Client Layer Down (CLD)". PLI [14] is used to indicate Remote Defect Indication (RDI), see figure 4.
The cHEC[l5:0] is a CRC-16 according to the definition in ITU-T G.7041/Y.1303. The frame alignment for the iGFP is done according to ITU-T G.7041/Y.1303, with the option for correction of single bit errors in the header enabled. The iGFP header is scrambled according to ITU-T G.7041/Y.1303. The iGFP payload is scrambled according to ITU-T G.7041/Y.1303. Note that this also includes the signalling channel.
The transmitter 21 and the receiver 23 signals may exchange information via in-band signalling on at least one link 26. Such in-band signalling may for instance also be used by the transmitter, where it informs the receiver on each link via in-band signalling that it will or will not accept segments 50 sent on that link. A solution for in-band signalling will be described in the following. The person skilled in the art however realizes that other solutions for in-band signalling may also be used.
In order to enable in-band signalling with iGFP, a special interpretation of the data field of the iGFP frame is used when the PLI = 2. This cannot occur for normal data frames as 64 bytes is the smallest Ethernet frame and a segmented frame will always contain minimum 3 Bytes (header + 1 Byte) in the final segment 50. Figure 5 illustrates the iGFP control frames (see also figure 3) while figure 6 illustrates the segment header for in-band signalling. The interpretation of Control B1 and Control B2 is shown in figure 6. The CRC-4 field is a checksum protection of the other 12 bits in the segment header. The CRC-4 detects all single bit errors in the control words.
By using 4 bits for tag and 8 bits for tag data, see figure 6, it is possible to transfer 16 unique bytes of in band information between the two end points. Figure 7 illustrates the iGFP signalling frames with definition of tag data. It is allowed to repeatedly send the same in band element an unlimited number of times without violating the protocol. The receiver should consider identical elements as duplicates. A sequence of duplicates is considered to be terminated once a different non error element has been received (an element with CRC-4 error may very well be the same element). The iGFP in band signalling is only sent in the absence of data.
The 'control data' field of an acknowledge tag is shown in figure 8. Each ACK identifies the specific tag being acknowledged in addition to providing specific information for each type of tag being acknowledged. Figure 9 defines the B1 data field for the acknowledge tag. In figure 10, an in-band signalling
example is shown. In-band signalling messages are exchanged through a simple protocol.
Not receiving an ACK within a reasonable time may be due to that the ACK has been lost / corrupted or that the element to be acknowledged has been lost / corrupted. Since the in-band signalling has less priority than Ethernet data it may be that the data or ACK does not get the chance to be transmitted. The recommended action of a missing ACK is to retransmit the data (duplicates are allowed) in hope that the far end will send / re send the missing ACK. Due to the dependency of the "User Data" traffic load the in-band protocol definition does not specify a final timeout and leaves this to the implementation. The implementation should also have some means to identify the transmission status of an element to be sent (it makes no sense to expect a reply to an unsent element).
The format of the messages, see figure 10, is: - Msg-id: identifies the message (one byte)
- Msg-D: message data (max 253 bytes)
- Msg-CS: CRC-16 (the two last bytes)
The message is protected by a CRC-16 while the CRC-4 of the control bytes is needed in case the control bytes are used for hardware based control functions (e.g. synchronized switching between definitions of RL-IMEs). The CRC-4 is also the same for the segment header and the implementation for the termination of the segments 50 can then be shared with the termination of the control bytes.
The definition of the messages, see figure 10, is shown in figure 1 1 . The messages defined in the figure can be sent at any time. As long as a participating link stays up it may normally be assumed that there is no need to re exchange the information. The message sender refers to the transmitter 21 while the message receiver refers to the receiver 23.
Three cases where the information should be exchanged via in-band signalling are shown in figures 12,13 and 14. Figure 12 shows the case where Node #1 (the transmitter) restarts its controlling SW while the RL-IME hardware continues to operate. When the control SW on Node #1 is up it sends a request to Node #2 (the receiver) for its "Far End Id" followed by pushing its own "Far End Id" to Node #2. This message exchange will occur on all links. Figure 13 shows the case where one node gets a LOF (Loss of Framing) on a link 26. It must then refresh the "Far End Id". Since the LOF may be experienced only at one end it is no need to push the local "Far End Id". Figure 14 shows a case where a node's Id for some reason is changed. It is then responsible for pushing a "Far End Id" to the other node. iGFP uses the standard GFP frame alignment, see ITU-T G.7041 Y.1303. This is a variation of the HEC-based (Header Error Checksum) frame delineation mechanism defined for Asynchronous Transfer Mode. The RL- IME Link layer is in the Loss Of Frame (LOF) state whenever the state machine is in state HUNT (State in the framer where the alignment is not secured-searching for alignment) or PRESYNC (see ITU-T G.7041/Y.1303). This means that a single HEC error (after the single bit error correction) will cause LOF. The declaration of LOF depends on the speed of the link. For a 2Mbit link it may take up to 8 msec. For a 100 Mbit link it may take up to 0.16 msec. iGFP uses idle frames for rate adaptation, see ITU-T G.7041/Y.1303. The frames differ from the standardized frames in that they have the CLO (Client Layer Down) and RDI (Remote Defect Indication) bit encoding in the header. This precludes the predefinition of idle frames. The idle frames are inserted on the transmit side and discarded on the receive side and are thus invisible on the outside of the RL-IME Link layer.
The RL-IME layer can propagate a client signal failure by setting the CLD bit in the iGFP header. This is a source adaptation to sink adaptation signalling
scheme and the filtering criteria must be defined by the adaptations. Figure
15 illustrates fault propagation of CLD.
When the iGFP sink is in the LOF state, RDI shall be set in iGFP source. There is no filtering time. There is only a 1 :1 correspondence so that RDI at all times reflect the LOF state. The declaration of RDI from LOF is declared until RDI is received depends on the speed of the link. For a 2Mbit link it may take up to 8 msec. For a 100 Mbit link it may take up to 0.16 msec. For an RDI to actually be transmitted, LOF must be present at the time when an iGFP header is generated. If the LOF is absent at this short time it will not be sampled before the next iGFP header is transmitted. Figure 16 illustrates loss of framing generating a RDI.
The iGFP frames are sent according to the following priorities:
1 . User Data frames.
2. Signalling frames 3. Idle frames
The RL-IME group layer takes the incoming Ethernet frames, multiplexes them over a set of links 26 and reassembles them on the receive side into a stream of Ethernet frames. The multiplexing and de-multiplexing preserves the frame sequence, see figure 2. The inverse multiplexing is performed by dividing the incoming frame into segments 50. The segments are numbered and then transmitted on the links with spare capacity. On the receiving side the frame is reassembled by looking at the sequence number and when all segments are available the frame is forwarded.
Since the link capacities and delays vary, the segments 50 may be received out of order. The Group layer is an extension of the iGPF protocol implemented by the addition of a segment header. The segment header is
16 bits. The location of the segment header in the iGFP frame and the definition of the segment header are shown in figure 17. Figure 18 illustrates
the segment header with segment identification and figure 19 illustrates segment header field definitions.
The RL-IME Group protocol itself is not sensitive to segment 50 sizes, but a constant segment size eases the implementation of the reassembly function. A fixed size of 128 byte data in each segment has thus been chosen. This means that the iGFP frame including the iGFP segment header will be 128+6 bytes.
The size of the distributed segments 50 of each packet 29,30,31 is fixed except for the last segment of said packet where the size may vary. This means that an exception to the segment size rule is done for the "LS" marked segment (see figures 18-19, which may be smaller so as to improve the link utilization. Thus the segment sizes will be 128 for all but the last segment, which may be 1 ...128. Segment size 0 is not allowed and will be interpreted as a control frame as PLI will be 2. Since there may be a dynamic variation of the link speed for the participating links it is necessary to not have a fixed multiplexing scheme at the transmitting side, but to rather send more data on links with higher speed than on the links with lower speed. The receiving side is for this reason able to pick up segments 50 and reassemble them according to the information in the segment header independently of how the far end transmitter implements its multiplexing scheme.
For the transmitter the multiplexing is easiest implemented by selecting the transmitter with the lowest Tx FIFO fill-level (or below a threshold fill-level) for the next segment 50 (note: the Tx FIFO must be able to hold the entire segment to obtain parallelism). Once a segment has been assigned to a link, that segment will keep the Tx FIFO above the threshold level for some time while the transmitter empties the Tx FIFO. Once the fill-level goes below the threshold the link is once more competing for new segments.
For configuration management, the transmitter 21 may, when a link 26 is
added or removed, send a message via the in-band signalling to instruct the receiver to interrupt transmission of any new segments 50 via said link. For instance, a disable command will cause the local side to stop sending data segments in addition to sending RDI towards the far end to instruct it to stop sending data as well. Figure 20 shows the disable sequence and it also shows that it is not required to initiate the disable action simultaneously on both sides. Note that node #1 has no means to decide whether the incoming idle is due to the outgoing RDI=1 or not. Hence, the link can/should be removed after a reasonable time regardless of the status of the incoming stream. This time will be implementation dependant.
This configuration management improves error avoiding and fault handling when a link is added or removed. Adding or removing of links can be thereby be done without loss of traffic. In particular, for the configuration shown in figure 1 it will be possible to add or remove RL-IME links without traffic disturbance, with the prerequisite that the control system grants some time (implementation specific) between a link is disabled and the underlying communication channel is removed.
For a realisation of the RL-IME protocol, the transmitter 21 and the receiver 23 may operate almost independently of each other. The only dependency is the LOF (Loss of Framing) to RDI (Remote Defect Indication) feedback as shown in figure 16. For the transmitter, this means that it should stop sending data and set RDI=1 upon LOF detected by the receiver. The IM uses buffering resources in the transmitter 21 and the receiver 23 to optimize the division and distribution of the packet data stream, which will be described in the following. Figure 21 shows a block diagram of an example of a RL-IME transmitter 21 . A person skilled in the art realizes that the transmitter may be realised in different ways.
An iGFP function 32 provides the link layer and handles the in-band signalling. The in-band channels are accessed via the control and status interface. The iGFP function will automatically send in-band cells or idle cells
when there are no segments 50 to be transmitted. Upon under-run of segment data, the iGFP function will complete the current segment by generating a zero data pattern. This is done to keep the far end in sync at the link level. Each of the iGFP instances reports its status to the Tx Scheduler. The status consists of: enabled/disabled, buffer fill-level, link status (LOF from iGFP Rx).
The transmitter may fill the segments 50 to be distributed at a high speed into a transmission buffer 33 per link and wherein the link 26 empties the transmission buffers at a lower speed. This allows filling all the buffers in sequence, while the emptying is done in parallel. The transmitter may select a transmission buffer for filling based on the fill level of the transmission buffer. Once the level is below a certain threshold a complete segment is written to that link buffer. The size of the iGFP transmission buffer should be as little as possible, but it must be large enough to keep all links busy at all times during high traffic load.
A Tx Scheduler 34 waits for a Tx Segmentation 35 to request a link 26 for transmission of a new segment 29,30,31 . Once such a request is detected, the Tx Scheduler uses the fill level of the links that are both enabled and that does not experience a LOF condition. In case no link is ready for the new segment 50 the Tx Scheduler waits until one is ready. Once a link is ready to accept a new segment (the entire segment) the Tx Scheduler connects this link to the Tx Segmentation via a multiplexer 51 and signals back to the Tx Segmentation that it may start its transfer.
The Tx Segmentation 35 waits for the packet buffer 37 to indicate that it has selected a new packet for transmission (pkt ready). The Tx Segmentation requests the Tx Scheduler to find a suitable link and writes a segment 50 worth of data from the packet to the selected link. The "request link" and "segment write" is repeated until the entire packet has been sent.
The transmitter 21 buffers packets to be distributed into the packet buffer 37. The packet buffer 37 operates as a store and forward buffer for the Ethernet packets received on an ETH interface 49. Packets may be picked from this buffer according to a priority selection algorithm. The packet buffer 37 implements flow control. The internals of this buffer functionality is not described here, but it may implement different algorithms for selecting packets for transmission (plain first-in-first-out, dropping old packets, selection based on priority, etc.)
The receiver 23 is a little more advanced than the transmitter. This is due to that the variation in link speed and the variation in the link transmission delay must be handled here. Figure 22 shows a block diagram of the realisation of the RL-IME receiver. A person skilled in the art realizes that the receiver may be realised in different ways.
An iGFP 38 function provides the link layer and handles the in-band signalling. The in-band channels are accessed via the control and status interface. The iGFP inspects the incoming traffic and terminates in-band signalling as well as idle cells. Other incoming iGFP frames are stripped for the iGFP header and the payload is forwarded to the link buffer together with start of frame (SOF), end of frame (EOF) and CRC-4 status for the segment 50 header. The data interface towards the link buffer is 8 bits wide and there is no flow control. This means that the iGFP may retain its synchronisation even if there is a "traffic jam" along the data path. Each of the iGFP instances reports its status both via registers and to the link buffer. The status consists of: LOF, RDI and CLD. The receiver fills the distributed segments 50 into a link buffer 39 per link 26. The size of this buffer is given by the maximum relative speed variation amongst the links and by the maximum delay variation amongst the links. The link buffer in the receiver consists of three parts: Store 40, data buffer storage 41 and Retrieve 42. The storing part 40 writes two and two 8 bit data
as an 18 bit value into the data buffer. The two added bits are to be interpreted as a tag to identify the corresponding 16 bits:
- TAG: Segment header
- TAG: Segment data body - TAG: Segment end, odd bytes
- TAG: Segment end, even bytes
The reason for using 16 bit data is to double the internal bandwidth when handling the data through the data path. Segments 50 with a CRC-4 error will not be stored. The data buffer storage 41 is an 18 bit wide dual port RAM. The size required is depending on the speed and delay variation amongst the participating links.
Due to no flow control when storing into the data buffer 41 some parts of the segments 50 may be missing due to overflow or entire segments may be missing due to detected CRC-4 errors. This will be handled by reading the tags stored with the data. Note that the buffer should be scaled to handle all normal traffic variation (speed/delay) and that overflows will be due to bit errors not detected by the CRC-4 (out of sequence SNO may temporarily slow down the Rx Scheduler). SNO refers to sequence number. The "Retrieve" 42 will search for a segment 50 header, present its value to the Rx Scheduler and then await further instructions:
- In case the SNO presented is less than the currently expected SNO it will receive a "drop" command.
- In case the SNO presented is larger than the currently expected SNO nothing will happen.
- In case the SNO matches the currently expected SNO read commands will be issued until the entire segment 50 has been read.
Any tag other than "segment data body" will end the segment. An Rx Scheduler 43 consists of four parts: SNO hunt 44, Collect 45, data buffer 46 and Serialize 47.
The SNO hunt 44 is responsible for finding the link that can provide the next segment 50 to be processed. When it finds a SNO that is less than the expected SNO it will issue a drop command to that link to reveal the next segment in that link's buffer. Once a link has been selected it waits until the "Collect" signals that the segment has been retrieved. Then a new search for SNO+1 is started. In case a SNO is missing the SNO hunt will go for the next one (see "reassembly buffer" for consequence).
The collect part 45 reads the entire segment 50 from the selected link (still in 16 bits format) and stores it in the (small) data buffer. Together with the data (and tag) it is also stored information extracted from the FS and LS. This is done to be able to identify the start and the end of the packet to be reassembled later on. The filling of the data buffer is subject to flow control. Once a segment 50 has been completed a new request to the SNO hunt is being issued.
The data buffer 46 is a very small buffer that may be filled at a rate twice the rate that it is being emptied. This gives the producer additional time to switch to another link and resume writing to the buffer before it is empty. The serialize part 47 converts the data back from 16 bit to 8 bit, adds start of frame (SOF) and end of frame (EOF) related to the Ethernet frame. Any missing segments 50 may cause one or more SOF/EOF to be missing.
The receiver 21 collects the content of the distributed segments 50 in correct order and reassembles these into a reassembly buffer 48 as complete
packets. The reassembly buffer identifies Ethernet frames to be the data between an SOF/EOF pair. However, transmission errors may cause frames to be shorter due to missing segments. In case the missing segments contain the FS/LS indication, this will result in missing SOF/EOF. For a missing EOF, the following SOF (Start of Frame) indicates the completion of the previous Ethernet frame. For a missing SOF the presence of data after an EOF indicates the start of a new Ethernet frame. A missing EOF followed by a missing SOF will concatenate Ethernet frames. For such a condition a maximum reassembled frame size should be enforced. The rationale for sending the obviously bogus Ethernet frames is to reduce the complexity of this implementation and let the specialised Ethernet hardware connected to the ETH interface 49 do the error handling and the statistics. It is also assumed that sending the bogus Ethernet frames will not cause any extra load compared to the situation where the frames have been correctly received .
The invention is not to be regarded as being limited to the examples described above. A number of additional variants and modifications are possible within the scope of the subsequent patent claims. For instance, it may be used in different links, such as microwave radio links, copper based links or optical links. Any type of link is possible.
The invention disclosed in this document describes a system (protocols and apparatus) to efficiently make use of the aggregated speed of several independent low speed links to transport high speed data. This document uses Ethernet and microwave radio links as specific examples. However, the theory applies in general to all cases where data flowing on a high speed link is to be transported over several independent links.
The present invention has the advantage that it allows for better traffic distribution amongst links. The traffic is balanced between the links due to
the division of the packets and the dynamic distribution over several links. Links can also be gracefully removed or added with the present solution.
Moreover, the invention allows for a dynamic change in link bandwidth that will not introduce transmission errors. Error avoiding and fault handling is improved. A non-identical bandwidth of the participating links does not complicate the distribution of traffic over the links. This means that each link will automatically adapt to a change in speed without loss of data (which requires that the layer below supports error free bandwidth change). The links may have a different transmission delay. Moreover, with the present invention each link 26 may be operated at a bandwidth different from the other links (which means automatic adaptation). An additional advantage is that the present invention can be simply realized.
Claims
1 . Method for distributing a packet data stream over a plurality of links (26), wherein the stream received by a transmitter (21 ) is divided by the transmitter (21 ) and distributed via the links (26) to a receiver (23), which receiver (23) aggregates the distributed stream, the method comprising the steps of:
- the transmitter (21 ) dividing (27) the packets (29,30,31 ) in the stream into segments (50) and using an inverse multiplexer protocol, IM, to distribute these segments (50) dynamically in parallel over the links (26), - the receiver (23) identifying (28) the distributed segments (50) using the IM and reassembles the segments (50) into packets (29,30,31 ) in order to aggregate the stream.
2. Method according to any of the preceding claims wherein the transmitter (21 ) and the receiver (23) exchange information via in-band signalling on at least one link (26).
3. Method according to claim 2 wherein when a link (26) is added or removed, the transmitter (21 ) sends a message via the in-band signalling to instruct the receiver (23) to interrupt transmission of any new segments (50) via said link (26). 4. Method according to any of the claims 2 - 3 wherein the transmitter (21 ) informs the receiver (23) on each link (26) via in-band signalling that it will or will not accept segments (50) sent on that link (26).
5. Method according to any of the preceding claims wherein the IM uses compact framing procedure, iGFP, with fault propagation. 6. Method according to claim 5 wherein a compact framing procedure, iGFP, function (32) in the transmitter and the receiver providing at least one link and handling in-band signalling.
7. Method according to any of the preceding claims wherein the transmitter (21 ) buffers packets (29,30,31 ) to be distributed into a packet buffer (37) in the transmitter (21 ).
8. Method according to any of the preceding claims wherein the transmitter (21 ) fills the segments (50) to be distributed at a high speed into a transmission buffer (33)) per link (26) in the transmitter (21 ) and wherein the link (26) empties the transmission buffers (33) at a lower speed.
9. Method according to any of the preceding claims wherein the transmitter (21 ) comprises a scheduler (34) connecting a link ready for new segments via a multiplexer (51 ) to a segmentation function (35) in the transmitter (21 ), the scheduler further signalling back to the segmentation function (35) that it may start its transfer.
10. Method according to claim 9 wherein the segmentation function (35) writes a segment (50) to the connected link. 1 1 . Method according to any of the preceding claims wherein the receiver (23) fills the distributed segments (50) into a link buffer (29) per link (26) in the receiver (23).
12. Method according to claim 1 1 wherein the receiver (23) collects the content of the distributed segments (50) in correct order and reassembles these into a reassembly buffer (48) in the receiver (23) as complete packets (29,30,31 ).
13. Method according to any of the preceding claims wherein the receiver (23) comprises a scheduler (43) with at least a serial number, SNO, hunt (44), a collect part (45) and a data buffer (46), the receiver performing the steps of:
- the SNO hunt (44) finding a link (26) that can provide the next segment (50) to be processed,
- the collect part (45) reading an entire segment (50). from the found link (26) and storing it in the data buffer (46).
14. Method according to any of the preceding claims wherein the size of the distributed segments (50) of each packet (29,30,31) is fixed except for the last segment (50) of said packet (29,30,31) where the size varies.
15. Transmitter (21) adapted for distributing a packet data stream over a plurality of links (26), wherein the transmitter (21) is adapted to distribute the stream via the links (26) to the receiver (23) according to claim 14, characterized in that the transmitter (21) being adapted to divide (27) the packets (29,30,31) in the stream into segments (50) and use an inverse multiplexer protocol, IM, to distribute these segments (50) dynamically in parallel over the links (26).
16. Transmitter according to claims 15 wherein the transmitter (21) comprises a packet buffer (37) being adapted to buffer packets (29,30,31) to be distributed.
17. Transmitter according to any of the claims 15 - 16 wherein the transmitter (21) comprises a transmission buffer (22) per link, which is adapted to receive at a high speed segments (50) to be distributed.
18. Transmitter according to any of the claims 15 - 17 wherein the transmitter (21) comprises a scheduler (34) adapted to connect a link ready for new segments via a multiplexer (51) to a segmentation function (35) in the transmitter (21), the scheduler further being adapted to signal back to the segmentation function (35) that it may start its transfer. 9. Transmitter according to claim 18 wherein the segmentation function (35) is adapted to write a segment (50) to the connected link.
20. Transmitter according to any of the claims 15 - 19 wherein the transmitter (21 ) comprises a compact framing procedure, iGFP, with fault propagation function (32) that is adapted to provide the link and handle in-band signalling. 21 . Receiver (23) adapted for collecting a packet data stream over a plurality of links (26), wherein the receiver (23) is adapted to aggregate the stream distributed by the transmitter (21 ) according to claim 13, characterized in that the receiver (23) is adapted to identify (28) the segments (50) distributed by the transmitter (21 ) using an inverse multiplexer protocol, IM, and reassemble the segments (50) into packets (29,30,31 ) in order to aggregate the stream.
22. Receiver according to claim 21 wherein the receiver (23) comprises a link buffer (29) per link (26), which is adapted to receive the distributed segments (50).
23. Receiver according to any of the claims 21 - 22 wherein the receiver (23) comprises a reassembly buffer (48), which is adapted to receive the reassembled segments (50) as complete packets (29,30,31 ).
24. Receiver according to any of the claims 21 - 23 wherein the receiver (23) comprises a compact framing procedure, iGFP, with fault propagation function 32 that is adapted to provide the link and handle in-band signalling.
25. Receiver according to any of the claims wherein the receiver (23) comprises a scheduler (43) with at least a serial number, SNO, hunt (44), a collect part (45) and a data buffer (46): - the SNO hunt (44) being adapted to find a link (26) that can provide the next segment (50) to be processed,
- the collect part (45) being adapted to read an entire segment (50) from the found link (26) and store it in the data buffer (46).
26. System being adapted for distributing a packet data stream over a plurality of links (26), wherein the stream received by a transmitter (21 ) in the system is divided by the transmitter (21 ) and distributed via the links (26) to a receiver (23) in the system, which receiver (23) aggregates the distributed stream, characterized in that the transmitter (21 ) being adapted to divide (27) the packets (29,30,31 ) in the stream into segments (50) and use an inverse multiplexer protocol, IM, to distribute these segments (50) dynamically in parallel over the links (26), the receiver (23) being adapted to identify (28) the distributed segments (50) using the IM and reassemble the segments (50) into packets (29,30,31 ) in order to aggregate the stream.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2009/061265 WO2011026508A1 (en) | 2009-09-01 | 2009-09-01 | Method for distributing data traffic over radio links |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2009/061265 WO2011026508A1 (en) | 2009-09-01 | 2009-09-01 | Method for distributing data traffic over radio links |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011026508A1 true WO2011026508A1 (en) | 2011-03-10 |
Family
ID=41527846
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2009/061265 WO2011026508A1 (en) | 2009-09-01 | 2009-09-01 | Method for distributing data traffic over radio links |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2011026508A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017005324A1 (en) * | 2015-07-08 | 2017-01-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Wireless communication arrangement with two devices communicating with each other via a radio link in which a common radio interface constitutes a communication protocol for data streams arranged to support a segmentation of data frames |
CN111264079A (en) * | 2019-11-14 | 2020-06-09 | 深圳市汇顶科技股份有限公司 | Data transmission method, electronic device, system and storage medium |
WO2020195233A1 (en) * | 2019-03-28 | 2020-10-01 | 日本電気株式会社 | Wireless packet transmitting device, wireless packet transmitting method, and non-transitory computer-readable medium |
WO2024052912A1 (en) * | 2022-09-11 | 2024-03-14 | Ceragon Networks Ltd. | Communication balancing over non-symmetric radio links |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6134246A (en) * | 1998-01-26 | 2000-10-17 | Samsung Electronics Co., Ltd. | Inverse multiplexing within asynchronous transfer mode communication networks |
US6819658B1 (en) * | 1997-07-15 | 2004-11-16 | Comsat Corporation | Method and apparatus for segmentation, reassembly and inverse multiplexing of packets and ATM cells over satellite/wireless networks |
-
2009
- 2009-09-01 WO PCT/EP2009/061265 patent/WO2011026508A1/en active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6819658B1 (en) * | 1997-07-15 | 2004-11-16 | Comsat Corporation | Method and apparatus for segmentation, reassembly and inverse multiplexing of packets and ATM cells over satellite/wireless networks |
US6134246A (en) * | 1998-01-26 | 2000-10-17 | Samsung Electronics Co., Ltd. | Inverse multiplexing within asynchronous transfer mode communication networks |
Non-Patent Citations (3)
Title |
---|
"Application Note 3849 - Ethernet-over-PDH Technology Overview", INTERNET CITATION, XP002459059, Retrieved from the Internet <URL:http://pdfserv.maxim-ic.com/en/an/AN3849.pdf> [retrieved on 20071116] * |
ENRIQUE HERNANDEZ-VALENDA ET AL: "The Generic Framing Procedure (GFP): An Overview", IEEE COMMUNICATIONS MAGAZINE, IEEE SERVICE CENTER, PISCATAWAY, US, vol. 40, no. 5, 1 May 2002 (2002-05-01), pages 63 - 71, XP011092828, ISSN: 0163-6804 * |
ITU-T TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU: "ITU-T Recommendation G.7041/Y.1303 (08/2005): Generic framing procedure (GFP)", ITU-T RECOMMENDATION G.7041/Y.1303, XX, XX, 1 August 2005 (2005-08-01), pages 1 - 57, XP002363710 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017005324A1 (en) * | 2015-07-08 | 2017-01-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Wireless communication arrangement with two devices communicating with each other via a radio link in which a common radio interface constitutes a communication protocol for data streams arranged to support a segmentation of data frames |
US20180152388A1 (en) * | 2015-07-08 | 2018-05-31 | Telefonaktiebolaget Lm Ericsson (Publ) | Wireless Communication Arrangement With Two Devices Communicating With Each Other Via a Radio Link In Which a Common Radio Interface Constitutes a Communication Protocol for Data Streams Arranged to Support a Segmentation of Data Frames |
RU2677863C1 (en) * | 2015-07-08 | 2019-01-22 | Телефонактиеболагет Лм Эрикссон (Пабл) | Configuring wireless communication with two devices, connecting each other through radio link, to which common radio interface constitutes protocol of connection for data flows, designed with possibility of supporting segmentation of data frames |
US10560392B2 (en) | 2015-07-08 | 2020-02-11 | Telefonaktiebolaget Lm Ericsson (Publ) | Wireless communication arrangement with two devices communicating with each other via a radio link in which a common radio interface constitutes a communication protocol for data streams arranged to support a segmentation of data frames |
WO2020195233A1 (en) * | 2019-03-28 | 2020-10-01 | 日本電気株式会社 | Wireless packet transmitting device, wireless packet transmitting method, and non-transitory computer-readable medium |
CN111264079A (en) * | 2019-11-14 | 2020-06-09 | 深圳市汇顶科技股份有限公司 | Data transmission method, electronic device, system and storage medium |
WO2024052912A1 (en) * | 2022-09-11 | 2024-03-14 | Ceragon Networks Ltd. | Communication balancing over non-symmetric radio links |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US4494230A (en) | Fast packet switching system | |
JP3682082B2 (en) | Apparatus and method for packet processing in packet switching network and frame processing system for frame relay network | |
US4488289A (en) | Interface facility for a packet switching system | |
EP2050199B1 (en) | Expedited communication traffic handling apparatus and methods | |
US20030133465A1 (en) | System for reordering sequenced based packets in a switching network | |
EP1198105A2 (en) | High speed transmission line interface | |
GB2344029A (en) | Transmission of data packets of different size and priority | |
KR19990066799A (en) | Inverse Multiplexing Process Performed in ATM Network | |
EP1943792B1 (en) | Scatter and gather scheme for aggregating multiple high speed point-to-point interfaces | |
EP2371100B1 (en) | Radio link aggregation | |
US7643504B2 (en) | Credit based flow control in an asymmetric channel environment | |
US6810424B2 (en) | Link layer device and method of translating packets between transport protocols | |
JPH07321842A (en) | Apparatus for interfacing packet switching network to plurality of data terminals,module for interfacing plurality of end points to system that switches frame relay packet,and method for interfacing terminal to system that switches data packet | |
US20040131069A1 (en) | Virtual output queue (VoQ) management method and apparatus | |
WO2011026508A1 (en) | Method for distributing data traffic over radio links | |
EP0878071B1 (en) | Cell aligners | |
US20040213241A1 (en) | Port swapping for inverse multiplexed digital subscriber lines | |
EP2907256B1 (en) | Pre-fill retransmission queue | |
JPH10126421A (en) | Data transmissive communication link in fixed size packet | |
JP2003518874A (en) | data communication | |
EP1340344B1 (en) | Latency management for a network | |
CN116057865A (en) | Method and equipment for adjusting physical interface in flexible Ethernet group | |
US7042845B1 (en) | System and method for time division multiplexed switching of data using a high-speed packet switch | |
US7088738B1 (en) | Dynamic fragmentation of information | |
US6931029B1 (en) | System and method for synchronizing with data received over an unreliable asynchronous medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09782449 Country of ref document: EP Kind code of ref document: A1 |
|
DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 09782449 Country of ref document: EP Kind code of ref document: A1 |