A METHOD OF RECOMBINING DATA UNITS
TECHNICAL FIELD
This invention relates to a method of recombining data units and in particular, though not solely to a method of recombining data units in ATM enabled transmission networks. BACKGROUND ART
Asynchronous Transfer Mode (hereinafter referred to as ATM) is a system for managing the delivery of packetised data between points in a communications network.
A key property of ATM is that the data to be transported through the network is first divided into a number of fixed-size cells. Cells consist of 53 bytes allocated as X bytes of administrative content - known as overhead - and Y bytes of transport data - known as payload. The number of cells required to transport a given quantity of data can be calculated by dividing the amount of data to be transported by Y and rounding up to the next whole number. A cell is always treated as a single block of data and cannot be divided.
The cell's overhead contains information that is used by the switching equipment to route the cell through the network from its source to its destination. In particular, the overhead contains a field known as a Virtual Circuit Path Indicator (VCI/VPI) which is used to identify the route through the network that the cell should take. Specifically, management functions occur within the network to configure the equipment to route cells according to a predefined set of rules. The set of rules used to route any given ATM cell through the network are indicated by the cell's VCI/VPI.
A common application for ATM is the transportation of Internet Protocol (IP) packets. IP packets are similar to ATM cells but may be of variable length, and the data within the packet may be formatted in a variety of ways (for example, while ATM cells have a fixed format involving X bytes of overhead and Y bytes of payload, IP packets have no fixed relationship between the size of their overhead and payload sections). The transport of any given IP packet through an ATM network will typically require more than one ATM cell.
The division of an IP packet into the required number of ATM cells is a process known as segmentation. Segmentation begins with the process of dividing the IP packet into a sequence of N, Y byte segments. Each segment is then reformatted into an ATM cell by the addition of an X byte overhead. A process known as adaptation is then responsible for inserting information into the last cell in the sequence to facilitate the recovery of the original IP packet. Segmentation and adaptation occurs at the point at which the IP packet is introduced to the ATM network, and the opposite process - reassembly - occurs at the point at which the IP packet is extracted from the network.
The segmentation and adaptation of an IP packet to the required number of correctly formatted ATM cells is governed by a specification known as "ATM Adaptation Layer if where n identifies the specific revision of the adaptation specification that is used, and when the original data consists of IP packets, then the recovery of the original IP packet from a sequence of ATM cells is known as reassembly.
IP packet reassembly requires that all the cells of a packet data unit (PDU) (which is a group of ATM cells that carry a single IP packet) be available and ordered in the correct sequence. The payload sections of all the cells are extracted and concatenated to form a single block of data that is an integer multiple of Y bytes long. Managing the processes administrative overhead is an activity which is required to be performed at a rate sufficient that all of the ATM cells that arrive on the network link can be fully processed in real time.
However, present hardware implementations of systems adapted to perform high rate IP packet reassembly are highly resource intensive and require specialised hardware, such as generic or network shin a computing framework.
Moreover, such systems are considered by the applicant to demonstrate an inefficient use of those resources in the management of large numbers of partially completed PDU's at efficient rates and at the high speeds associated with full capacity on a high speed network.
It is therefore an objection of the present invention to provide a method of recombining data units which goes at least some way towards overcoming the above disadvantages or which will at least provide the public or industry with a useful choice.
It is acknowledged that the term 'comprise' may, under varying jurisdictions, be attributed with either an exclusive or an inclusive meaning. For the purpose of this specification, and unless otherwise noted, the term 'comprise' shall have an inclusive meaning - i.e. that it will be taken to mean an inclusion of not only the listed components it directly references, but also other non- specified components or elements. This rationale will also be used when the term 'comprised' or 'comprising' is used in relation to one or more steps in a method or process.
All references, including any patents or patent applications, cited in this specification are hereby incorporated by reference. No admission is made that any reference constitutes prior art. The discussion of the reference states what their authors assert, and the applicants reserve the right to challenge the accuracy and pertinency of the cited documents. It will be clearly understood that, although a number of prior art publications are referred to herein, this reference does not constitute an admission that any of these documents forms parts of the common general knowledge in the art, in New Zealand or in any other country.
Further aspects and advantages of the present invention will become apparent from the ensuing description which is given by way of example only.
DISCLOSURE OF INVENTION
In a first aspect, the invention may broadly be said to consist of in a method of recombining data units each belonging to one of a plurality of data streams, into their respective data streams utilising a plurality of data storage areas comprising the steps of:
(i) receiving a data unit belonging to a particular data stream;
(ii) determining whether the received data unit is the first data unit received from that particular data stream; (a) writing the received data unit to a particular data storage area if it is the first data unit received from that particular data stream; or
(b) writing the received data unit to a data storage area already associated with that particular data stream if it is not the first data unit received from that particular data stream; (iii) repeating steps (i) and (ii) until the last data unit of a particular data stream is received or until a data storage area is full, and
(iv) transmitting the written data unit(s) from a data storage area.
Preferably the step of transmitting the written data units comprises transmitting the written data units associated with the particular data stream for which the last data unit was received and/or the full data storage area.
Preferably, the transmitted written data unit(s) form a recombined data unit, and the method comprises carrying out steps (i) to (iv) on the recombined data units.
Preferably, the data units are ATM (Asynchronous Transfer Mode) cells.
Preferably, the data units are Internet Protocol fragments. Preferably, the data units are received via at least one data input port.
Preferably, a plurality of indicators are employed, wherein each indicator is adapted to specify an addressable location in the data storage areas.
Preferably, each unique indicator points to at least one data storage area.
Preferably, a data stream or portion of a data stream is assigned a unique indicator.
Preferably, the unique indicators are stored at a first indicator storage area prior to being assigned to a data stream or portion of a data stream.
Preferably, the unique indicators utilised are arranged in the first indicator storage area as a queue. Preferably, once a unique indicator has been assigned to a data stream or portion of a data stream it is entered in a table and then deleted from the queue of unique indicators located within the first indicator storage area.
Preferably, once the last data unit of a particular data stream is received and written to a particular data storage area, or the data storage area associated with a particular data stream is full, the unique indicator pointing to that particular data storage area is written to a second indicator storage area and then deleted from the table area.
Preferably, once a unique indicator is written to the second indicator storage area, the content of the addressable location within the data storage area specified by the unique indicator is transmitted from the particular data storage area. Preferably, the unique indicators are arranged in the second indicator storage area as a queue.
Preferably, once the content of the addressable location within the data storage area is transmitted from the particular data storage area, the unique indicator which had been assigned to the data stream, or portion thereof, is written back to the first indicator storage area at the end of the queue. Preferably, the method is controlled by computer software and wherein the data storage areas, the first and second indicator storage areas, and the computer software share the same physical memory.
In a second aspect the invention consists in a computer program adapted to cause a computer to carry out the method according to the first aspect. The present invention is adapted to provide a method for recombining data units belonging to a plurality of data streams utilising a plurality of data storage areas. Preferably such data units may belong to a particular data stream. For the purposes of this specification a data stream may be defined as a data transmission from one point in a computer network to another point. Preferably such data transmissions may be made over computer networks with the transmissions involved being made using Internet protocols.
In a further preferred embodiment the data units being combined in conjunction with the present invention may be ATM (Asynchronous Transfer Mode) cells. As discussed above, an ATM cell consists of 53 bytes of data which incorporate administrative content (known as overhead) and
transport data (known as payload). Such ATM cells are always treated as a single block of data that cannot be divided.
The data units being combined may preferably be Internet Protocol (IP) packets or fragments. An IP fragment is a portion of an IP Packet and is formed by dividing an IP Packet usually, but not only when, the size of that IP Packet exceeds the maximum transfer unit supported by a specific transmission network.
The present invention combines data units which belong to a particular data stream, and preferably ATM cells belonging to a PDL) (that is, a group of ATM cells that carry a single IP packet). The present invention may combine such ATM cells at high data reception rates. However, those skilled in the art should appreciate that the present invention may be employed to combine other data units and may further be adapted for use with other types of communications protocols and also in other applications as well. Reference to ATM cells being transmitted and combined over an ATM network only should not be seen as limiting. Those skilled in the art should appreciate that other units of data, such as IP fragments, are also envisioned by the applicant to be combinable at high speeds in accordance with the present invention and reference to the above throughout the specification should in no way be seen as limiting.
Preferably the present invention may be adapted to receive a data unit, such as an ATM cell, belonging to a particular data stream. The ATM cells which are segmented from an IP packet may be associated with a single packet data unit (PDU), where each cell of a PDU has assigned to it a unique Virtual Circuit Path Indicator (VCI/VPI). The VCI/VPI is adapted to indicate a cell's route through a network and a cell may be recognised as belonging to a particular PDU by referencing the VCI/VPI assigned to it.
The ATM cells of a PDU may be received via at least one input port. For the purposes of this specification an input port may be defined as, but need not be limited by the definition as, a component associated with a computer which is adapted to receive a signal or current, including a stream of data units. The provision of an input port provides a point for the introduction of ATM cells into a transmission network which includes the necessary processing elements for combining data units according to the method of the present invention. Such processing elements may include a computer system loaded with appropriate computer software. Those skilled in the art should appreciate that a computer system may be formed from a programmable logic unit or other processing device on which computer software may be run. Computer software may be defined as a set of instructions able to be interpreted by a computer system and capable of performing a task.
Preferably the present invention may utilise a plurality of data storage areas. Such a plurality of data storage areas may be provided by a single contiguous block of computer memory which has been divided into a plurality of fixed size memory locations. Thus the memory location following the terminal end of any given data storage area may be the onset of a subsequent data storage area and so forth. For example, a contiguous block of memory having 1 ,000,000 bytes of storage capacity may be divided into 1000 storage areas each having 1000 bytes of memory capacity. In such an instance the first storage area may comprise locations pertaining to bytes 0 to 999, where a second storage area may comprise locations 1000-1999 and so on.
However, those skilled in the art should appreciate that the storage areas employed in conjunction with the present invention may be constructed in any number or variety of different ways. For example, in alternative embodiments the storage areas employed may be variably sized and may be fragmented.
Preferably the present invention employs a plurality of unique indicators, wherein each unique indicator is adapted to point to at least one of said plurality of data storage areas. Such an indicator may also be referred to as a pointer, and may be defined as a functional unit adapted to specify an addressable point. In the context of the present specification such an addressable point may be a location in the data storage area.
Preferably each PDU or portion thereof received is assigned a unique indicator. By making such an assignment each ATM cell contained within a PDU may be associated with a specific and unique data storage area.
Preferably the plurality of unique indicators employed are stored at a first indicator storage area prior to being assigned to a PDU or portion thereof.
The indicators utilised may be arranged in the first indicator storage area as a queue. A queue is a well defined object in computing terms and the behavioural requirements of the queue of the first indicator storage area may be that of a conventional "First In First Out" (FIFO) queue, or circular buffer. In the context of the present specification the FIFO queue will operate as a queuing discipline in which the unique indicators in the queue leave the queue in the same order in which they arrive. In particular, a unique indicator that is assigned to a PDU (or portion thereof) will be the unique indicator that has been in the FIFO queue the longest. The indicators may alternatively be arranged in a stack, a linked list, a tree or other data structure. Those skilled in the art should appreciate that a variety of data structures are also envisioned for use with the present invention and reference to FIFO queues only throughout the specification should be seen as limiting.
Preferably once an indicator has been assigned to a PDU or portion thereof it is entered in a table and then deleted from the queue of indicators located within the first indicator storage
area. By assigning a unique indicator to a PDU, and recording such an assignment in a table, each ATM cell which forms that PDU may be effectively associated and addressed to a specific storage area. Consequently, the ATM cells of a PDU, or portion of a PDU, may be stored together in a contiguous block of memory within a data storage area. The ability to assign unique indicators to PDU's adapted to point to an addressable storage area is not to be confused with the allocation of a VCI/VPI (Virtual Circuit/Path Indicator) to a PDU. The VCI/VPI is adapted to differentiate the PDU to which an ATM cell belongs as although the ATM cells of a PDU are introduced into a transmission network in sequential order, the ATM cells from any number of other PDU's may be interleaved between the cells of a specific PDU. All the ATM cells of a PDU have the same VCI/VPI, which as noted above, is used to identify the route through the network that the ATM cell should take.
The table employed may be implemented as a lookup table data structure in a standard memory component of a computing system. A lookup table formed in accordance with such an embodiment may be adapted to store a variety of information, where such information may be formatted as a plurality of records.
Therefore, by entering in such a lookup table the assignment of a unique indicator to a PDU, an index of those storage areas associated with the unique indicator, and consequently the cells of the PDU, may be formed. The records thus entered within the lookup table may be formatted as necessary to facilitate fast and efficient parsing as required or as desired. In addition to the entry of unique indicators into the table, a record of the VCI/VPI's pertaining to the ATM cells received via the input port may also be entered into an index associated with the lookup table. Furthermore, once a unique indicator has been entered into the lookup table it may be deleted from the queue of indicators located within the first indicator storage area.
Preferably the present invention may be adapted to determine whether a received ATM cell is the first ATM cell received from a particular PDU. Such a determination may be made by parsing entries made in the index of the lookup table to determine whether a VCI/VPI pertaining to the particular PDU associated with the ATM cell has been entered. If it has, then it can be considered that the first ATM cell of a PDU has been received. Conversely, if the entries contained within the index contain no record of the VCI/VPI of the received ATM cell, then it can be assumed that the first ATM cell of a PDU has not previously been received.
As the index may contain large numbers of entries, and the parsing of these entries is required to be performed at high speeds, the use of at least one high speed memory component, such as a Content Addressable Memory (CAM), may be employed in conjunction with the present invention.
Preferably the present invention is adapted to write the payload of the received ATM cell to a particular data storage area if the received ATM cell is the first ATM cell received from a particular PDU. Having identified that the received ATM cell is the first ATM cell received from a PDU, a unique indicator (adapted to point to an addressable storage area) is taken from the head of the queue of indictors (located within the first indicator storage area) and associated to the ATM cell, where the details of the association are entered into the lookup table.
Once the indicator and association details have been entered into the table the payload of the ATM cell is written to a storage area. Specifically, the ATM cell is written to the region of the storage area being pointed at by the unique indicator assigned to the PDU to which the ATM cell belongs.
The payload of a received ATM cell is written to a data storage area already associated with that particular PDU if it is not the first ATM cell received from that particular PDU. If the VCI/VPI of the ATM cell received is recognised in the index of the lookup table it can be assumed to be the nth subsequent cell of a PDU and that a data storage area has already been associated with the PDU to which the ATM cell belongs and that the PDU is presently being reassembled in that data storage area.
The received ATM cell is therefore written to the storage area at a position immediately following the position in the storage area occupied by the previously received ATM cell from that PDU. As ATM cells have a fixed size, the position in the storage area occupied by the previously received ATM cell is calculable by multiplying the number of cells already received for that PDU by the size of each cell. A record of the number of cells received for a specific PDU is also entered in the lookup table to facilitate the writing of a received ATM cell at the correct position within the storage area, being the storage area at which cells for the PDU are being reconstructed. As the payload of the ATM cell is copied to the storage area, a CRC (cyclic redundancy check) calculation is performed on its contents.
If this cell is the first cell received from a particular PDU then the CRC calculation starts from a known initial value and the partial result is stored in the lookup table. If this cell is the nth cell of a PDU then the CRC calculation starts with the partial CRC value retrieved from the lookup table, with the revised result written back into the status lookup table. If this cell is the last cell of a PDU then the CRC calculation starts with the partial CRC value that is retrieved from the status lookup table, and the updated result is compared with the cell's CRC field.
In the event of a mismatch the packet is judged to be bad and is dropped by placing its unique indicator back on the queue of the first indicator storage area. This causes the storage area at which the PDU was being reconstructed to be reclaimed and its contents lost.
Preferably once the last ATM cell of a particular PDU is received and written to a particular data storage area, or the data storage area associated with a particular PDU is full, the indicator pointing to that particular data storage area is written to a second indicator storage area and then deleted from the lookup table. The indicators in the second indicator storage area may be arranged as a queue, being a queue similar to that used to store the indicators in the first indicator storage area or any of the previously mentioned data structures.
Thus, once the last ATM cell of a particular PDU is received and written to a particular data storage area, or the data storage area associated with a particular PDU is full, the indicator is written to the end of the queue of indicators presently residing in the second indicator storage area. The last cell of a PDU is designated by the setting of a flag within that last cell's administration data.
The entry in the lookup table relating to the unique indicator, and the associated VCI/VPI's of the PDU, are then deleted. Cases may arise when an ATM cell arrives with an unrecognized VCIΛ/PI (indicating that it is the first cell of a PDU) yet its flag is set (indicating that it is also the last cell of a PDU). This scenario indicates that the PDU contains only a single cell. In such circumstances a unique indicator is allocated to the ATM cell from the head of the queue of indicators located in the first indictor storage area, and then the indicator is moved to the end of the queue of indicators located in the second indicator storage area.
Preferably once an indicator is written to the second indicator storage area, the content of the data storage area being pointed at by the indicator is transmitted from the particular data storage area.
The step of transmitting the written data units comprises transmitting the written data units associated with the particular data stream for which the last data unit was received and/or the full data storage area.
When an indicator appears in the second indicator storage area the content of the data storage area being pointed to is read-out sequentially, starting at the indicator and moving through the storage area as far as the end of last cell in the PDU. This process requires not only that the indicator pointing to the storage area is transmitted, but also that a count of the number of ATM cells held in the storage area be maintained.
The transmitted units of ATM cells may form either a whole reassembled IP packet or an IP fragment. Accordingly, the present invention may firstly be used to reassemble, for example, ATM cells into IP fragments from which the ATM cells were produced. The invention may then
be carried out on the IP fragments in the same way to thereby reassemble IP packets from which the IP fragments were produced.
Preferably once the content of the data storage area is transmitted from the particular data storage area, the indicator which had been assigned to the data stream, or portion thereof, is written back to the first indicator storage area at the end of the queue. This enables that indicator, and the storage area associated with the indicator to be recycled or allocated to later received PDU's.
Preferably the lookup table, and the first and second indicator storage areas and associated pointers adapted to carry out the method according to the present invention coexist at the same area of physical memory.
The present invention provides many advantages over the prior art.
The movement of indicators from the first indicator storage area, to the look-up table, to second indicator storage area and back to the first indicator storage area represents a highly efficient and effective technique for managing a large number of incoming data units, such as ATM cells. Further, this system provides flexibility in the number of simultaneously active streams of ATM cells that can be supported. Such a provision is of advantage.
The present invention is adapted to manage large numbers of partially completed PDU's efficiently and at the high speeds associated with full capacity on a high speed digital network, which is of advantage. As the total number of indicators stored in the first and second indicator queues will vary during the system's operation, the number of indicators will always be less than or equal to the total number of areas available for occupation within the data storage area. As the sizes of the two queues have a clear relationship to each other, they may both share the same physical memory without any risk of one queue corrupting the other. Taking advantage of this relationship reduces the number of physical memory components required to implement the two queues, thus simplifying the hardware construction required to perform the method of the present invention which is of advantage.
BRIEF DESCRIPTION OF DRAWINGS
Further aspects of the present invention will become apparent from the following description which is given by way of example only and with reference to the accompanying drawings in which: Figure 1 : is a block schematic diagram of hardware elements and associated data flow lines employed there between in implementing a system and method of recombining data units in accordance with a preferred embodiment of the present invention; and
Figure 2: is a schematic diagram illustrating the relationship in memory between the queues formed within the first and second indicator storage areas, the lookup table and the pointers.
BEST MODES FOR CARRYING OUT THE INVENTION
The hardware elements of figure 1 are adapted for use by a system 1 for recombining data units, each data unit belonging to a respective data stream. Such a system includes an input port 2 which provides a point for the introduction of ATM cells or IP fragments into a transmission network associated with the system 1. The order of receipt of data units which belong to a particular data stream may be controlled to ensure that the data units are ordered (for example, an ATM system ensures that cells are received in the same order in which they are sent). However, adjacent (in time) cells may be from any particular data stream. Also included in figure 1 are a plurality of data storage areas 5A - 5F (there may be more or less than six areas), first indicator storage area 6 and second indicator storage area 7. First * indicator storage area 6 is adapted to store a plurality of unique indicators (not shown), where each unique indicator is adapted to point to one of said plurality of data storage areas 5. The indicators are arranged in first indicator storage area 6 as a queue. Firstly, when an ATM cell or IP fragment is received via input port 2, its associated VCI/VPI is checked by the system 1 against all VCI/VPI's currently entered into index component 3, which is associated with lookup table 4.
Such a check enables the determination of whether the received ATM cell or IP fragment is the first received from a particular PDU. A record of the VCIΛ/PI of the received ATM cell or IP fragment within the entries contained within the index 3 of the lookup table 4 indicates that the first ATM cell or IP fragment of a PDU has been previously received via input port 2. If the received data unit is the first in a PDU to be received then its VCI/VPI is written to the index 3.
Once the first ATM cell or IP fragment is received it is assigned a unique indicator from the queue of indicators stored in the first indicator storage area 6. By making such an assignment
each ATM cell or IP fragment contained within a PDU may be associated with a specific and unique addressable point within the data storage areas 5.
On the assignment of an indicator to a PDU or portion thereof, the assigned indicator is entered into lookup table 4 and then deleted from the queue of indicators located within the first indicator storage area 6.
If the received ATM cell or IP fragment is the first to be received from a particular PDU it is written to a particular data storage area, for example area 5A, that is the storage area being pointed at by the unique indicator previously assigned to the PDU.
If the received ATM cell or IP fragment is not the first received from a particular PDU it is written to the data storage area, for example area 5B, already associated with that particular PDU. Therefore, if the VCI/VPI of the ATM cell or IP fragment received is recognised in the index 3 it can be assumed to be the nth subsequent cell of a PDU and that a data storage area 5A - 5F has already been associated with the PDU to which the ATM cell belongs and that the PDU is presently being reassembled in that data storage area. The received ATM cell or IP fragment is then written to the storage area 5 at the position immediately following the position in the storage area occupied by the previously received ATM cell or IP fragment. A record of the number of data units received for a specific PDU is also entered into the lookup table 4 to facilitate the writing of a received ATM cell at the correct position within the storage area 5, being the storage area at which cells for the PDU are being reassembled?
As the payload of the ATM cell or IP fragment is copied to the storage area 5, a CRC (cyclic redundancy check) calculation is performed (as discussed above).
Once the last ATM cell or IP fragment of a particular PDU is received and written to a data storage area 5, or the data storage area 5 associated with a particular PDU becomes full, the indicator pointing to that particular data storage area 5 is written to a second indicator storage area 7 and then deleted from the lookup table 4. The entry in the lookup table 5 relating to the unique indicator, and the associated VCI/VPI's of the PDU, are then deleted.
When a PDU contains only a single cell or fragment a unique indicator is allocated to the PDU from the head of the queue of indicators located in the first indictor storage area 6, and the indicator is then moved to the end of the queue of indicators located in the second indicator storage area 7.
Once an indicator is written to the second indicator storage area 7 the content of the data storage area 5 being pointed at by the indicator is transmitted from the particular data storage area 5 by transmission system 8. Thus when an indicator appears in the second indicator storage area 7 the content of the data storage area 5 being pointed at is read-out sequentially,
starting at the indicator and moving through the particular storage area 5A - 5F as far as the end of last cell in the PDU.
The combined and subsequently transmitted units of ATM cells or IP fragments may form either a whole reassembled IP packet or an IP fragment. Finally, once the content of the data storage area 5 is transmitted by transmission system 8 the indicator which had been assigned to the PDU is written back to the first indicator storage area 6 at the end of the queue. This enables that indicator, and the storage area associated with the indicator, to be allocated to later received PDU's.
Figure 2 illustrates the relationship in memory between the first and second indicator storage areas, the lookup table and the process associated with transmitting data from the data storage area.
Shown schematically with respect to figure 2 is a single contiguous physical memory region 9 in which the first indicator storage areas 10, the second indicator storage area 11 , the lookup table 12 and the associated pointers 13 used to carry out the present invention coexist. It can be seen that the memory region 9 is arranged as a circular buffer so that the last memory location is adjacent to the first memory location. Alternatively, the first indicator storage area 10 and the second indicator storage area 11 be located in a single contiguous block of memory and the look-up table 12 and pointers 13 may be located in a separate single contiguous block of memory separate from the memory occupied by the first and second indicator storage areas. The applicants believe that the method provided for in this specification is able to at least service 1 Gbyte/sec of overall data unit throughput (that is the number of bytes that may be processed per second), 20 million ATM cells or IP fragments per second and the simultaneous processing of at least 65,000 active VCI/VPI's.
The applicants further consider that the performance characteristics of prior art methods of data reassembly do not approach these performance levels.
Aspects of the present invention have been described by way of example only and it should be appreciated that modifications and additions may be made thereto without departing from the scope thereof as defined in the appended claims.