US20030229844A1 - Graceful degradation of serial channels - Google Patents

Graceful degradation of serial channels Download PDF

Info

Publication number
US20030229844A1
US20030229844A1 US10/396,287 US39628703A US2003229844A1 US 20030229844 A1 US20030229844 A1 US 20030229844A1 US 39628703 A US39628703 A US 39628703A US 2003229844 A1 US2003229844 A1 US 2003229844A1
Authority
US
United States
Prior art keywords
data
stripe
channels
stripes
group
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/396,287
Inventor
Akash Bansal
Jaisimha Bannur
Anujan Varma
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Intel Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corp filed Critical Intel Corp
Priority to US10/396,287 priority Critical patent/US20030229844A1/en
Assigned to INTEL CORPORATION reassignment INTEL CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BANSAL, AKASH, BANNUR, JAISIMHA, VARMA, ANUJAN
Publication of US20030229844A1 publication Critical patent/US20030229844A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/40Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection

Definitions

  • High-speed store-and-forward devices such as switches and routers, used in today's communication networks have a large amount of data passing through them. These devices typically include a set of line cards, which perform various operations within the communication networks. Communication between these line cards usually takes place over a backplane, which provides connectivity among the line cards, e.g., via dedicated point-to-point or switched communication paths. With advances in serial communication technologies, the preferred choice for high-speed backplanes today is to use one or more high-speed serial links (channels). High-speed serial data can be carried over either electrical backplanes or optical backplanes.
  • the transmitting line card must convert electrical signals to optical signals and send the optical signals over fiber, and the destination line card must receive the optical signals from the fiber and reconvert them to electrical signals.
  • the backplane may be used to switch data between line cards or may transport the data without switching.
  • Serializers and deserializers are used, in conjunction with an encoding scheme, such as 8-bit to 10-bit encoding, to create a self-clocked high-speed serial electrical data stream.
  • FIG. 1 illustrates an exemplary system having multiple line cards connected through serial links over a point-to-point or switched backplane, according to one embodiment
  • FIGS. 2A and B illustrate exemplary charts explaining how a frame is striped (interleaved) across data channels, according to one embodiment
  • FIG. 3 illustrates an exemplary transmission module, according to one embodiment
  • FIG. 4 illustrates an exemplary receiving module, according to one embodiment
  • FIG. 5 illustrates an exemplary flowchart of the processing of a frame received over a plurality of channels, according to one embodiment
  • FIG. 6 illustrates an exemplary parity byte calculation, according to one embodiment
  • FIG. 7 illustrates an exemplary transmission module, according to one embodiment
  • FIG. 8 illustrates an exemplary receiving module, according to one embodiment
  • FIGS. 9A and B illustrate an exemplary flowchart of the processing of a frame received over a plurality of channels, according to one embodiment.
  • FIG. 10 illustrates an exemplary frame and stripe format, according to one embodiment.
  • FIG. 1 illustrates an exemplary system 100 for transmitting data amongst various sources and destinations.
  • the system may transmit the data using any number of protocols including Asynchronous Transfer Mode (ATM), Internet Protocol (IP), and Time Division Multiplexing (TDM).
  • ATM Asynchronous Transfer Mode
  • IP Internet Protocol
  • TDM Time Division Multiplexing
  • the data may be sent in variable length or fixed length blocks, such as cells, packets or frames.
  • the communication lines used to transmit data may be fiber, copper, or other mediums.
  • the system includes at least one store-and forward device 105 , such as a router or packet switch.
  • the store-and-forward device 105 includes multiple line cards connected together through serial links over a point-to-point or switched backplane.
  • a plurality of ingress modules 110 are connected through a backplane 120 to a plurality of egress modules 130 .
  • the backplane 120 may be electrical or optical.
  • the ingress modules 110 and the egress modules 130 are typically two sides of a line card.
  • the line cards may be Ethernet (e.g., Gigabit, 10 Base T), ATM, Fibre channel, Synchronous Optical Network (SONET), and Synchronous Digital Hierarchy (SDH), amongst others.
  • the data transmitted over the backplane is broken up into segments (e.g., frames).
  • the segment includes a single packet or a set of packets sent from a source line card to destination line card. Each segment has a defined maximum length which does not exceed a predetermined maximum length.
  • the data (e.g., frames) being transmitted from one card to another card over the backplane is striped over multiple high-speed serial channels N between the cards. Striping is accomplished by breaking the data up and transmitting portions of the data over each of the N channels. For example, if the frame to be striped had W bytes, each channel would transmit W/N bytes.
  • a first byte (e.g., byte 0) is transmitted over a first channel (e.g., channel 0), a second byte (e.g., byte 1) is transmitted over a second channel (e.g., channel 1), and so on until an N th byte (e.g., byte N ⁇ 1) is transmitted over the N th channel (e.g., channel N ⁇ 1).
  • N th byte e.g., byte N ⁇ 1
  • the next byte (e.g., byte N) is transmitted on the first channel (e.g., channel 0).
  • the process is repeated until all W bytes in the frame are transmitted.
  • One way to look at this is that the data is broken up into W/N groups, with each group having N bytes.
  • One byte (of the N bytes) from each group is then transmitted to each data channel (of the N data channels).
  • the bytes belonging to a frame that travel on one specific channel is called a stripe.
  • the sequence of bytes 0, N, 2N, 3N, etc. would make up stripe 0 (e.g., the part of the frame traveling on channel 0).
  • some channels may have less than W/N bytes (or may have no data) and that the last group may have less than N bytes.
  • FIG. 2A illustrates an exemplary chart explaining how a frame including W bytes is striped (interleaved) across N data channels.
  • the first column lists the byte number (0 to W ⁇ 1), the second column lists the data channel number (0 to N ⁇ 1) and the third column the group number (0 to (W/N) ⁇ 1).
  • bytes 0 to N ⁇ 1 are striped over data channels 0 to N ⁇ 1 to form a first group (group 0); bytes N to 2N ⁇ 1 are striped over data channels 0 to N ⁇ 1 to form a second group (group 1); and so on until bytes ((W/N) ⁇ 1)N to W ⁇ 1 are striped over data channels 0 to N ⁇ 1 to form a last group (group (W/N) ⁇ 1).
  • FIG. 2B illustrates an exemplary chart explaining how a frame including 26 bytes is striped (interleaved) across 6 data channels.
  • This example would create 4 groups having 6 bytes each and a fifth group having two bytes.
  • the first six bytes (bytes 0 to 5) are striped over each data channel (channels 0 to 5) to form a first group (group 0); the next six bytes (bytes 6 to 11) are striped over each data channel (channels 0 to 5) to form a second group (group 1); and so on until the last two bytes (bytes 24-25) are striped over the first two data channels (channels 0 and 1) to form a last group (group 4).
  • the various embodiments are in no way intended to be limited to the striping assignments discussed above. Rather, the bytes can be transmitted over the channels in any order. For example, byte 0 could be assigned to channel (N ⁇ 1), byte 1 could be assigned to channel (N ⁇ 2) and so on. Moreover, the various embodiments are not limited to transmitting the data byte by byte. The data could be transmitted bit by bit, sector by sector, block of bytes by block of bytes, or block of bits by block of bits, where a block can be defined by a user. Furthermore, the various embodiments are not limited to receiving frames that are organized as W bytes. Rather the frames could be organized by bits, sectors, or other ways.
  • a cyclic redundancy code (CRC) is computed for the frame.
  • the CRC is inserted at the end of the frame and is transmitted along with the frame (data). Also, a separate CRC is computed for each stripe and is sent as part of the stripe.
  • FIG. 3 illustrates an exemplary transmission module 300 , according to one embodiment.
  • the transmission module 300 receives an input frame that is W bytes long at a CRC generator 310 that generates a CRC for the entire frame.
  • the entire frame including the CRC is provided to a striper 320 .
  • the striper 320 divides the W bytes into N groups and selects the channel for each group to be transmitted over. That is, one group will be transmitted over each of the N channels.
  • a CRC is generated for each stripe and is inserted at the end of the stripe by a CRC module 330 .
  • the stripe and the associated CRC for each channel are then provided to a transmitter 340 for transmitting over the backplane.
  • N transmitters 340 There are a total of N transmitters 340 , one for each data channel.
  • the transmission module can be reconfigured to stripe the data over fewer channels. Up to P channels can be configured out of the system so that the data can be striped over a minimum of N-P channels.
  • the data sent over each channel is received and buffered and the frame is recreated.
  • a CRC is computed for the entire received frame and compared to the CRC that was transmitted with the frame in order to check for errors in the received frame.
  • a CRC is computed for each channel and compared to the transmitted CRC for the channel to check for errors in the strip. If the computed CRC does not match with the transmitted CRC at the end of the stripe, the data within the stripe is deemed to be in error.
  • the CRC error indication for each channel is ORed with an error signal from a corresponding physical receiver device. The physical receiver device indicates errors such as loss of signal etc.
  • the ORed error signal is referred to as a channel error. If there is an error for the overall frame and/or one or more channel errors the frame is discarded.
  • Each channel is provided with a channel-specific error counter and an associated threshold register.
  • any channel-specific error count exceeds the threshold, there is a provision for an interrupt to be issued to the processor (or custom hardware) controlling the system.
  • the software or custom hardware
  • the software would set the value of the threshold such that in a given interval, if the number of errors exceed the value specified in the threshold register, it is likely that the channel has a permanent hardware problem.
  • the system may then be shut down to replace one or more components to restore full throughput. If the error count does not exceed the threshold in the specified interval, it is likely that any errors that occurred were random in nature and may be ignored. At the end of each specified interval, the channel-specific error counts are reset. Note that the software (or custom hardware), needs to keep a rolling average of the per-channel error count.
  • Fixing a channel may cause down time on the system.
  • the CPU custom hardware may reconfigure the system to stripe the data over fewer channels (N ⁇ 1 channels, N ⁇ 2 channels, etc.).
  • the data can be striped over any number of channels in the range N-P (minimum) to N (maximum). This allows for P channels to fail, and still have data transmission over the backplane and through the switching chips to the destination line card, albeit at a reduced bandwidth.
  • the value of P is determined by the reduction of throughput that can be tolerated.
  • FIG. 4 illustrates an exemplary receive module 400 , according to one embodiment.
  • the receive module 400 includes N serial channel receivers 410 for receiving and buffering data over the N data channels.
  • a CRC module 420 generates a CRC for each data channel (stripe).
  • the stripes are provided to a destriper 430 that takes the N channels and converts it back into a W byte (or bit, etc.) frame.
  • the destriper 430 forwards the frame to a CRC computation module 440 that computes the CRC for the entire frame and then compares the generated frame CRC to the transmitted frame CRC, and the generated stripe CRCs to the transmitted stripe CRCs to determine errors. Errors in any channel are recorded and compared to an error threshold. If the channel exceeds the error threshold it is configured out of the system until it can be repaired. Up to P channels can be configured out of the system so that the striped data can be transmitted using as little as N-P channels (minimum number of channels).
  • FIG. 5 illustrates an exemplary flowchart of the processing of a frame received over a plurality of channels.
  • the received frame is checked for an error indicated by the frame-level CRC ( 500 ). If there is no error in the frame level CRC ( 500 No), then each individual stripe is checked for errors based on the CRC or an error indication from the physical receiver device ( 505 ). If there is no error in any channel ( 505 No), then the frame is declared good and is sent out for further processing ( 510 ). If there are one or more channels with errors ( 505 Yes), then the error is an uncorrectable error. The frame is discarded and the corresponding per-channel error counters are incremented ( 515 ).
  • an interrupt is issued to a CPU, or custom hardware ( 540 ).
  • the CPU custom hardware receives the interrupt indicating that the threshold(s) has been exceeded, and makes a determination as to whether there are enough usable channels remaining to support the system ( 545 ). That is, if we take the faulty channel out of service are the number of channels remaining at least equal to the minimum number of channels (N-P) required to meet the bandwidth requirements of the system? If the number of channels remaining after deactivating the faulty channels is less than the minimum number of channels ( 545 No), then the system is shut down for servicing ( 550 ). If the number of channels remaining after deactivating the faulty channels is still at least equal to the minimum number of channels ( 545 Yes), then the system is reconfigured to utilize only working channels ( 555 ).
  • the CPU custom hardware may switch from a graceful degradation mode (the mode described above where if a channel(s) has multiple errors that exceed a threshold they can be configured out of the system) to a single channel error correction mode.
  • the single channel error correction mode allows a single channel error to be corrected.
  • the N serial channels are divided into M data channels and K parity channels.
  • the parity channels are used to send parity information associated with the data being sent. If the data is being striped by bytes then the parity will be transmitted by bytes as well.
  • the parity bytes (or other grouping) are used for correction of errors.
  • the M data channels are divided into groups and each group is associated with a parity channel. As there are K parity channels it follows that there will be K groups of data channels and each group will have M/K channels. For each byte within a group (e.g., M/K channels) there is an associated parity byte in the corresponding parity channel.
  • each parity byte is generated by XORing (that is, computing the logical Exclusive-OR operation of) each bit of each associated byte in the data channels within the group. For example, bit 0 of the parity byte is the XOR of the bits in position zero of all the bytes in that group, bit 1 is the XOR of the bits in position one of all the bytes in that group, and so on.
  • FIG. 6 illustrates an exemplary parity byte calculation for a group of 3 data channels that are utilized to transmit a plurality of bytes.
  • the parity byte is calculated by XORing each bit of the data bytes.
  • the various embodiments are in no way intended to be limited to generating the parity data by XORing associated bits together. Rather, there are multiple methods for generating parity data (e.g., bits, bytes, etc.).
  • the data for that channel can be recovered.
  • the data for the channel is recovered by XORing the data from all the other channels (data and parity) in the group.
  • the recovered data replaces the received data for that channel (stripe).
  • the frame is reassembled from all the received channels (stripes) and the corrected channel (stripe).
  • a second CRC check is then performed on the frame. If this CRC check passes, the entire frame is deemed error-free and accepted by the receiver. An error count is incremented for the data channel having the failure. If the error count exceeds some predefined threshold, action may be taken to fix the channel (discussed later).
  • FIG. 7 illustrates an exemplary transmission module 700 , according to one embodiment.
  • the transmission module 700 is much like the transmission module 300 of FIG. 3 with the exception that the N serial channels are divided into M data channels and K parity channels.
  • a CRC generator 710 receives a frame and generates a CRC for the entire frame.
  • the entire frame including the CRC is provided to a striper 720 .
  • the striper 720 divides the frame into M groups and selects the channel for each group to be transmitted over.
  • a CRC is generated for each stripe and is inserted at the end of the stripe by a CRC module 730 .
  • the stripe and the associated CRC for each channel are then provided to a transmitter 740 for transmitting over the backplane.
  • Each parity channel is associated with a group of M/K data channels.
  • the M/K data channels associated with each parity channel are provided to a parity generator 750 that generates the parity data associated with the group of data channels.
  • the parity generator 750 is an XOR gate for each bit of the data being transmitted at a time.
  • the XOR gate receives an associated bit from each of the group of M/K data channels.
  • a CRC is generated for each parity channel stripe and is inserted at the end of the parity stripe by a CRC module 760 .
  • the parity stripe and the associated CRC for each parity channel are then provided to a transmitter 770 for transmitting over the backplane.
  • the transmitters 740 and the transmitters 770 are illustrated separately for convenience of pointing out the difference between the parity channels and the data channels. However, it should in no way be construed to require different CRC modules and transmitters for parity channels and data channels. Rather, the CRC modules and the transmitters could be the same. In fact, according to one embodiment, the transmission module 700 can easily be reconfigured to have more data channels or more parity channels.
  • FIG. 8 illustrates an exemplary receive module 800 , according to one embodiment.
  • the receive module 800 is much like the receive module 400 of FIG. 4 with the exception that parity bits are received and used for regeneration (correction) of errored stripes.
  • the receive module 800 includes M serial channel receivers 810 for receiving and buffering data over the M data channels.
  • a CRC module 820 generates a CRC for each data channel (stripe).
  • the stripes are provided to a destriper 830 that takes the M channels and converts it back into a frame.
  • the destriper 830 forwards the frame to a CRC computation module 840 that computes the CRC for the entire frame and then compares the generated frame CRC to the transmitted frame CRC, and the generated stripe CRCs to the transmitted stripe CRCs to determine errors.
  • the receive module 800 also includes K serial channel receivers 850 for receiving parity data.
  • a CRC module 860 generates a CRC for each parity channel (stripe).
  • Each parity channel and the data channels that are associated with the parity channel as part of a group, are provided to error correction modules 870 .
  • the error correction modules 870 are XOR gates.
  • the error correction modules 870 provide the corrected data channels to the destriper 830 .
  • receivers 810 and the receivers 850 are illustrated separately for convenience of pointing out the difference between the parity channels and the data channels.
  • the receivers and CRC modules could be the same type of receivers.
  • the receiver module 800 can easily be reconfigured to have more data channels or more parity channels.
  • FIGS. 9A & B illustrate an exemplary flowchart of the processing of a frame received over a plurality of channels.
  • the flowchart of FIGS. 9A & B is similar to the flowchart of FIG. 5 with the exception that a switch to the single channel error correction mode (described above) from the graceful degradation mode (described above) is possible.
  • the mode of the system may be switched at any point in time either automatically or by an operator.
  • One possible occasion for the switch in mode would be the case where the error threshold has not been exceeded for any particular channel but that there have been multiple single channel failures causing multiple frames to be discarded.
  • the system could reconfigure itself to include parity channels (thus reducing the bandwidth available for the data) so that the errors could be self-correcting.
  • the channel error determination ( 520 ) is modified to be a determination as to whether there was a single channel error ( 900 ). If the errors were not limited to a single channel error ( 900 No), the frame is discarded ( 905 ). Also, the frame error count is incremented and the channel error counts for the errored channels are incremented ( 905 ). If the error was limited to a single channel error ( 900 Yes), a determination is made as to whether the system is in single-channel error correction mode ( 910 ).
  • the frame is discarded, the frame error count is incremented and the channel error count for the errored channel is incremented ( 920 ). If the errors were limited to a single channel per group ( 910 Yes), a determination is made as to whether the channel having the error is the parity channel ( 930 ). If the channel error was not in the parity channel ( 930 No), the data is the channel is recreated (corrected) by XORing all the other stripes within that group, the frame is generated using the corrected stripe, the frame is sent out, and the channel error count for the errored channel is incremented ( 940 ). If the channel error was in the parity channel ( 930 Yes), the frame is sent out, and the parity channel error count is incremented ( 950 ).
  • the error count is compared to the threshold ( 535 ) and if exceeded the CPU receives an interrupt.
  • the CPU custom hardware
  • receives an interrupt ( 540 ) indicating that a threshold(s) has been exceeded a determination will be made as to whether the system is in graceful degradation mode ( 960 ). If the system is not in graceful degradation mode ( 960 No), the CPU (custom hardware) will perform the necessary actions ( 970 ).
  • the necessary actions may be to shut down and repair the system now, tag the system for shut down/repair in the future, or switch to graceful degradation mode. If the system was in graceful degradation mode ( 960 Yes), the system proceeds to determine if the number of usable channels is sufficient 545 .
  • test data there is also a provision for test data to be sent over one or more channels. Errors detected on the received data are recorded per-channel. This can be used by the software (or custom hardware) to test individual channels before deciding the number of channels to use, or as a periodic self-diagnostic feature.
  • the system may act in both the single channel error correction mode and the graceful degradation mode at the same time.
  • This embodiment is similar to the embodiment of FIGS. 9A and B with the exception that the graceful degradation determination 960 and associated perform action 970 , as well as single channel error correction determination 910 and associated discard frame 920 are no longer required.
  • the system will utilize both data and parity channels so that single channel error corrections can be made.
  • the CPU custom hardware
  • the CPU will make a determination if there are sufficient channels available to deactivate the channel or channels 545 .
  • the determination is not just made on data channels as in the embodiment of FIGS. 9A and B (at least N-P channels).
  • a determination will have to be made as to whether enough data channels are available with enough associated parity channels to be able to handle single channel error detection. Therefore a minimum number of data channels and associated parity channels need to be defined.
  • FIG. 10 illustrates an exemplary frame and stripe format.
  • high speed serial electrical channels are used over a backplane, and switched using a crossbar.
  • the encoding scheme used for framing is the industry-standard 8-bit to 10-bit encoding. It provides a “Start of Frame Delimiter” (SFD) symbol, an “End of Frame Delimiter” (EFD) symbol, and an “Idle Character” symbol.
  • SFD Start of Frame Delimiter
  • ESD End of Frame Delimiter
  • Idle Character an exemplary frame and stripe format.
  • the input data 1000 is broken into frames, and each frame is appended with a frame-level CRC 1010 .
  • the frame-level CRC 1010 is computed over all the data bytes (or bits, etc.) of the frame.
  • the data is striped over the seven data channels 1020 , byte by byte.
  • the data can be striped across the channels in other manners, such as bit by bit or group by group.
  • a parity channel 1030 transmits parity data for the group which is the XOR of all the data from the seven data channels.
  • the parity data is derived by the bit-wise XOR of the corresponding bytes of each of the seven data channels.
  • Each of the channels initially transmits a first stripe for the frame 1040 that is the start of frame symbol. This is followed by the data stripe (data for the seven data channels 1020 and parity data for the parity channel 1030 ) 1050 .
  • a last data stripe 1060 is the CRC computed for each stripe.
  • a last stripe for the frame 1070 is the end of frame symbol. If only a single channel fails, the channel can be recreated (fixed) by XORing all the other channels in the group. If one of the channels records enough errors to cross the threshold level, the system could either shut down for servicing (now or later) or could reconfigure itself to a graceful degradation mode in which case that channel would be reconfigured out of the system and the data would be transmitted over the remaining channels.
  • the system can be reconfigured between single channel error correction mode and graceful degradation mode, if a single channel failed once the system was reconfigured to remove the channel (whether a data channel or the parity channel) the data would continue be transmitted over seven channels but would have no parity channel and thus no error correction.
  • the system can continue to correct errors while at the same time reconfiguring to deactivate faulty channels. For example, if a data channel failed the data could be striped over the remaining six data channels and the parity channel could continue be used for error correction. However, if the system needed at least 6 data channels to meet bandwidth requirements if a second channel failed the parity channel would have to be deactivated which would eliminate the error correction capability at that point.
  • Different implementations may feature different combinations of hardware, firmware, and/or software.
  • some implementations feature computer program products disposed on computer readable mediums.
  • the programs include instructions for causing processors to perform techniques described above.

Abstract

In general, in one aspect, the disclosure describes an apparatus that includes a transmission module to split a data segment into a plurality of data stripes and transmit each data strip over an associated serial channel, a reception module to receive the plurality of data stripes over the associated serial channels and track a number of errors per channel, and a controller to deactivate a serial channel and reconfigure said transmission module and said reception module to utilize remaining data channels for striping data if the number of errors in the serial channel exceeds a threshold.

Description

  • This application claims priority under 35 U.S.C. §119(e) of U.S. Provisional Application No. 60/367,630 entitled “Error Detection and Correction of Data Striped Over Multiple Serial Channels” filed on Mar. 25, 2002 which is herein incorporated by reference, but is not admitted to be prior art.[0001]
  • BACKGROUND
  • High-speed store-and-forward devices, such as switches and routers, used in today's communication networks have a large amount of data passing through them. These devices typically include a set of line cards, which perform various operations within the communication networks. Communication between these line cards usually takes place over a backplane, which provides connectivity among the line cards, e.g., via dedicated point-to-point or switched communication paths. With advances in serial communication technologies, the preferred choice for high-speed backplanes today is to use one or more high-speed serial links (channels). High-speed serial data can be carried over either electrical backplanes or optical backplanes. If an optical backplane is used, the transmitting line card must convert electrical signals to optical signals and send the optical signals over fiber, and the destination line card must receive the optical signals from the fiber and reconvert them to electrical signals. The backplane may be used to switch data between line cards or may transport the data without switching. Serializers and deserializers are used, in conjunction with an encoding scheme, such as 8-bit to 10-bit encoding, to create a self-clocked high-speed serial electrical data stream.[0002]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 illustrates an exemplary system having multiple line cards connected through serial links over a point-to-point or switched backplane, according to one embodiment; [0003]
  • FIGS. 2A and B illustrate exemplary charts explaining how a frame is striped (interleaved) across data channels, according to one embodiment; [0004]
  • FIG. 3 illustrates an exemplary transmission module, according to one embodiment; [0005]
  • FIG. 4 illustrates an exemplary receiving module, according to one embodiment; [0006]
  • FIG. 5 illustrates an exemplary flowchart of the processing of a frame received over a plurality of channels, according to one embodiment; [0007]
  • FIG. 6 illustrates an exemplary parity byte calculation, according to one embodiment; [0008]
  • FIG. 7 illustrates an exemplary transmission module, according to one embodiment; [0009]
  • FIG. 8 illustrates an exemplary receiving module, according to one embodiment; [0010]
  • FIGS. 9A and B illustrate an exemplary flowchart of the processing of a frame received over a plurality of channels, according to one embodiment; and [0011]
  • FIG. 10 illustrates an exemplary frame and stripe format, according to one embodiment.[0012]
  • DETAILED DESCRIPTION
  • FIG. 1 illustrates an [0013] exemplary system 100 for transmitting data amongst various sources and destinations. The system may transmit the data using any number of protocols including Asynchronous Transfer Mode (ATM), Internet Protocol (IP), and Time Division Multiplexing (TDM). The data may be sent in variable length or fixed length blocks, such as cells, packets or frames. The communication lines used to transmit data may be fiber, copper, or other mediums. The system includes at least one store-and forward device 105, such as a router or packet switch. The store-and-forward device 105 includes multiple line cards connected together through serial links over a point-to-point or switched backplane. A plurality of ingress modules 110 are connected through a backplane 120 to a plurality of egress modules 130. The backplane 120 may be electrical or optical. The ingress modules 110 and the egress modules 130 are typically two sides of a line card. The line cards may be Ethernet (e.g., Gigabit, 10 Base T), ATM, Fibre channel, Synchronous Optical Network (SONET), and Synchronous Digital Hierarchy (SDH), amongst others. According to one embodiment, the data transmitted over the backplane is broken up into segments (e.g., frames). In a packet switch, the segment includes a single packet or a set of packets sent from a source line card to destination line card. Each segment has a defined maximum length which does not exceed a predetermined maximum length.
  • In order to meet bandwidth requirements the data (e.g., frames) being transmitted from one card to another card over the backplane is striped over multiple high-speed serial channels N between the cards. Striping is accomplished by breaking the data up and transmitting portions of the data over each of the N channels. For example, if the frame to be striped had W bytes, each channel would transmit W/N bytes. According to one embodiment, a first byte (e.g., byte 0) is transmitted over a first channel (e.g., channel 0), a second byte (e.g., byte 1) is transmitted over a second channel (e.g., channel 1), and so on until an N[0014] th byte (e.g., byte N−1) is transmitted over the Nth channel (e.g., channel N−1). Once each channel has transmitted a byte, the next byte (e.g., byte N) is transmitted on the first channel (e.g., channel 0). The process is repeated until all W bytes in the frame are transmitted. One way to look at this is that the data is broken up into W/N groups, with each group having N bytes. One byte (of the N bytes) from each group is then transmitted to each data channel (of the N data channels). The bytes belonging to a frame that travel on one specific channel is called a stripe. For example, the sequence of bytes 0, N, 2N, 3N, etc. would make up stripe 0 (e.g., the part of the frame traveling on channel 0). Note that some channels may have less than W/N bytes (or may have no data) and that the last group may have less than N bytes.
  • FIG. 2A illustrates an exemplary chart explaining how a frame including W bytes is striped (interleaved) across N data channels. The first column lists the byte number (0 to W−1), the second column lists the data channel number (0 to N−1) and the third column the group number (0 to (W/N)−1). As illustrated, [0015] bytes 0 to N−1 are striped over data channels 0 to N−1 to form a first group (group 0); bytes N to 2N−1 are striped over data channels 0 to N−1 to form a second group (group 1); and so on until bytes ((W/N)−1)N to W−1 are striped over data channels 0 to N−1 to form a last group (group (W/N)−1).
  • FIG. 2B illustrates an exemplary chart explaining how a frame including 26 bytes is striped (interleaved) across 6 data channels. This example would create 4 groups having 6 bytes each and a fifth group having two bytes. As illustrated, the first six bytes ([0016] bytes 0 to 5) are striped over each data channel (channels 0 to 5) to form a first group (group 0); the next six bytes (bytes 6 to 11) are striped over each data channel (channels 0 to 5) to form a second group (group 1); and so on until the last two bytes (bytes 24-25) are striped over the first two data channels (channels 0 and 1) to form a last group (group 4).
  • The various embodiments are in no way intended to be limited to the striping assignments discussed above. Rather, the bytes can be transmitted over the channels in any order. For example, [0017] byte 0 could be assigned to channel (N−1), byte 1 could be assigned to channel (N−2) and so on. Moreover, the various embodiments are not limited to transmitting the data byte by byte. The data could be transmitted bit by bit, sector by sector, block of bytes by block of bytes, or block of bits by block of bits, where a block can be defined by a user. Furthermore, the various embodiments are not limited to receiving frames that are organized as W bytes. Rather the frames could be organized by bits, sectors, or other ways.
  • Before transmission, a cyclic redundancy code (CRC) is computed for the frame. The CRC is inserted at the end of the frame and is transmitted along with the frame (data). Also, a separate CRC is computed for each stripe and is sent as part of the stripe. [0018]
  • FIG. 3 illustrates an [0019] exemplary transmission module 300, according to one embodiment. The transmission module 300 receives an input frame that is W bytes long at a CRC generator 310 that generates a CRC for the entire frame. The entire frame including the CRC is provided to a striper 320. The striper 320 divides the W bytes into N groups and selects the channel for each group to be transmitted over. That is, one group will be transmitted over each of the N channels. A CRC is generated for each stripe and is inserted at the end of the stripe by a CRC module 330. The stripe and the associated CRC for each channel are then provided to a transmitter 340 for transmitting over the backplane. There are a total of N transmitters 340, one for each data channel. In the event of channel failures (discussed later) the transmission module can be reconfigured to stripe the data over fewer channels. Up to P channels can be configured out of the system so that the data can be striped over a minimum of N-P channels.
  • The data sent over each channel (stripe) is received and buffered and the frame is recreated. A CRC is computed for the entire received frame and compared to the CRC that was transmitted with the frame in order to check for errors in the received frame. In addition, a CRC is computed for each channel and compared to the transmitted CRC for the channel to check for errors in the strip. If the computed CRC does not match with the transmitted CRC at the end of the stripe, the data within the stripe is deemed to be in error. The CRC error indication for each channel is ORed with an error signal from a corresponding physical receiver device. The physical receiver device indicates errors such as loss of signal etc. The ORed error signal is referred to as a channel error. If there is an error for the overall frame and/or one or more channel errors the frame is discarded. [0020]
  • Each channel is provided with a channel-specific error counter and an associated threshold register. When any channel-specific error count exceeds the threshold, there is a provision for an interrupt to be issued to the processor (or custom hardware) controlling the system. The software (or custom hardware) would set the value of the threshold such that in a given interval, if the number of errors exceed the value specified in the threshold register, it is likely that the channel has a permanent hardware problem. The system may then be shut down to replace one or more components to restore full throughput. If the error count does not exceed the threshold in the specified interval, it is likely that any errors that occurred were random in nature and may be ignored. At the end of each specified interval, the channel-specific error counts are reset. Note that the software (or custom hardware), needs to keep a rolling average of the per-channel error count. [0021]
  • Fixing a channel may cause down time on the system. According to one embodiment, the CPU (custom hardware) may reconfigure the system to stripe the data over fewer channels (N−1 channels, N−2 channels, etc.). In general, the data can be striped over any number of channels in the range N-P (minimum) to N (maximum). This allows for P channels to fail, and still have data transmission over the backplane and through the switching chips to the destination line card, albeit at a reduced bandwidth. The value of P is determined by the reduction of throughput that can be tolerated. [0022]
  • FIG. 4 illustrates an exemplary receive [0023] module 400, according to one embodiment. The receive module 400 includes N serial channel receivers 410 for receiving and buffering data over the N data channels. A CRC module 420 generates a CRC for each data channel (stripe). The stripes are provided to a destriper 430 that takes the N channels and converts it back into a W byte (or bit, etc.) frame. The destriper 430 forwards the frame to a CRC computation module 440 that computes the CRC for the entire frame and then compares the generated frame CRC to the transmitted frame CRC, and the generated stripe CRCs to the transmitted stripe CRCs to determine errors. Errors in any channel are recorded and compared to an error threshold. If the channel exceeds the error threshold it is configured out of the system until it can be repaired. Up to P channels can be configured out of the system so that the striped data can be transmitted using as little as N-P channels (minimum number of channels).
  • FIG. 5 illustrates an exemplary flowchart of the processing of a frame received over a plurality of channels. Initially, the received frame is checked for an error indicated by the frame-level CRC ([0024] 500). If there is no error in the frame level CRC (500 No), then each individual stripe is checked for errors based on the CRC or an error indication from the physical receiver device (505). If there is no error in any channel (505 No), then the frame is declared good and is sent out for further processing (510). If there are one or more channels with errors (505 Yes), then the error is an uncorrectable error. The frame is discarded and the corresponding per-channel error counters are incremented (515).
  • If there was a frame level CRC error ([0025] 500 Yes), then a determination is made as to whether there are any channel errors (520). If there are no channel errors (520 No), then the frame is discarded and the frame error count is incremented (525). If there were channel errors (520 Yes) then the frame is discarded, the frame error count is incremented, and the channel error count(s) are incremented (530). After any incrementing of channel errors (515, 525, 530) the error count of each channel is compared to error thresholds (535). If the threshold is not exceeded (535 No) for every channel then the process is complete. If the threshold is exceeded for at least one channel (535 Yes) an interrupt is issued to a CPU, or custom hardware (540). The CPU (custom hardware) receives the interrupt indicating that the threshold(s) has been exceeded, and makes a determination as to whether there are enough usable channels remaining to support the system (545). That is, if we take the faulty channel out of service are the number of channels remaining at least equal to the minimum number of channels (N-P) required to meet the bandwidth requirements of the system? If the number of channels remaining after deactivating the faulty channels is less than the minimum number of channels (545 No), then the system is shut down for servicing (550). If the number of channels remaining after deactivating the faulty channels is still at least equal to the minimum number of channels (545 Yes), then the system is reconfigured to utilize only working channels (555).
  • The exemplary process flow described above is not the only embodiment but is merely an example. Numerous modifications could be made to the process flow (e.g., the order could be rearranged and/or individual operations could be combined or split apart). [0026]
  • Discarding an entire frame because of a single channel failure is not desirable. This situation may be perpetuated if multiple channels continually fail at different times, with no channel failing enough to exceed the threshold. In this case numerous frames are discarded, but none of the channels is deactivated. According to one embodiment, the CPU (custom hardware) may switch from a graceful degradation mode (the mode described above where if a channel(s) has multiple errors that exceed a threshold they can be configured out of the system) to a single channel error correction mode. The single channel error correction mode allows a single channel error to be corrected. [0027]
  • In this embodiment, the N serial channels are divided into M data channels and K parity channels. The parity channels are used to send parity information associated with the data being sent. If the data is being striped by bytes then the parity will be transmitted by bytes as well. The parity bytes (or other grouping) are used for correction of errors. The M data channels are divided into groups and each group is associated with a parity channel. As there are K parity channels it follows that there will be K groups of data channels and each group will have M/K channels. For each byte within a group (e.g., M/K channels) there is an associated parity byte in the corresponding parity channel. In a preferred embodiment, each parity byte is generated by XORing (that is, computing the logical Exclusive-OR operation of) each bit of each associated byte in the data channels within the group. For example, [0028] bit 0 of the parity byte is the XOR of the bits in position zero of all the bytes in that group, bit 1 is the XOR of the bits in position one of all the bytes in that group, and so on.
  • FIG. 6 illustrates an exemplary parity byte calculation for a group of 3 data channels that are utilized to transmit a plurality of bytes. The parity byte is calculated by XORing each bit of the data bytes. The various embodiments are in no way intended to be limited to generating the parity data by XORing associated bits together. Rather, there are multiple methods for generating parity data (e.g., bits, bytes, etc.). [0029]
  • If there is channel error on only a single data channel (e.g., error is confined to a single stripe) associated with a group of channels, the data for that channel (stripe) can be recovered. The data for the channel is recovered by XORing the data from all the other channels (data and parity) in the group. The recovered data replaces the received data for that channel (stripe). Once the data is recovered for the errored channel (stripe), the frame is reassembled from all the received channels (stripes) and the corrected channel (stripe). A second CRC check is then performed on the frame. If this CRC check passes, the entire frame is deemed error-free and accepted by the receiver. An error count is incremented for the data channel having the failure. If the error count exceeds some predefined threshold, action may be taken to fix the channel (discussed later). [0030]
  • If there is only a single channel error on a parity channel associated with a group of channels, the received data within the group is processed normally (as if there were no error). An error count is incremented for the parity channel having the failure. If the error count exceeds some predefined threshold, action may be taken to fix the channel (discussed later). [0031]
  • It should be noted that if there is a channel error (data or parity as discussed above) that there should also be a frame level error. If there is a channel error but no frame level error the frame will be discarded and the channel error count is incremented. [0032]
  • If errors are detected in more than one channel (whether data or parity) in a group, the error is uncorrectable. The frame is discarded and a frame error counters are incremented. If the error count exceeds some predefined threshold, action may be taken to fix the channel. [0033]
  • If the frame level CRC indicates an error, but there are no channel errors detected in any group, then it is an uncorrectable error. The frame is discarded and a frame error counter is incremented. [0034]
  • The above descriptions apply for each group of channels. The actions in one group are independent of the actions required (or taken) in other groups. [0035]
  • FIG. 7 illustrates an [0036] exemplary transmission module 700, according to one embodiment. The transmission module 700 is much like the transmission module 300 of FIG. 3 with the exception that the N serial channels are divided into M data channels and K parity channels. A CRC generator 710 receives a frame and generates a CRC for the entire frame. The entire frame including the CRC is provided to a striper 720. The striper 720 divides the frame into M groups and selects the channel for each group to be transmitted over. A CRC is generated for each stripe and is inserted at the end of the stripe by a CRC module 730. The stripe and the associated CRC for each channel are then provided to a transmitter 740 for transmitting over the backplane. Each parity channel is associated with a group of M/K data channels. The M/K data channels associated with each parity channel are provided to a parity generator 750 that generates the parity data associated with the group of data channels. As previously mentioned, if the data channels are transmitting data byte by byte the parity data will be a parity byte. In a preferred embodiment, the parity generator 750 is an XOR gate for each bit of the data being transmitted at a time. The XOR gate receives an associated bit from each of the group of M/K data channels. A CRC is generated for each parity channel stripe and is inserted at the end of the parity stripe by a CRC module 760. The parity stripe and the associated CRC for each parity channel are then provided to a transmitter 770 for transmitting over the backplane.
  • It should be noted that the [0037] transmitters 740 and the transmitters 770, as well as the CRC modules 730 and the CRC modules 760, are illustrated separately for convenience of pointing out the difference between the parity channels and the data channels. However, it should in no way be construed to require different CRC modules and transmitters for parity channels and data channels. Rather, the CRC modules and the transmitters could be the same. In fact, according to one embodiment, the transmission module 700 can easily be reconfigured to have more data channels or more parity channels.
  • FIG. 8 illustrates an exemplary receive [0038] module 800, according to one embodiment. The receive module 800 is much like the receive module 400 of FIG. 4 with the exception that parity bits are received and used for regeneration (correction) of errored stripes. The receive module 800 includes M serial channel receivers 810 for receiving and buffering data over the M data channels. A CRC module 820 generates a CRC for each data channel (stripe). The stripes are provided to a destriper 830 that takes the M channels and converts it back into a frame. The destriper 830 forwards the frame to a CRC computation module 840 that computes the CRC for the entire frame and then compares the generated frame CRC to the transmitted frame CRC, and the generated stripe CRCs to the transmitted stripe CRCs to determine errors. The receive module 800 also includes K serial channel receivers 850 for receiving parity data. A CRC module 860 generates a CRC for each parity channel (stripe). Each parity channel and the data channels that are associated with the parity channel as part of a group, are provided to error correction modules 870. There is one error correction module 870 for each of the M/K groups. The error correction modules can recreate data lost on an individual channel within a group. As previously discussed, the data can be recreated by XORing all of the other channels (data and parity) within the group. Accordingly, in a preferred embodiment, the error correction modules 870 are XOR gates. The error correction modules 870 provide the corrected data channels to the destriper 830.
  • It should be noted that the [0039] receivers 810 and the receivers 850, as well as the CRC modules 820 and the CRC modules 860, are illustrated separately for convenience of pointing out the difference between the parity channels and the data channels. However, it should in no way be construed to require different receivers and CRC modules for parity channels and data channels. Rather, the receivers and CRC modules could be the same type of receivers. In fact, according to one embodiment, the receiver module 800 can easily be reconfigured to have more data channels or more parity channels.
  • FIGS. 9A & B illustrate an exemplary flowchart of the processing of a frame received over a plurality of channels. The flowchart of FIGS. 9A & B is similar to the flowchart of FIG. 5 with the exception that a switch to the single channel error correction mode (described above) from the graceful degradation mode (described above) is possible. The mode of the system may be switched at any point in time either automatically or by an operator. One possible occasion for the switch in mode would be the case where the error threshold has not been exceeded for any particular channel but that there have been multiple single channel failures causing multiple frames to be discarded. In this case the system could reconfigure itself to include parity channels (thus reducing the bandwidth available for the data) so that the errors could be self-correcting. [0040]
  • As previously mentioned, errors within a single channel per group can be corrected in the single channel error correction mode. Thus, the process will require a determination of the mode of the system. Referring to FIGS. 9A and B, the channel error determination ([0041] 520) is modified to be a determination as to whether there was a single channel error (900). If the errors were not limited to a single channel error (900 No), the frame is discarded (905). Also, the frame error count is incremented and the channel error counts for the errored channels are incremented (905). If the error was limited to a single channel error (900 Yes), a determination is made as to whether the system is in single-channel error correction mode (910). If it is determined that the system is not in single channel error correction mode (910 No) the frame is discarded, the frame error count is incremented and the channel error count for the errored channel is incremented (920). If the errors were limited to a single channel per group (910 Yes), a determination is made as to whether the channel having the error is the parity channel (930). If the channel error was not in the parity channel (930 No), the data is the channel is recreated (corrected) by XORing all the other stripes within that group, the frame is generated using the corrected stripe, the frame is sent out, and the channel error count for the errored channel is incremented (940). If the channel error was in the parity channel (930 Yes), the frame is sent out, and the parity channel error count is incremented (950).
  • After any incrementing of channel errors, whether the frames were discarded ([0042] 920, 940) or the frames were transmitted 950, the error count is compared to the threshold (535) and if exceeded the CPU receives an interrupt. However, as the single channel error correction mode cannot deactivate and reconfigure channels a determination will have to be made as to which mode the system is in before proceeding. Accordingly, after the CPU (custom hardware) receives an interrupt (540) indicating that a threshold(s) has been exceeded a determination will be made as to whether the system is in graceful degradation mode (960). If the system is not in graceful degradation mode (960 No), the CPU (custom hardware) will perform the necessary actions (970). The necessary actions may be to shut down and repair the system now, tag the system for shut down/repair in the future, or switch to graceful degradation mode. If the system was in graceful degradation mode (960 Yes), the system proceeds to determine if the number of usable channels is sufficient 545.
  • According to one embodiment, there is also a provision for test data to be sent over one or more channels. Errors detected on the received data are recorded per-channel. This can be used by the software (or custom hardware) to test individual channels before deciding the number of channels to use, or as a periodic self-diagnostic feature. [0043]
  • The exemplary process flow described above is not the only embodiment but is merely an example. Numerous modifications could be made to the process flow (e.g., the order could be rearranged and/or individual operations could be combined or split apart). [0044]
  • According to another embodiment, the system may act in both the single channel error correction mode and the graceful degradation mode at the same time. This embodiment is similar to the embodiment of FIGS. 9A and B with the exception that the [0045] graceful degradation determination 960 and associated perform action 970, as well as single channel error correction determination 910 and associated discard frame 920 are no longer required. The system will utilize both data and parity channels so that single channel error corrections can be made. In the event that one or more of the channels exceed the associated thresholds 535, the CPU (custom hardware) will make a determination if there are sufficient channels available to deactivate the channel or channels 545. In this embodiment, the determination is not just made on data channels as in the embodiment of FIGS. 9A and B (at least N-P channels). In this embodiment, a determination will have to be made as to whether enough data channels are available with enough associated parity channels to be able to handle single channel error detection. Therefore a minimum number of data channels and associated parity channels need to be defined.
  • According to one embodiment, it would be possible to continue processing if there were channels to support the data but not the parity. However, the continued processing would be with limited or no single error correction. If the determination is that enough channels do not exist ([0046] 545 No) then the system is shut down for servicing (550). If the determination is made that there are enough channels available, the system is reconfigured to use only the working channels (555).
  • FIG. 10 illustrates an exemplary frame and stripe format. In this example, high speed serial electrical channels are used over a backplane, and switched using a crossbar. There are total of eight serial channels with seven channels used for data and one channel used for parity. All seven data channels belong to one group so that only a single parity channel is required to provide error correction for the seven data channels. The encoding scheme used for framing is the industry-standard 8-bit to 10-bit encoding. It provides a “Start of Frame Delimiter” (SFD) symbol, an “End of Frame Delimiter” (EFD) symbol, and an “Idle Character” symbol. The [0047] input data 1000 is broken into frames, and each frame is appended with a frame-level CRC 1010. The frame-level CRC 1010 is computed over all the data bytes (or bits, etc.) of the frame. The data is striped over the seven data channels 1020, byte by byte. The data can be striped across the channels in other manners, such as bit by bit or group by group. A parity channel 1030 transmits parity data for the group which is the XOR of all the data from the seven data channels. According to a preferred embodiment, the parity data is derived by the bit-wise XOR of the corresponding bytes of each of the seven data channels.
  • Each of the channels initially transmits a first stripe for the [0048] frame 1040 that is the start of frame symbol. This is followed by the data stripe (data for the seven data channels 1020 and parity data for the parity channel 1030) 1050. A last data stripe 1060 is the CRC computed for each stripe. A last stripe for the frame 1070 is the end of frame symbol. If only a single channel fails, the channel can be recreated (fixed) by XORing all the other channels in the group. If one of the channels records enough errors to cross the threshold level, the system could either shut down for servicing (now or later) or could reconfigure itself to a graceful degradation mode in which case that channel would be reconfigured out of the system and the data would be transmitted over the remaining channels.
  • According to an embodiment in which the system can be reconfigured between single channel error correction mode and graceful degradation mode, if a single channel failed once the system was reconfigured to remove the channel (whether a data channel or the parity channel) the data would continue be transmitted over seven channels but would have no parity channel and thus no error correction. [0049]
  • According to an embodiment in which the system supports both single channel error correction mode and graceful degradation mode, the system can continue to correct errors while at the same time reconfiguring to deactivate faulty channels. For example, if a data channel failed the data could be striped over the remaining six data channels and the parity channel could continue be used for error correction. However, if the system needed at least [0050] 6 data channels to meet bandwidth requirements if a second channel failed the parity channel would have to be deactivated which would eliminate the error correction capability at that point.
  • Although the detailed description has been illustrated by reference to specific embodiments, various changes and modifications may be made. Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment. [0051]
  • Different implementations may feature different combinations of hardware, firmware, and/or software. For example, some implementations feature computer program products disposed on computer readable mediums. The programs include instructions for causing processors to perform techniques described above. [0052]
  • The various embodiments are intended to be protected broadly within the spirit and scope of the appended claims. [0053]

Claims (31)

1. An apparatus, comprising:
a transmission module to split a data segment into a plurality of data stripes and transmit each data strip over an associated serial channel;
a reception module to receive the plurality of data stripes over the associated serial channels and track a number of errors per channel; and
a controller to control the operation of the apparatus, said controller to deactivate a serial channel and reconfigure said transmission module and said reception module to utilize remaining data channels for striping data if the number of errors in the serial channel exceeds a threshold.
2. The apparatus of claim 1, wherein said transmission module generates an error code for each stripe and transmits the error code along with the data stripe.
3. The apparatus of claim 2, wherein the error code is a cyclic redundancy code (CRC).
4. The apparatus of claim 1, wherein said reception module checks for errors by computing an error code for each data stripe and comparing it to a transmitted error code.
5. The apparatus of claim 1, wherein said reception module checks for errors in physical receiving devices for each serial channel.
6. The apparatus of claim 1, wherein said controller reconfigures the apparatus to perform error correction.
7. The apparatus of claim 1, wherein said controller reconfigures said transmission module and said reception module to utilize a subset of the serial channels for data stripes and a subset of the serial channels for parity stripes, wherein the data stripes are organized into at least one group and each group has an associated parity stripe to be utilized for error correction, the parity stripe being generated based on the data stripes within the group.
8. The apparatus of claim 7, wherein said transmission module generates the parity stripe for a group by XORing each of the associated data stripes within the group.
9. The apparatus of claim 7, wherein said reception module recreates a data stripe containing errors based on remaining data stripes and the parity stripes within an associated group, if errors are limited to a single data stripe within the group.
10. The apparatus of claim 9, wherein said reception module recreates the data stripe by XORing each of the other data stripes and the parity stripe within the group.
11. A method, comprising:
dividing a data segment into a plurality of data stripes and associating each data stripe with one of a plurality of serial channels;
transmitting the data stripes from a first card to a second card over the associated serial channels;
receiving the data stripes at the second card;
detecting errors in the data stripes;
tracking number of errors per serial channel; and
deactivating a serial channel and reconfiguring remaining serial channels for striping data if the number of errors in the serial channel exceeds a threshold.
12. The method of claim 11, further comprising creating an error code for each data stripe and appending the error code to the end of the data stripe.
13. The method of claim 12, wherein the error code is a cyclic redundancy code (CRC).
14. The method of claim 11, further comprising computing an error code for the data segment and appending it at the end of the data segment.
15. The method of claim 11, wherein said detecting includes computing an error code for each data stripe and comparing it to a transmitted error code within each data stripe.
16. The method of claim 11, further comprising discarding the data segment if an error is detected.
17. The method of claim 11, further comprising recreating the data segment from the data stripes if no errors are detected.
18. The method of claim 11, wherein a minimum number of serial channels are required to provide an appropriate bandwidth for striping data segments.
19. The method of claim 18, further comprising terminating operation if minimum number of serial channels are not available.
20. The method of claim 11, further comprising
organizing the data stripes into at least one group;
generating a parity stripe for each group based on the data stripes within the group, wherein each parity stripe is utilized for error correction within the group;
associating each parity stripe with one of the serial channels;
transmitting the parity stripes from the first card to the second card over the associated serial channels;
receiving the parity stripes at the second card; and
recreating a data stripe containing errors by utilizing the associated parity stripe, wherein said recreating is limited to a single data stripe per group containing errors.
21. The method of claim 20, wherein said generating includes generating the parity stripe by XORing each of the data stripes within the group.
22. The method of claim 20, wherein said generating includes generating the parity stripe by bitwise XORing each of the data stripes within the group.
23. The method of claim 20, further comprising discarding a data segment if more than one data stripe in a group contains errors.
24. A store and forward device, comprising:
a plurality of interface cards to receive and transmit data;
a backplane connecting the plurality of cards together via a plurality of serial channels;
a transmission module to split a data segment into a plurality of data stripes and transmit each data stripe over an associated serial channel;
a reception module to receive the plurality of data stripes over the associated serial channels and track a number of errors per channel; and
a controller to control the operation of the apparatus, wherein said controller may deactivate a serial channel and reconfigure said transmission module and said reception module to utilize a remaining data channels for striping data if the number of errors in the serial channel exceeds a threshold.
25. The device of claim 24, wherein
said transmission module generates an error code for each stripe and transmits the error code along with the stripe; and
said reception module checks for errors by computing an error code for each stripe and comparing it to a transmitted error code.
26. The device of claim 24, wherein said controller reconfigures said transmission module and said reception module to utilize a subset of the serial channels for data stripes and a subset of the serial channels for parity stripes, wherein the data stripes are organized into at least one group and each group has an associated parity stripe to be utilized for error correction, the parity stripe being generated based on the data stripes within the group.
27. The device of claim 26, wherein said reception module recreates a data stripe containing errors based on remaining data stripes and the parity stripes within an associated group, if errors are limited to a single data stripe within the group.
28. The device of claim 24, wherein said interface cards are Ethernet cards.
29. The device of claim 24, wherein said backplane is an optical backplane.
30. A computer program product, disposed on a computer readable medium, the program including instructions for causing a processor to:
track a number of errors per serial channel carrying stripes of data segments; and
deactivate a serial channel and reconfigure remaining serial channels for striping data if the number of errors in the serial channel exceeds a threshold.
31. The program of claim 30, further comprising instructions that cause the processor to divide a data segment into stripes for transmission via the serial channels.
US10/396,287 2002-03-25 2003-03-24 Graceful degradation of serial channels Abandoned US20030229844A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US10/396,287 US20030229844A1 (en) 2002-03-25 2003-03-24 Graceful degradation of serial channels

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US36763002P 2002-03-25 2002-03-25
US10/396,287 US20030229844A1 (en) 2002-03-25 2003-03-24 Graceful degradation of serial channels

Publications (1)

Publication Number Publication Date
US20030229844A1 true US20030229844A1 (en) 2003-12-11

Family

ID=29715157

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/396,177 Expired - Fee Related US7246303B2 (en) 2002-03-25 2003-03-24 Error detection and recovery of data in striped channels
US10/396,287 Abandoned US20030229844A1 (en) 2002-03-25 2003-03-24 Graceful degradation of serial channels

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/396,177 Expired - Fee Related US7246303B2 (en) 2002-03-25 2003-03-24 Error detection and recovery of data in striped channels

Country Status (1)

Country Link
US (2) US7246303B2 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060047993A1 (en) * 2004-09-02 2006-03-02 Benhase Michael T Apparatus, system, and method for error checking and recovery of transmitted data in a SCSI enviroment
US20070179733A1 (en) * 2006-02-01 2007-08-02 International Business Machines Corporation Methods and apparatus for testing a link between chips
US20080024949A1 (en) * 2006-07-28 2008-01-31 Fujitsu Network Communications, Inc. Method and System for Automatic Attempted Recovery of Equipment from Transient Faults
US7460563B1 (en) * 2002-08-31 2008-12-02 Cisco Technology, Inc. Determination of interleaving channels in a serial interleaver
US20100199018A1 (en) * 2009-02-03 2010-08-05 Fujitsu Limited Data transfer system, data transmitting apparatus, data receiving apparatus, and data transfer method
US20150131673A1 (en) * 2006-02-28 2015-05-14 Cisco Technology, Inc. Fabric channel control apparatus and method
US20150193291A1 (en) * 2014-01-09 2015-07-09 Phison Electronics Corp. Error handling method, memory storage device and memory controlling circuit unit
CN104810059A (en) * 2014-01-23 2015-07-29 群联电子股份有限公司 Error treatment method, memory storage apparatus and memory control circuit unit
WO2020008929A1 (en) * 2018-07-06 2020-01-09 日本電信電話株式会社 Transmission device, reception device, and transmittal system
CN111736130A (en) * 2020-07-01 2020-10-02 成都汇蓉国科微系统技术有限公司 Configurable block type matrix transposition system and method based on FPGA

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7570654B2 (en) * 2003-12-22 2009-08-04 Intel Corporation Switching device utilizing requests indicating cumulative amount of data
US7623524B2 (en) * 2003-12-22 2009-11-24 Intel Corporation Scheduling system utilizing pointer perturbation mechanism to improve efficiency
US7324541B2 (en) * 2003-12-22 2008-01-29 Intel Corporation Switching device utilizing internal priority assignments
US7457248B1 (en) * 2004-02-10 2008-11-25 Cisco Technology, Inc. Graceful shutdown of network resources in data networks
US20050207436A1 (en) * 2004-03-18 2005-09-22 Anujan Varma Switching device based on aggregation of packets
JP4260688B2 (en) * 2004-06-09 2009-04-30 富士通株式会社 Data transmission device, data transmission / reception system, data transmission device control method, and data transmission / reception system control method
US7646772B2 (en) * 2004-08-13 2010-01-12 Cisco Technology, Inc. Graceful shutdown of LDP on specific interfaces between label switched routers
JP2006323434A (en) * 2005-05-17 2006-11-30 Nec Corp Data processor and memory correction method therefor
US7929577B2 (en) * 2005-10-13 2011-04-19 Via Technologies, Inc. Method and apparatus for packet error detection
TWI290284B (en) * 2005-10-13 2007-11-21 Via Tech Inc Method and electronic device of packet error detection on PCI express bus link
KR100995621B1 (en) * 2006-02-24 2010-11-22 후지쯔 가부시끼가이샤 Error control device
US7801121B1 (en) * 2006-04-20 2010-09-21 Altera Corporation Serial communications systems with cyclic redundancy checking
US20080159145A1 (en) * 2006-12-29 2008-07-03 Raman Muthukrishnan Weighted bandwidth switching device
US20090024900A1 (en) * 2007-07-18 2009-01-22 Cisco Technology, Inc. Cyclic redundancy checking in lane-based communications
FR2924553B1 (en) * 2007-12-04 2010-04-23 Thales Sa METHOD FOR IMPROVING THE INTEGRITY OF COMMUNICATION MEANS
JP4798164B2 (en) * 2008-04-02 2011-10-19 ソニー株式会社 Transmitting apparatus and method, receiving apparatus and method, and program
GB2460434A (en) * 2008-05-30 2009-12-02 Roke Manor Research Encoding data with parity information and transmitting it over at least three paths to enable data reconstruction in the event of a path failure
US20100262887A1 (en) * 2009-04-13 2010-10-14 Lockheed Martin Corporation High Integrity Data Network System and Method
US8539296B2 (en) * 2009-05-11 2013-09-17 Empire Technology Development Llc Coordination of packet and acknowledgment retransmissions
JP6039699B2 (en) * 2012-07-23 2016-12-07 株式会社日立製作所 Storage system and data management method
EP3202858A1 (en) * 2016-02-02 2017-08-09 Kronos International, Inc. The preparation of matt paints and printing inks
US11319834B2 (en) * 2018-01-25 2022-05-03 Raytheon Technologies Corporation On-board estimator sensor drift detection in engine control

Citations (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4092732A (en) * 1977-05-31 1978-05-30 International Business Machines Corporation System for recovering data stored in failed memory unit
US4331956A (en) * 1980-09-29 1982-05-25 Lovelace Alan M Administrator Control means for a solid state crossbar switch
US4335458A (en) * 1978-05-02 1982-06-15 U.S. Philips Corporation Memory incorporating error detection and correction
US4695999A (en) * 1984-06-27 1987-09-22 International Business Machines Corporation Cross-point switch of multiple autonomous planes
US4775987A (en) * 1984-03-20 1988-10-04 British Telecommunications Plc Broadband digital transmission systems
US5065396A (en) * 1990-01-02 1991-11-12 At&T Bell Laboratories Inverse multiplexer and demultiplexer techniques
US5127000A (en) * 1989-08-09 1992-06-30 Alcatel N.V. Resequencing system for a switching node
US5191578A (en) * 1990-06-14 1993-03-02 Bell Communications Research, Inc. Packet parallel interconnection network
US5260935A (en) * 1991-03-01 1993-11-09 Washington University Data packet resequencer for a high speed data switch
US5274785A (en) * 1992-01-15 1993-12-28 Alcatel Network Systems, Inc. Round robin arbiter circuit apparatus
US5442752A (en) * 1992-01-24 1995-08-15 International Business Machines Corporation Data storage method for DASD arrays using striping based on file length
US5483523A (en) * 1993-08-17 1996-01-09 Alcatel N.V. Resequencing system
US5581566A (en) * 1995-01-06 1996-12-03 The Regents Of The Univ. Of California Office Of Technology Transfer High-performance parallel interface to synchronous optical network gateway
US5649157A (en) * 1995-03-30 1997-07-15 Hewlett-Packard Co. Memory controller with priority queues
US5682493A (en) * 1993-10-21 1997-10-28 Sun Microsystems, Inc. Scoreboard table for a counterflow pipeline processor with instruction packages and result packages
US5832278A (en) * 1997-02-26 1998-11-03 Advanced Micro Devices, Inc. Cascaded round robin request selection method and apparatus
US5848434A (en) * 1996-12-09 1998-12-08 Intel Corporation Method and apparatus for caching state information within a directory-based coherency memory system
US5860097A (en) * 1996-09-23 1999-01-12 Hewlett-Packard Company Associative cache memory with improved hit time
US5859835A (en) * 1996-04-15 1999-01-12 The Regents Of The University Of California Traffic scheduling system and method for packet-switched networks
US5978951A (en) * 1997-09-11 1999-11-02 3Com Corporation High speed cache management unit for use in a bridge/router
US6055625A (en) * 1993-02-16 2000-04-25 Fujitsu Limited Pipeline computer with a scoreboard control circuit to prevent interference between registers
US6061345A (en) * 1996-10-01 2000-05-09 Electronics And Telecommunications Research Institute Crossbar routing switch for a hierarchical crossbar interconnection network
US6167508A (en) * 1998-06-02 2000-12-26 Compaq Computer Corporation Register scoreboard logic with register read availability signal to reduce instruction issue arbitration latency
US6170032B1 (en) * 1996-12-17 2001-01-02 Texas Instruments Incorporated Priority encoder circuit
US6188698B1 (en) * 1997-12-31 2001-02-13 Cisco Technology, Inc. Multiple-criteria queueing and transmission scheduling system for multimedia networks
US6282686B1 (en) * 1998-09-24 2001-08-28 Sun Microsystems, Inc. Technique for sharing parity over multiple single-error correcting code words
US6321306B1 (en) * 1999-11-09 2001-11-20 International Business Machines Corporation High performance multiprocessor system with modified-unsolicited cache state
US6359891B1 (en) * 1996-05-09 2002-03-19 Conexant Systems, Inc. Asynchronous transfer mode cell processing system with scoreboard scheduling
US20020039388A1 (en) * 2000-02-29 2002-04-04 Smart Kevin J. High data-rate powerline network system and method
US6408378B2 (en) * 1998-04-03 2002-06-18 Intel Corporation Multi-bit scoreboarding to handle write-after-write hazards and eliminate bypass comparators

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08163091A (en) 1994-11-29 1996-06-21 Canon Inc Radio communication ststem
US6934904B2 (en) * 2001-04-30 2005-08-23 Sun Microsystems, Inc. Data integrity error handling in a redundant storage array
US7171594B2 (en) * 2002-03-06 2007-01-30 Hewlett-Packard Development Company, L.P. Pausing a transfer of data
US20040107398A1 (en) * 2002-07-02 2004-06-03 Johnson Ian Robert Error detection in received data transmissions

Patent Citations (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4092732A (en) * 1977-05-31 1978-05-30 International Business Machines Corporation System for recovering data stored in failed memory unit
US4335458A (en) * 1978-05-02 1982-06-15 U.S. Philips Corporation Memory incorporating error detection and correction
US4331956A (en) * 1980-09-29 1982-05-25 Lovelace Alan M Administrator Control means for a solid state crossbar switch
US4775987A (en) * 1984-03-20 1988-10-04 British Telecommunications Plc Broadband digital transmission systems
US4695999A (en) * 1984-06-27 1987-09-22 International Business Machines Corporation Cross-point switch of multiple autonomous planes
US5127000A (en) * 1989-08-09 1992-06-30 Alcatel N.V. Resequencing system for a switching node
US5065396A (en) * 1990-01-02 1991-11-12 At&T Bell Laboratories Inverse multiplexer and demultiplexer techniques
US5191578A (en) * 1990-06-14 1993-03-02 Bell Communications Research, Inc. Packet parallel interconnection network
US5260935A (en) * 1991-03-01 1993-11-09 Washington University Data packet resequencer for a high speed data switch
US5274785A (en) * 1992-01-15 1993-12-28 Alcatel Network Systems, Inc. Round robin arbiter circuit apparatus
US5442752A (en) * 1992-01-24 1995-08-15 International Business Machines Corporation Data storage method for DASD arrays using striping based on file length
US6055625A (en) * 1993-02-16 2000-04-25 Fujitsu Limited Pipeline computer with a scoreboard control circuit to prevent interference between registers
US5483523A (en) * 1993-08-17 1996-01-09 Alcatel N.V. Resequencing system
US5682493A (en) * 1993-10-21 1997-10-28 Sun Microsystems, Inc. Scoreboard table for a counterflow pipeline processor with instruction packages and result packages
US5581566A (en) * 1995-01-06 1996-12-03 The Regents Of The Univ. Of California Office Of Technology Transfer High-performance parallel interface to synchronous optical network gateway
US5649157A (en) * 1995-03-30 1997-07-15 Hewlett-Packard Co. Memory controller with priority queues
US5859835A (en) * 1996-04-15 1999-01-12 The Regents Of The University Of California Traffic scheduling system and method for packet-switched networks
US6359891B1 (en) * 1996-05-09 2002-03-19 Conexant Systems, Inc. Asynchronous transfer mode cell processing system with scoreboard scheduling
US5860097A (en) * 1996-09-23 1999-01-12 Hewlett-Packard Company Associative cache memory with improved hit time
US6061345A (en) * 1996-10-01 2000-05-09 Electronics And Telecommunications Research Institute Crossbar routing switch for a hierarchical crossbar interconnection network
US5848434A (en) * 1996-12-09 1998-12-08 Intel Corporation Method and apparatus for caching state information within a directory-based coherency memory system
US6170032B1 (en) * 1996-12-17 2001-01-02 Texas Instruments Incorporated Priority encoder circuit
US5832278A (en) * 1997-02-26 1998-11-03 Advanced Micro Devices, Inc. Cascaded round robin request selection method and apparatus
US5978951A (en) * 1997-09-11 1999-11-02 3Com Corporation High speed cache management unit for use in a bridge/router
US6188698B1 (en) * 1997-12-31 2001-02-13 Cisco Technology, Inc. Multiple-criteria queueing and transmission scheduling system for multimedia networks
US6408378B2 (en) * 1998-04-03 2002-06-18 Intel Corporation Multi-bit scoreboarding to handle write-after-write hazards and eliminate bypass comparators
US6167508A (en) * 1998-06-02 2000-12-26 Compaq Computer Corporation Register scoreboard logic with register read availability signal to reduce instruction issue arbitration latency
US6282686B1 (en) * 1998-09-24 2001-08-28 Sun Microsystems, Inc. Technique for sharing parity over multiple single-error correcting code words
US6321306B1 (en) * 1999-11-09 2001-11-20 International Business Machines Corporation High performance multiprocessor system with modified-unsolicited cache state
US20020039388A1 (en) * 2000-02-29 2002-04-04 Smart Kevin J. High data-rate powerline network system and method

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7460563B1 (en) * 2002-08-31 2008-12-02 Cisco Technology, Inc. Determination of interleaving channels in a serial interleaver
US7376863B2 (en) 2004-09-02 2008-05-20 International Business Machines Corporation Apparatus, system, and method for error checking and recovery of transmitted data in a SCSI environment
US20060047993A1 (en) * 2004-09-02 2006-03-02 Benhase Michael T Apparatus, system, and method for error checking and recovery of transmitted data in a SCSI enviroment
US20070179733A1 (en) * 2006-02-01 2007-08-02 International Business Machines Corporation Methods and apparatus for testing a link between chips
US7324913B2 (en) * 2006-02-01 2008-01-29 International Business Machines Corporation Methods and apparatus for testing a link between chips
US20080133169A1 (en) * 2006-02-01 2008-06-05 Scott Douglas Clark Methods and apparatus for testing a link between chips
US9654419B2 (en) * 2006-02-28 2017-05-16 Cisco Technology, Inc. Fabric channel control apparatus and method
US20150131673A1 (en) * 2006-02-28 2015-05-14 Cisco Technology, Inc. Fabric channel control apparatus and method
US20080024949A1 (en) * 2006-07-28 2008-01-31 Fujitsu Network Communications, Inc. Method and System for Automatic Attempted Recovery of Equipment from Transient Faults
US7664980B2 (en) * 2006-07-28 2010-02-16 Fujitsu Limited Method and system for automatic attempted recovery of equipment from transient faults
US20100199018A1 (en) * 2009-02-03 2010-08-05 Fujitsu Limited Data transfer system, data transmitting apparatus, data receiving apparatus, and data transfer method
US9229798B2 (en) * 2014-01-09 2016-01-05 Phison Electronics Corp. Error handling method, memory storage device and memory controlling circuit unit
US20150193291A1 (en) * 2014-01-09 2015-07-09 Phison Electronics Corp. Error handling method, memory storage device and memory controlling circuit unit
CN104810059A (en) * 2014-01-23 2015-07-29 群联电子股份有限公司 Error treatment method, memory storage apparatus and memory control circuit unit
WO2020008929A1 (en) * 2018-07-06 2020-01-09 日本電信電話株式会社 Transmission device, reception device, and transmittal system
JP2020010180A (en) * 2018-07-06 2020-01-16 日本電信電話株式会社 Transmission device, reception device and transmission system
CN111736130A (en) * 2020-07-01 2020-10-02 成都汇蓉国科微系统技术有限公司 Configurable block type matrix transposition system and method based on FPGA

Also Published As

Publication number Publication date
US20040017778A1 (en) 2004-01-29
US7246303B2 (en) 2007-07-17

Similar Documents

Publication Publication Date Title
US7246303B2 (en) Error detection and recovery of data in striped channels
US20070005248A1 (en) Data reconstruction in link-based interconnects
JP5696957B2 (en) Multilane transmission apparatus and multilane transmission method
EP0642228B1 (en) Apparatus for transmitting and receiving interleaved error correction coded data on a transmission frame
CA2278820C (en) Forward error correction for high speed optical transmission systems
EP3013008B1 (en) Transmitting data traffic in a communication network
EP2975858B1 (en) Method for processing data in the ethernet, physical layer chip and ethernet device
JP7282187B2 (en) FLEXIBLE ETHERNET COMMUNICATION METHOD AND NETWORK DEVICE
US7573871B2 (en) Apparatus for processing OTN frames utilizing an efficient forward error correction
US20200091932A1 (en) Bit block stream bit error detection method and device
CN113875207A (en) Method for providing path signaling overhead in 64B/66B character streams of ITU-T metropolitan area transport networks
US11329910B2 (en) Transmission apparatus including a plurality of switches and output circuits for transmitting data and transmission method
US20100262887A1 (en) High Integrity Data Network System and Method
US20070147434A1 (en) Data transmission method and data transmission device
JP2012161103A (en) Method and device for realizing data error reporting
US6920604B2 (en) Systems and methods for high speed serial encoding and decoding for data and control interfaces
JP6235987B2 (en) Optical transmission system and optical transmission method
Shacham Packet recovery and error correction in high-speed wide-area networks
US6959019B2 (en) Aharmonic interleaving of forward error corrected (FEC) signals
US6831927B1 (en) Fault protection for hitless and errorless switching of telecommunications signals
EP1202599A2 (en) System and method for routing IP packets using an optical core
JP6348870B2 (en) Communication apparatus and communication method
CN112311497A (en) Communication method and communication equipment
CN1770765A (en) GFP data performance monitoring method and GFP device
CN117354647A (en) Data frame verification method and related equipment

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTEL CORPORATION, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BANSAL, AKASH;BANNUR, JAISIMHA;VARMA, ANUJAN;REEL/FRAME:014306/0668;SIGNING DATES FROM 20030530 TO 20030630

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION