EP0965197A1

EP0965197A1 - Event recording and analysis in telecommunications networks

Info

Publication number: EP0965197A1
Application number: EP97918847A
Authority: EP
Inventors: Andrew Louis Martin
Original assignee: Martin Communications Pty Ltd
Current assignee: Tele IP Ltd
Priority date: 1996-09-27
Filing date: 1997-09-26
Publication date: 1999-12-22
Also published as: NO991498L; NO991498D0; JP2001501395A; CA2267254A1; AUPO264996A0; WO1998013966A1; BR9711547A

Abstract

A method of recording events at stations (12) located at geographically separate locations across a large telecommunications network (10) is disclosed, together with event recorders (16) and a telecommunications system whereby the time-correlation of recorded events is greatly facilitated. Radio signals from an earth satellite system are used to generate precision time signals at each location and these time signals are used to time-stamp event locally recorded event data at each station (12). The recorded data and time-stamped data from each station (12) may then be downloaded to a control centre (20) and automatically arranged in time sequence with a precision of about one microsecond across the whole network.

Description

EVENT RECORDING AND ANALYSIS IN TELECOMMUNICATIONS NETWORKS

TECHNICAL FIELD This invention relates to event recording, monitoring and analysis in telecommunication networks, to event correlation in large and complex telecommunication systems, to event recorders for use at local stations in such networks, and to methods of event correlation therein.

The invention has particular application to the recording and time-sequence analysis of faults or alarms in large telecommunications networks, though it also has value in the recording and analysis of other events, such as call-traffic data, to enable the performance of a network to be assessed.

In this specification, a large network is one in which the geographical spread of nodes or exchanges is such that the time taken for a signal to pass across the network is significantly greater than the time taken to switch signals within an exchange.

BACKGROUND TO THE INVENTION

The analysis of recorded events to determine the cause of faults and to assess system or component performance is of critical importance for the maintenance and efficiency of large complex telecommunications networks. It is, however, a most challenging task, given the complexity of such networks, the large number of events being recorded and the great rapidity in which trains of causally linked events can propagate and branch in such networks.

Modern telecommunications networks are said to be the most complex systems devised by humans. Not only are there many millions of components — switches, links, multiplexers, modems, computer and software elements — interconnected in a vast and complex manner, but the distribution of call traffic is usually under adaptive software (computer) control so that the system adapts 'intelligently' to local congestion or faults. Thus, the route taken across the network by a given call or data packet between one subscriber and another is difficult if not impossible to determine as it depends upon local instantaneous traffic and fault conditions.

In such complex systems minor component, transmission, switching or routing faults may be obscured for some time by the fault-tolerant or adaptive character of the network until a substantial system failure is triggered by a 'last-straw' fault or by unusual traffic conditions. There is also the danger of 'fault avalanches' where one fault, or a particular combination of faults, trigger a train of subsequent faults, as in a chain-reaction. Such fault avalanches can occur with great rapidity, perhaps involving thousands of faults and alarms occurring over wide areas of the network in a few seconds. Another complexity which bedevils fault and alarm monitoring in such systems is that some faults, such as the loss of data frames on a bearer, are not remotely alarmed until multiple attempts to automatically correct the problem at the local exchange have failed and the data loss becomes significant. Thus, the alarm for such a fault may occur many milliseconds after the initial fault event was recorded locally. Other fault conditions may be alarmed immediately they occur. Furthermore, few if any large systems are homogeneous in regard to the age and type of equipment employed so that the speed and character of event recording will vary from place to place across the network. All this means that it is extremely difficult piece together a reliable sequence of events in many fault situations involving more than one exchange or station.

Nevertheless, effective event monitoring and analysis is vital in modern telecommunications systems: it is key to gauging and improving system efficiency: it is essential for the resolution of fault avalanches and for discerning the peculiar combination of events which contribute to a last-straw or other unexpected system failures. Generally, one or more event recorders or monitors are included in each network node or exchange and each is connected to monitor many events — perhaps hundreds — occurring in that node or exchange, the time and nature of each event being automatically recorded by the monitor. Each monitor may be arranged to automatically report events which it has recorded to a central system or area control centre, or it may be periodically interrogated by the control centre to 'read' or download the recorded events, in this way, well over 100,000 events may be monitored at points spread over a geographical area the size of a continent.

In analysing the cause of apparently related faults involving a plurality of nodes or exchanges, it is obviously desirable to determine the relative order and timing of the recorded events; that is, to correlate the events. While it is not difficult to correlate local events in, say, one exchange with microsecond accuracy, it is extremely difficult to correlate events in adjacent exchanges to the order of hundreds of milliseconds — let alone events occurring in widely separated parts of the network — because of inherent difficulties in synchronising of the clocks in different exchanges. In this situation, recourse must be made to computationally-intensive, rule-based 'expert systems', or to computer-modelling. US patent No. 5,666,481 to Lundy illustrates the former approach and provides many references to current literature on the problem. US patent No. 5,646,864 illustrates the latter approach. The cost — and, indeed, the practicality — of applying any such computational technique depends critically upon the accuracy with which one event within the network can be timed with respect to another. Telecom Australia has reportedly spent more than two years and $20 million in a major research project aimed at refining the current methods of event recording to obtain better than 10 millisecond precision across the Australian network.

The problem of network-wide synchronisation will be greatly reduced in next- generation digital networks employing the common synchronous digital hierarchy (SDH) architecture. Indeed, it is reported that Siemens has devised a method (called, EMOS) of synchronising clocks throughout an SDH system with microsecond precision. Unfortunately, this synchronisation method cannot be applied to current- generation networks which do not implement the SDH timing architecture. Not only is the EMOS synchronisation system expensive to implement on a system-wide basis but, like the conventional method indicated above, it is achieved via the network itself and is therefore vulnerable to network faults.

OBJECTIVES OF THE INVENTION

It is therefore an object of the present invention to provide means for improving event recording, monitoring and analysis in telecommunications networks (and especially those having a large geographical spread) by enabling a more precise and less costly way of ensuring the synchronisation of local clocks used for recording events. This will allow improved fault/event recording, monitoring and analysis for telecommunications networks, improved telecommunications systems and networks incorporating such fault/event recording or monitoring systems, and improved event recorders for use in telecommunications networks.

OUTLINE OF INVENTION

The present invention is based upon the realisation that radio signals from earth satellite systems, such as the GPS satellite system, can be used to ensure that clocks used to log events in a telecommunications network can be synchronised with great precision in a manner which is independent of the integrity of the network or its configuration. By the use of a GPS-referenced clock having a UTC-format time- signal output which is accurate to the second, together with the use of means for generating synchronised sub-second intervals, precise universal event timing can be achieved at as many locations across a telecommunications network as desired. Indeed, microsecond precision in event correlation across a large network can be achieved in this manner. Moreover, such clocks can be built easily and cheaply from commercial components and, even if one is used with each of the thousands of event recorders in a large telecommunications network, the substantial cost savings which will be afforded over conventional methods of synchronisation will compounded by large cost savings in event correlation and analysis. The need for separate lines or channels devoted to the transmission of synchronising pulses throughout the network is eliminated.

Thus, from one aspect, the invention consists of a method of recording events occurring in each of a plurality of stations across a telecommunications network wherein events occurring at each station are time-stamped to a common time reference by using time signals derived from the radio signals emitted by an earth satellite system, such as the GPS system. By using a local oscillator to generate multiple pulses per second, a pulse-counter to count the oscillator pulses, and the synchronising pulses derived from the satellite signals to regularly reset the counter, a second of time can be sub-divided as required while remaining in precise synchronism with the remainder of the time signals. A UTC-format time-signal code accurate to the microsecond can be readily achieved in this way. That is, events can be time-stamped to a highly precise time reference which is common across the network. This level of event-correlation precision has hitherto been wholly impossible on existing networks.

From another aspect, the invention comprises an event recorder for use at multiple sites in a large telecommunications system, which recorder comprises: an input for receiving event data registered by a plurality of event monitors; clock means for generating a precision time-signal code, including sub-second time intervals, from the radio signals received from an earth satellite system; and, memory means for recording the event data as it is received together with the time-signal code corresponding to the time of receipt of the event data.

From another aspect, the present invention comprises a telecommunications network having a plurality of such event recorders together with at least one central station capable of interrogating each recorder, downloading the time-stamped event data recorded thereby and automatically correlating event data from different recorders according to time.

DESCRIPTION OF EXAMPLES

Having broadly portrayed the nature of the present invention, three example of the implementation of the invention to monitors used a large telecommunications network will now be indicated by way of illustration only. Two hardware-based examples and one firmware-based example will be described, the hardware-based examples being described with reference to the accompanying drawings in which:

Figure 1 is a diagram of a small part of the telecommunications network indicating telecommunications links between exchanges incorporating event- recorders and the Internet,

Figure 2 is a block circuit diagram of a stand-alone event recorder comprising the first example of the implementation of this invention, and Figures 3A and 3B represent two plug-in boards (a data-capture board and a GPS clock board respectively) suitable for use with the bus system of a computer, comprising the second example of the implementation of this invention.

5 Referring to Figure 1 , the telecommunications network 10 of this example comprises a plurality of switches or exchanges 12 interconnected by telecommunications links 14. Each exchange includes an event recorder 16 adapted to record events (including faults and alarms) occurring within the exchange and detected by a plurality of event monitors or detectors 17 together with the time according to a local

10 clock 18. In this example, each recorder functions as a datalogger which has an Internet address and is therefore capable of being interrogated via the Internet (generally indicated at 19) which may include portion of network 10. Also connected to the Internet is a central control/analysis station 20 with its own local clock 18a so that, either automatically or under human control, station 20 can be used to

15 interrogate any or all of recorders 16 to download their event data — each with its local time-stamp. Where event data is downloaded from recorders in different exchanges, the central station is preferably programmed to automatically correlate and present the events to the operator in time sequence according to their time- stamps. If further analysis is necessary, this raw event data may then be used as

20 the input for a trouble-ticket system like that of US patent 5,666,481 and/or as input in a windowed correlation system of the type disclosed in US patent 5,646,864.

Typical transmission and transfer delays (signified by numerals followed by μs, indicating microseconds) have been inserted in some links 14 and in the exchanges

25 12. Local clocks 18 and control-centre clock 18a are diagrammatically shown indicating different times because, hitherto, clocks across a large network would normally be out of synchrony with one another by as much as a few seconds. It will be appreciated that this factor, coupled with (i) the substantial transmission and transfer times, (ii) the variable parallel paths in a typical network and (iii) the

30 variability of exchange type and equipment all combined to make event correlation and analysis in large telecommunication networks an extremely difficult problem where accurate local clocks were not available or used. Figure 2 illustrates a first example of the way in which the invention may be implemented in the hardware of each recorder 16. In this example, the local clock 18 of Figure 1 is shown as a GPS receiver/clock module 22 which is based upon a board-based GPS receiver sold by Rockwell under the trade mark JUPITER using its ZODIAC chipset, GPS clock 22 being connected to a GPS antenna 24 (also available from Rockwell) via aerial feed-line 25. GPS clock 22 has two outputs: (i) a data-bus 26 on which a UTC time signal is generated indicating the year, month, day, hour and second; and, (ii) a synch-line 28 which carries a precisely timed synchronising pulse of one microsecond duration each second to signify when the indicated UTC time is valid. This synchronising pulse may hereafter be called the 'synch-pulse'.

The synch-pulse is fed via line 28a to each of a series of BCD decade counters 30a, 30b and 30c (eg 74HC160 chips) to precisely and simultaneously reset all counters once a second, and the synch-pulse is fed via line 28b to an interrupt port 32 on a microprocessor 34 to provide a precise time reference for microprocessor operations. The first counter decade 30a in the series receives a 1 MHz clock input on line 36 from a divider circuit 38 which is fed in-turn from the clock oscillator 40 of microprocessor 34. Conveniently, the microprocessor (eg an AT89C52) has a clock rate of 16 MHz and circuit 38 (eg, a 74HC161 chip) is arranged to divide by a factor of 16. The carry-output of counter decade 30a is fed as the clock input to counter 30b and so-on down the counter series. For the sake of simplicity, only three counter decades are illustrated but it will be appreciated that a series of six decade counters would be needed to collectively provide an output count of microseconds in precise synchrony with the GPS clock 22. With the three decades illustrated, it can be assumed (for the sake of illustration) that divider 38 is arranged to divide by a factor of 16,000 rather than 16, giving a millisecond rather than a microsecond count, decade 30a counting milliseconds, decade 30b counting tens of milliseconds and decade 30c counting hundreds of milliseconds (as indicated on their respective output busses).

The outputs of counters 30a and 30b are fed to an 8-bit latch or buffer 42a (eg a 74HC374 chip) while the output of counter 30c is fed to a similar latch 42b. Thus latches 42a and 42b are together presented with a continuous and synchronised millisecond count. This count is precisely synchronised to the UTC-format coded time-signal generated by clock module 22. [Microprocessor 34 may be reset if desired by using a reset circuit 46.] 5 In this example, event data is fed to recorder 16 via each of two 16-channel data input busses 50a and 50b via two input conditioning circuits 52a and 52b (eg 74HC4538 chips) into a multiplexer 54 (eg 74HC138 chips). Upon activation by a 'select' signal on line 56 from microprocessor output port 58, data in the multiplexer

10 54 is placed on its output bus 60 and presented to a non-volatile memory 62 for recording. The presence of data on event-data buses 50a and 50b (after passage through their respective conditioning circuits 52a and 52b) is detected by an OR gate 64 which generates a signal on its output line 66 which enables latches 42a and 42b to store the millisecond count presented thereto at that instant. Output line 66 is also

15 connected, via line 66a, to the hardware-interrupt port 32 of processor 34 so that, when incoming data is signalled by OR gate 64, microprocessor generates an output-enable signal on line 68 from its output port 58 that is connected to latches 42a and 42b. Upon receipt of this signal, latches 42a and 42b place the stored millisecond reading onto output bus 70 which also presents that data to memory 62. 0 At the same time, the UTC time code fed to processor 34 via bus 26 is presented at the processor's data-output port 72 and, from thence, onto bus 74 which is joined with bus 70 so that the UTC time-code is also presented to memory 62. Finally, processor 34 generates a memory address at its address port 76 to indicate where all the data now presented to the memory device is to be stored. The address 5 information is conveyed to memory 26 via address bus 78 and effects the recording of the presented event data as a time-stamped unit, completing the event-recording cycle.

An alternative way of presenting the UTC coded time-signal from clock module 22 0 to memory 26 for storage is to latch it into latch 42b (say) and to present it on latch output bus 70 along with the sub-second count. This route is indicated by broken line 80. Finally, communication between event recorder 16 and central control unit 20 (Figure 1 ) via network 19 is effected in the conventional manner via one of the processor's serial ports 82, serial lines 84 and an interface circuit 86. Thus, upon receipt of instructions from controller 20, event data accumulated in memory 60 can be downloaded automatically. Interface 86 may be permanently connected to network 19 and provided with an Internet address so that dedicated data-communications lines are not necessary.

It will be appreciated that, instead of a stand-alone system of the type illustrated in Figure 2, recorder 16 can be implemented as plug-in cards on a computer bus so that the separate microprocessor (34) of the first example is not needed. A system of this type is illustrated (in simplified form) in Figures 3A and 3B which represent two separate plug-in cards designed to interface with a VME bus. Referring particularly to Figure 3A, card 100 (shown in broken lines) has an VME bus connector 102 at one end and a 32 channel event connector 104 at the other.

Incoming event data from connector 104 is fed on input-bus 106, via signal conditioner 108, to a latch circuit 110 which is enabled by OR circuit 1 12 that detects the presence of incoming data on any of the input channels of bus 106a, the output of OR circuit 1 12 being fed to latch 110 via line 114. The presence of incoming data is also signalled to the computer's processor via line 114a and VME bus connector

102.

The output of a 1 MHz signal generator 116 is fed to a chain of six decade counters 1 18, the output from which — representing a microsecond count — is fed via bus line 120 to VME bus connector 102. Once a second a synchronising pulse on line 122 from connector 102 resets all counter decades 1 18.

When the computer processor (not shown) receives the signal on line 120 and via bus connector 102 indicating the presence of data, it will address card 100 via connector 102 and address-bus 124 which is connected to a VME interface circuit 126. When interface 126 recognises the appropriate code on bus 124, it will enable — via control line 128 — the output of data held in latch 110 onto data-bus 106b. This data is read by the computer processor together with the microsecond count on data-bus 120.

Figure 3B shows a GPS clock card 150 suitable for use in conjunction with the data- capture card of Figure 3A. Card 150 is provided with a VME bus connector 152, an VME interface circuit 154, a data latch circuit 156 and a GPS clock module 158. A connector 160 is provided for the GPS aerial (not shown) and is connected to module 158 via aerial line 162. The UTC time output from GPS module 158 is placed on data bus 164 and presented to latch 156. Upon an input-enable command from the computer processor via an address bus 166, interface 154 and latch-enable line 168, the time data is accepted by latch 156. Then, shortly thereafter, the latched time data is placed on the VME bus via bus 164a upon the receipt of an output- enable command from the processor via the VME interface 154 and line 170. The synch-pulse from GPS module 158 (that effects resetting of counters 118 via line 122 in Figure 3A) appears on output line 172 which is connected direct to bus- connector 152.

Since the VME-based of the second example functions in a very similar manner to the stand-alone circuit of the first example, it will not be described further.

As already indicated, the capture of the event and time data, as well as the generation of the microsecond count can be implemented in firmware on a microprocessor or by means of a stored software program in a computer. This would essentially eliminate the need for card 100 of Figure 3A, except that an input card for the event data is still required. An example of a suitable firmware program is given below, the program comprising a series modules having discrete functions.

Firmware Module Description

1 Network: interfaces the network to the local microcontroller. 2 Datajink: performs message transport between the local microprocessor and the network. 3 Network_message: formats network command and data messages. 4 Network_buffer: provides buffer space for storing (receive and transmit) unformatted debug and control messages.

5 Datajnterpret: processes commands and data received from the network, generating and required response.

5 6 Event_reader: reads event inputs.

7 Event_processor: processes input events.

8 Event_buffer: provides buffer space for storing processed event inputs.

9 Hardwarejnterrupt: processes hardware interrupts.

10 RTC: provides a reference derived real-time clock.

10 11 Timer_driver: provides timing information for the UTC module.

12 UTC_message: assembles and frames the UTC date and time message.

13 UTCjnterface: provides the physical interface to the UTC source.

14 Time_date: provides the real time and date referenced to UTC.

15 Timer: provides event timing for Event_processor. 15 16 Memoryjnterface: interfaces non-volatile memory.

17 Memory_driver: effects transfer of data to memory.

18 Schedular: schedules and assigns tasks for other modules.

19 Event_control: a state machine that controls Event_reader.

20 To derive UTC referenced real-time clock information, UTC_message receives the UTC time code via UTCjnterface once every second. Using the GPS synch-pulse and an oscillator input, RTC effects the subdivision of each second as required.

Event_read samples the inputs at discrete times determined by Schedular using 5 timing information from RTC. Event_processor processes data from Event_reader to determine changes in input state using, for example, the following algorithm: Step 1 Change of State Information = (Old Input Sample) XOR (Current

Input Sample) Step 2 Old Input Sample is replaced by Current Input Sample. 0

If a change of state is determined for any of the inputs sampled, the UTC time code is read from Time_date and stored with the Current Input Sample in memory by Memory_driver and Memoryjnterface. Events held by Event_buffer, Schedular directs Memoryj river and Memoryjnterface to transfer the event data to nonvolatile memory. Messages received by Network via Datajink are formatted and buffered by Networkjnessage and Network_buffer, respectively. Any requests for input information will be formatted by Network ormat and handed to Networkj uffer and from thence to Datajnterpreter to place any information in non-volatile memory on the network.

It will be appreciated that, using the recorders of the examples, events recorded anywhere within the large network are time-stamped with high precision with reference to a single universal time reference. It is then a simple matter for the central control/analysis station to interrogate selected recorders, or to automatically receive event reports from each recorder, via the Internet connections. And it is also a simple matter to automatically filter the event/alarm signals at the control/analysis station according to event type and time of occurrence. However, in conventional terms, the preparation of such a listing represents the output of a time consuming and computer-intensive analysis by expert systems engineers.

Claims

1 A method of recording, in a time-correlated manner, events which occur in a plurality of stations at geographically separate locations within a telecommunications network, the method comprising the steps, at each station, of: employing radio signals from an earth satellite system receivable at all of the locations to generate a precision time-signal code which includes the designation of sub-second time intervals; recording selected events as they occur in the station together with the precision time-signal code corresponding to the time when each of said events is received or recorded.

2 A method according to claim 1 wherein the satellite system is the GPS system and wherein a GPS clock is employed at each station to generate (i) a primary time- signal code having one-second precision and (ii) a synchronising pulse indicating when said primary time-signal code is valid, the method including the steps, at each station, of: generating a pulse-train having a plurality of pulses per second; employing said pulse-train to drive a pulse counter to generate a pulse count; employing said synchronising pulse to regularly reset said counter to generate a sub- second count that is synchronised with said primary time-signal code; combining said primary time-signal code and said pulse count to form said precision time-signal code.

3 A method according to claim 1 or claim 2 including the steps of: remotely interrogating a plurality of said stations from a network center to automatically download the recorded events, together with their corresponding precision time- signal codes; and automatically correlating said downloaded recorded events in sequence according to their corresponding precision time-signal codes.

4 A method according to claim 3 including the steps of assigning each station an Internet address and interrogating selected stations by the use of their respective Internet addresses. 5 A method of correlating events which occur in a plurality of geographically separate locations within a telecommunications network, the method comprising the steps of: at each location: employing radio signals from the GPS earth satellite system to generate a precision UTC-format time-signal code which designates time intervals of less than 10 milliseconds duration, recording each event as it occurs, together with the precision time-signal code corresponding to the time at which the event is recorded, to generate an event record, at a central location: remotely interrogating a plurality of said stations to effect the downloading of said event records from each of the interrogated stations, automatically arranging said downloaded event records in sequence according to the time of occurrence of the events by making reference to the precision time-signal code corresponding to each event.

5 A method according to any preceding claim wherein the precision time-signal code designates time intervals of the order of microseconds.

6 Event recording means for use in large telecommunications networks, comprising: event input means adapted to receive event data derived from a plurality of event recorders, radio clock means adapted to receive radio signals from an earth satellite system and to generate therefrom a precision time-signal code, including the designation of sub-second time intervals, at an output of said radio clock means, and memory means, connected to said event input means and to said output of the radio clock means, adapted to record event data as it is received together with the precision time-signal code corresponding to the time at which said event data was received. 7 Event recording means according to claim 6 wherein said radio clock means includes: primary clock circuit means for generating a primary time-signal code with a one second precision together with a synchronising pulse signifying when said primary time-signal code is valid, secondary clock circuit means for generating sub-second time intervals, said secondary circuit means comprising: a pulse generator adapted to generate a continuous train of regularly occurring pulses such that the interval between two successive pulses is substantially less than one second, a pulse counter connected to said pulse generator to so as to receive said pulse train, said pulse counter being adapted to count successive pulses as they are received, and said pulse counter being connected to receive said synchronising pulse and to have its count reset each time the synchronising pulse is received thereby, whereby the count registered in the pulse counter at any instant represents the sub-second time interval of said precision time-signal code, and whereby the combination of said primary time-signal code and a representation of the count registered in the pulse counter at any instant comprises said precision time-signal code.

8 Event recording means according to claim 6 or claim 7, wherein: buffer circuit means is connected to both said event input means and to said output of said radio clock means, said buffer circuit means being adapted to temporarily store said event data together with the precision time-signal code corresponding to the time at which the event data is temporarily stored, the combined event data and corresponding time-signal code hereinafter being referred to as time-stamped event data, and said memory means is adapted to receive and store said time-stamped event data as it is cleared from said buffer circuit means.

9 Event recording means according to claim 6, 7 or 8 wherein the precision time- signal code designates time intervals of the order of microseconds. 10 A telecommunications network including a plurality of stations at geographically spaced locations: a radio clock at each station adapted to receive radio signals from an earth satellite system covering all the locations, clock means at each station connected to the radio clock at that station and adapted to generate a precision time-signal code having sub-second time intervals, and event recording means at each station adapted to receive event data from local event monitors and connected to receive said precision time-signal code, said event recording means being adapted to record received event data together with the precision time-signal code corresponding to the time at which the event data is recorded.

11 A telecommunications network according to claim 10 including: a control center adapted to: remotely interrogate a plurality of said stations and to download recorded event and time data from each of the interrogated stations, automatically correlate the downloaded event records in sequence according to the time of occurrence of the events by making reference to the precision time- signal code corresponding to each event.

12 A telecommunications network including a plurality of stations at geographically spaced locations characterised in that each of said stations includes event recording means as claimed in any one of claims 6 to 9.