IE84921B1 - Mobile network user activity monitoring - Google Patents

Abstract

ABSTRACT A monitoring system (10) comprises a data collector (11, 14) for passively reading content being communicated in a network (SGSN-GGSN) and a processor (15) which filters the content to identify request messages and associated subscribers. It parses URLs of downloaded content into individual components to recognise the content, opens a record in a session cache for the request message and the content, and writes a flag to the record to indicate that the content is routed to the requesting subscriber. Also the system parses the content to identify key words indicating the content, and generates an output indicating quality of the content response to the request message. The processor increments a counter each time a record does not have a required key word, monitors size of the content, and determines that quality is poor if size is below a threshold. Also, it increments a counter each time a content size is below the threshold.

Description

Mobile Network User Activitv Monitoring INTRODUCTION Field of the Invention The invention relates to mobile networks, and particularly to monitoring of user activity.

Prior Art Discussion Mobile networks have been deployed that enable a mobile accessible, always available, wireless internet. To exploit the full potential from those new mobile access network technologies, and to meet high user expectations, these networks need to deliver the requested information to subscribers. Presently, various platforms exist which examine the traffic in the network and assess the network from a network- centric perspective. These can be standalone remote platforms or centrally located platfonns as described in US6,9l5,l 10 General Radio Packet System (GPRS) provides a network infrastructure to facilitate a range of data services that are provided by network operators worldwide. 3G is the common name for UMTS (Universal Mobile Telephony Service), which uses GPRS to provide data services.

Some of the main components referred to in the description below are: Mobile device. This can be a phone, PDA or data-card that provides network connectivity via GPRS to a PC or laptop.

Home Location Register and Visitor Location Register. These are the databases that hold information for every individual who has a subscription with the GPRS operator. Information present in the HLR includes supplementary services, authentication credentials, APN (Access Point Name), and the subscribers’ ISP (Internet Service Provider). For GPRS, subscriber information is exchanged between the HLR and SGSN. The VLR contains temporary subscriber information needed to provide services for visiting subscribers who are roaming overseas.

SGSN (Sewing GPRS Support Node). The SGSN forwards data to and from a mobile device within the SGSN service area and it also provides data routing and transfer to and from the SGSN service area. It serves all GPRS subscribers that are physically located within the geographical SGSN service area. In addition, an SGSN provides encryption and authentication, session management, and mobility management.

GGSN (Gateway GPRS Support Node). The GGSN provides connectivity between the GPRS core network and the customer's corporate network or to the Internet. It also provides GPRS session management, functionality for associating subscribers to the correct SGSN, and billing information.

The invention is directed towards achieving improved mobile network user activity monitoring.

SUMMARY OF THE INVENTION According to the invention, there is provided a mobile network use activity monitoring system comprising: a data collector comprising means for passively reading data including messages and content being communicated in a network; a processor comprising means for: filtering the data to identify request messages and associated subscribers; parsing URLs of content downloaded in response to request messages into individual components to recognise the content; opening a record in a session cache for a request message and corresponding content, and writing a flag to the record to indicate that the content is routed to the requesting subscriber; parsing the content in the record to identify key words indicating the content; and generating an output indicating quality of the content in view of what was requested in the request message.

In one embodiment, the processor comprises means for incrementing a counter each time content does not have a required key word.

In one embodiment, the processor comprises means for monitoring size of the content and determining that quality is poor if size is below a threshold.

In one embodiment, the threshold is configurable.

In one embodiment, the processor comprises means for incrementing a counter each time content size is below the threshold.

In another embodiment, the processor comprises a stack builder for dynamically loading protocol handlers and configuration information to adapt to changing protocol stacks on probed interfaces.

In one embodiment, each protocol handler is modular.

In one embodiment, each protocol handler is programmed to parse records, retrieve relevant information, and pass the information to a processing component.

In one embodiment, the processor comprises means for processing together information extracted from different protocols.

In one embodiment, the messages and content are cached in the fomi of objects.

In one embodiment, the objects are time-stamped and removed according to age.

In one embodiment, the processor comprises means for polling user addresses to determine if a server is active and for removing information concerning inactive sessions from the cache.

In one embodiment, the address is an IP address.

In one embodiment, the processor comprises a plurality of hardware nodes.

In a further embodiment, the processor comprises means for ensuring that all records for a particular session are handled by the same node according to a distribution scheme which distributes according to key fields.

In one embodiment, a router node comprises means for routing records according to the keys to minimise a subscriber base of records.

In one embodiment, the processor comprises means for monitoring a number of sessions up to a maximum threshold.

In one embodiment, the processor comprises means for monitoring only the most active users.

In one embodiment, the processor comprises means for maintaining a database of users detected in a preceding period to determine in real time the most active users.

DETAILED DESCRIPTION OF THE INVENTION Brief Description of the Drawings The invention will be more clearly understood from the following description of some embodiments thereof, given by way of example only with reference to the accompanying drawings in which:- Fig. 1 is a block diagram showing a data collector connected to a GPRS network; Fig. 2 is a block diagram showing a full monitoring system, having an alternative data collector, with a passive spanning port; Fig. 3 is a block diagram showing a full monitoring system, having a data collector with a fibre optic tap; Fig. 4 is a block diagram showing the processing server in more detail; Fig. 5, is a sample Gn interface stack handled by the system; Fig. 6 is a sample session record structure; Fig. 7 is a sample record used for GSN link discovery; and Fig. 8 is a representation of common fields used for data distribution.

Description of the Embodiments A communication network monitoring system monitors the actual data content transported between SGSN and GGSN network elements over a Gn interface in a 2.5G or 3G mobile network. Data collectors are situated at locations where measurements are desired in a monitored mobile network, such as at the Gn interfaces, and are connected in a non-intrusive manner to the mobile network. The system also comprises a processing server. The data collector provides an interface to the GPRS network, and the processing server analyses the information collected by the data collector. Physically, in this embodiment the data collector and the processing server are hosted on the one computer. The computer has a network interface card for connecting the processing server to the GPRS network.

This system analyses the data collected from a content-centric perspective. This allows the operator to provide revenue assurance by checking that the content requested by the subscriber is the data delivered. The data delivered by the network to the subscriber should not be a predefined message for the network but the actual content requested.

The system generates information which allows the network operator to identify where problems arise. For example, it is important for the operator to know if the subscriber experienced inadequate quality-of-service because: (a) there was insufficient bandwidth during the session, (b) there was congestion in the particular network cell at the time: (c) the mobile device was inadequate for the protocol being used; or (d) the server which was providing the content was over-congested, or shut down for maintenance or due to a fault.

Referring to Fig. 1 an SGSN and a GGSN are shown, and a data collector has non- invasive taps 1 on the link between them, and a data collector component 2.

Referring to Fig. 2, a data collector 10 comprises an Ethernet switch 11, a 100BaseT full duplex spanning port 12 linked by a 101 Base T full duplex interface 13 with a data collector component 14. The latter is in tum linked with a processing server 15.

A spanning port is maintained across the Gn interface using the Ethernet Switch connection. The spanning port is isolated to the SGSN port or GGSN port in order to minimize usage of spanning resources. The data collector will decode the GTP traffic from the spanned port.

Referring to Fig. 3 there is a fibre optic tap 21 feeding a data collector component 22, in turn connected to a processing server 23. As shown in Fig. 4 the processing server comprises a database 25, a report manager 26, and a memory 27.

The system 10 obtains the data it requires to perform content-centric revenue assurance directly from the standard logical interfaces. To exploit the full potential of .5G and 3G networks and devices, the operators need to ensure that the information requested by each subscriber is the information delivered to the subscriber. An important basis for these tasks is accurate assessment of the content traffic in the network.

The data collector passively analyses Gn traffic from Ethernet, Fast Ethernet or Gigabit fibre optic interfaces, and it operates by listening to all traffic on a Gn interface. Two connectivity methods are possible, Ethernet and Gigabit Ethernet fibre optic interfaces. Either deployment is completely non-intrusive. Both interfaces could be required. Gigabit fibre optic interfaces are generally configured between the SGSN and GGSN links (Fig. 1) whilst Ethernet links are generally deployed on intra-GGSN links (Fig. 3).

The network taps l provide passive a full-duplex Fast Ethernet monitoring. Taps are non-blocking devices, and pass through full-duplex data at line rate. A passive tap device can be installed at any suitable port over the Gn interface (SGSN-GGSN).

Once installed, the data collector is connected to the output ports from the tap. Taps are passive devices that can be left permanently in—line without causing any data stream interference. The network tap records and maintains the integrity of the full duplex connection. The internal function of the network tap amplifies the signal and creates two output ports (uplink and downlink). The data collector, using two network interface cards will recombine the output data and produce tag data records output from Gn, which is data that will be post analysed by the server.

Each tap 21 is an optic fibre Gigabit tap which allows the network to operate at a continuous flow while the tap is non—operational, thereby maintaining network traffic flow. If power is lost, traffic is still passed through the tap. These taps also have passive—link integrity that is maintained whether the network monitoring devices are on or off. The splitters are used to split light with minimal loss from one to two fibres, or to combine light from two fibres into one fibre. The splitter taps provide the ability to monitor traffic flowing in either direction of a link connection. The taps have a split ratio of 50/50.

The data collector 22 and the server 23 capture the packets and store them in the data record database 25. The packets collected contain information on the subscriber requests for content and the actual content received. Pre-filtering may be performed to allow the data collected to be filtered by the IMSI of the subscriber. The pre- filtering is performed in real time during the capture stage. All packets captured at the Gn interface belonging to a subscriber with a filtered IMSI are captured and stored in the database 25.

Prior to the collected data being stored as a data record on the processing server 23 it is initially stored in the memory 27. At this stage the header file of the messaging being sent between the SGSN and GGSN is parsed and the IMSI is extracted. If the IMSI of the messaging being sent is in the range of the filter used this record is created. The data record created contains the content that was requested. The content requested is identified by the partial URL of the data content requested. The requested partial URL of the data service requested is parsed into its individual components.

Each component is stored separately in the data record database 25 as part of the requested information.

When the content is transferred to the subscriber this record is given a flag, in this case an index number to tie it to the original request. The content contained in the reply is stored as part of the record and analysed.

Following the storage of a record containing the requested information and the content delivered the record is examined. The details of the information requested form part of the record, as does the information delivered. The data content captured by the data collector could be of any data type. The content package is opened and analysed to check if it contains the infonnation requested, by checking the captured content for key words. The key words are the words used in the initial request or are the final strings of the partial URL that made up the request location.

In one embodiment, once the content is analysed it is deleted from the data record database 25.

When a record is created the size of the content part of the record is analysed. If the size of the content in bits is less that a configurable variable the content is examined to check if it contains predefined strings indicating the lack of content availability.

If the returned content does not contain the key words of the requesting content a record within the report manager is created outlining the requested information and incrementing a counter. For each subsequent occasion that the same situation arises the same counter is incremented in the report manager.

If the retumed content is less than a predefined size and contains one of the predefined messages indicating the lack of content availability a separate counter is incremented to document this.

The reporting manager provides a user interface for displaying both the collected records and enables the displaying of information pertaining to the incremented counters during the process.

Stack builder In order to provide a complete overview of the user activity detected on the Gn interface, the data collector understands and parses a complete protocol stack. Fig. 5 depicts the main protocols that make up the Gn stack. In order to easily adapt to changes in protocol stacks (for example, in case of Gn, GT P over TCP), the system implements a dynamic stack builder that is capable of loading appropriate protocol handlers and read configuration information on start-up.

Protocol handler framework The system maintains a list of supported protocols and provides a plug-in based framework of protocol handlersEach protocol handler is a software module that can be loaded and built into the executable depending on the configuration of the stack builder. Protocol handlers implement well defined API for retrieving appropriate records, parsing each record, retrieving relevant information and passing the information for subsequent output or analysis. Implementation of a protocol handler is very specific and differs from protocol to protocol.

Combining details from multiple protocols — session/context cache As shown Fig. 5, in order to create an output record, the system processes multiple protocols. Each protocol contains valuable information that will be used by business lcvel logic and provide KPIs (key performance indicators) and raise alarms. For example, a GTP message with details related to PDP context creation, update, and delete has the following useful fields: Timestamp Rsp (PDPtime) o Client IP (SGSN) Client Port (SGSN) Server IP (GGSN) Server Port (GGSN) MAC Address Client (GSN) MAC Address Server (GSN) o IMSI This information needs to be cached and provided as an output in, for example, WAP connection (connect, redirect, disconnect). Such WAP fields can be: Client IP Address Client Port Server IP Address Server Port Method (con, redir, disc) Uri Request The system maintains session/context records open for a particular user’s transactions These session/context records are provided as output. In the above case, PDP context and WAP data are combined and cached in context records for further output. The more information captured from different protocol, the more valuable is the output from the system.

Fig. 6 shows the structure of session records. Important aspects are separation of protocol-specific fields into groups and consistence of field meanings across interfaces.

Optimization technigue: methods for removing time-out sessions As described above, the system caches session/ context records. The number of cached objects can be very high, and so the system employs two techniques for removing old records: l. Programmatic removal of old records The system stores a timestamp of each object in memory. A separate thread is scheduled to run every M milliseconds and removes sessions older then T.

Both M and T are configurable parameters of the system and can be adjusted to suit a particular data load and hardware specification.

. Detecting dead sessions The system can also find “dead” or expired sessions. Any user of the system with an open session is assigned an IP address. The system can ping this IP address using standard TCP/IP methods. Should ping operation fail, the IP address/session in question will be treated as timed-out and immediately removed from the session cache. Should the above technique be employed, the monitoring system will cease to be passive, since active actions will be undertaken.

Real-time filtering of processed records In order to improve the performance of the system, it can be set up so that only records of interest are handled, thus significantly reducing average processing time per record. The following filters can be used for minimizing the load: IP address MAC address 0 TCP port UDP port The system matches the above fields against a list of pre-configured filtering parameters.

Automatic discovery of filter entities Manual configuration of filter entities (new SGSNs/GGSNs) can not always be possible. New SGSNS/GGSNS are auto-discovered from the input packets. The list of GGSN/SGSN pairs is written to an output file at regular intervals. The system can turn on and off this feature through runtime configuration. Fig. 7 is a record on GTP traffic and can be used to identify GSN node MAC addresses: Distribution technique — using TEID in GTP-C for data redirection The system is capable of processing records on multiple hardware nodes, in a node cluster. Each processing node runs essentially the same software for analyzing the traffic. The main problem faced by the system is to ensure that all messages (records) that belong to the same session are processed by the same machine. If this requirement is not satisfied, the system will need to provide a costly messaging mechanism for extra communication between nodes of the cluster and sharing session cache.

This is addressed by identifying an easy to retrieve key field that is found in all low- level records. Key field values should be unique throughout sessions. Minimal parsing should be done in order to identify key field values. This is important since otherwise, the node/module that redirects records (router node or R node) for further processing by the cluster will be overloaded.

R node receives all records detected by the system, finds key fields and redirects records for real processing to cluster nodes. Key field values are simply divided by the total number of processing nodes. The reminder of this division is used to forward the records. Since key field values are unique, this guarantees that the same processing node handles all messages of the same sessions, and therefore build correct session/ context records.

In the GTP traffic, the fields of Fig. 8 are common and unique and therefore suitable for data distribution in a Gn interface. As shown in Fig. 8, TEID keys are a clear choice for key field on Gn traffic.

Selection of top and bottom users in large networks Some of the high level reports built using the Gn interface system outputs can be the following: 0 List all the current http downloads and rtp streams etc. in a summary screen 0 Display currently active calls of a particular caller in a summary screen and can drill down layers Peak usage time per user and per application Drill down into a particular time period in the day and then right down to the relevant user and protocol Display when users went over or are going over a certain threshold of usage.

Based on rate and volume downloaded Identify active users Define monitored user groups The system is capable of providing this information even running processing nodes with low hardware specifications. This is possible only by employing filtering and distribution techniques described above as well as minimizing the user base. For example on a network of 15 million subscribers, reports above will be possible to provide by analyzing 100.000 subscribers. In this case, the system should monitor .000 most active users in order to provide accurate reports. To detect most active users in the networkthe system maintains an in-memory database (IMDB) of users detected in the last M minutes (M is configurable). The table contains user identifiers, timestamps and usage statistics. As a record passes through the system, IMDB is updated with this information — if user information is found, the timestamp and usage statistics are updated, if user information is not found, its details are added to the database. lMDB is indexed by user ID in order to optimize search by user. The system is configured with an initial threshold I parameter that sets minimum value for a user records to be accepted as a top user record. A thread is used to run every T minutes and updates the top list using statistics retrieved from the database and update the I parameter (initial value is set using empirical methods).

The following are two sample records outputted by the processor: O0OO0002|3016|20051205124241537|2D051205124241642|[CUT]|2123|[C UT]|2123|00:0E:38:1D:7C:08|OO:OE:4O:712F4:07|[CUT]F|05|0|6C6C|O0OOOO00 |2186D1B9|2186CFBB|2186CFBB|12A51205|O04BOC53||||1|3|9|2|31|[CUT]|[C UT]|911497448116FO|1|3|9|2|31|10.111.1.51|128|||||||| O0OO0OO023|3020|20051205124248182|20051205124248185|[CUT]|2123|[C UT]|2123|0O:OE:38:1D:7C:O8|O0:OE:40:71:F4:07l[CUT]F|O5||6C81|004BOC53|| |2186CFBB|||||||||||[CUT]||911497448116FO||||||10.111.1.51|128|||||||| In the above two records, the network monitoring system provides details of a user transaction where an MMS message was sent. The records are a result of parsing of the following protocol stack: IP -> TCP -> HTTP -> MMS. User data in the session is cached until the MMS message is delivered correctly.

From the records above, a high-end business application can now obtain timestamps, user identification (IMSI), IP addresses and ports used, as well as flags or causes for analyzing the success or failure of the transaction. The following fields have been removed from the output records and marked as [CUT] to protect the privacy of the involved parties: IMSI, IP address, URLs, user agent details. IMSI, and IP, MAC addresses will later be used to filter the output of the data and analyze top and bottom users of the system for further filtering and reporting. Key Performance Indicators (KPIs) can then be built to provide performance data per user, service, IP address and other fields.

Determining the User’s Satisfaction The details above primarily relate to the capture of communication messages in computer-based telecom networks, the processing of these messages in order to extract interesting data and the mechanisms for doing this for heavily loaded networks. In addition to this, the system will also process the data extracted as described above and determine the user’s satisfaction with the service in use.

The system performs a holistic interpretation of a user’s interaction with a data session, in real-time, in order to determine the user’s satisfaction with the session. To do this the system builds session contexts, interprets the type of service being used, the integrity of the pages being accessed, whether or not the user managed to access the information that he required and whether or not the user could perform the functions that he wanted to perform.

The first step in performing the holistic interpretation is to collate user transactions into a session. The system has the ability to create sessions identified in the following manner: a. by collating discrete user transactions, with the same application protocol, taking place within a pre-defined timeframe, generated by the same user and from the same equipment; b. by collating discrete user transactions, related to the same content subject matter, taking place within a pre-defined timeframe, generated by the same user and from the same equipment; c. by collating discrete user transactions taking place between a user request for resources on the GPRS network and either a user or network initiated request to release these same resources; d. by collating discrete user transactions taking place between an initial user request for service and a determination that the user is inactive based on a period of inactivity during which no user requests are seen traversing the network.

All timeframes mentioned above are configurable and can either be defined relative to the first user transaction in the session or the last captured user transaction in the session.

Evidently from the above the system will be capable of determining the content subject matter. This is referred to as categorisation. It will also understand the subject matter (category) of web pages by either (a) linking to an external repository of URL categorisation or (b) programmatically determining the content categorisation by comparing key words and combinations of key works on the page to a locally stored repository of key word categorisations.

The system is also capable of determining if the content requested has been delivered by comparing keywords in the URL with keywords on the page delivered.

Furthermore the system can identify pages on which the user is either viewing pricing information for a product or service or is about to make a purchase or subscribe to a service. This is achieved by comparing the wording on the page to a pre-defined library of terms that are commonly used on these types of pages. The system will track a. if the user progresses to make a purchase after landing on such a page, and b. if the purchase/subscription attempt was successful.

Another key component of the user’s satisfaction will be the ability to find the desired content in a timely fashion. In order to determine if the user succeeded in doing this the system will track user searches in search engines and will determine if the user found the content required by monitoring the user behaviour once the search results are returned. An indication that the user has found the content that he wanted is that the user clicks on a link on the search results page and then spends a significant amount of time exploring the target website.

Indications that the user did not find the content required will include: a. the user follows a link on the search results page but quickly exits the target website without reading the content, b. the user returns repeatedly to the search results page and clicks on many links on the search results page, or c. the user tries numerous search terms using related words.

At the completion of a user session the system interprets the User Satisfaction Index for the session (Mean Opinion Score) by combining: a. the ability of the user to find the desired content in a timely manner, b. the success of the service in providing the content/ service requested, c. the ability of the user to purchase/subscribe online, d. the desire of the user to purchase/subscribe after viewing prices, and e. the quality of service, including response times, throughput, jitter and packet loss.

User Profiling By processing the data captured as described above, the system builds and maintain a profile of people using data services and potentially through integration with other data repositories, of people using voice services also. In addition, demographic information for individuals is collected by integrating with existing databases within the operator’s network where demographic information is housed. User profiles are maintained in a database which will have a well-defined API which will be exposed to third-party systems so that they can make decisions based on the profiles. The profiles are updated automatically by the system. Snapshots of profiles will be stored periodically in order to facilitate historical analysis of how profiles change over time.

The information maintained in a profile for each individual includes: Name IMSI o MSISDN 0 Age Address o Interests Sex I Handset/Device Locations where services are most frequently used 0 Times when services are most frequently used 0 Most frequently used services Most frequently accessed content Purchase Activity Mobility Average User Satisfaction Lndex o F fiends The invention is not limited to the embodiments described but may be varied in construction and detail.

Claims (1)

1. Claims A mobile network use activity monitoring system comprising: a data collector comprising means for passively reading data including messages and content being communicated in a network; a processor comprising means for: filtering the data to identify request messages and associated subscribers; parsing URLs of content downloaded in response to request messages into individual components to recognise the content; opening a record in a session cache for a request message and corresponding content, and writing a flag to the record to indicate that the content is routed to the requesting subscriber; parsing the content in the record to identify key words indicating the content; and generating an output indicating quality of the content in view of what was requested in the request message. A monitoring system as claimed in claim 1, wherein the processor comprises means for incrementing a counter each time content does not have a required key word. A monitoring system as claimed in claims 1 or 2, wherein the processor comprises means for monitoring size of the content and determining that quality is poor if size is below a threshold. A monitoring system as claimed in claim 3, wherein the threshold is configurable. A monitoring system as claimed in claims 3 or 4, wherein the processor comprises means for incrementing a counter each time content size is below the threshold. A monitoring system as claimed in any preceding claim, wherein the processor comprises a stack builder for dynamically loading protocol handlers and configuration information to adapt to changing protocol stacks on probed interfaces. A monitoring system as claimed in claim 6, wherein each protocol handler is modular. A monitoring system as claimed in claims 6 or 7, wherein each protocol handler is programmed to parse records, retrieve relevant information, and pass the information to a processing component. A monitoring system as claimed in any of claims 6 to 8, wherein the processor comprises means for processing together information extracted from different protocols. A monitoring system as claimed in any preceding claim, wherein the messages and content are cached in the form of objects. A system as claimed in claim 10, wherein the objects are time-stamped and removed according to age. A monitoring system as claimed in any preceding claim, wherein the processor comprises means for polling user addresses to detennine if a server is active and for removing information concerning inactive sessions from the cache. A monitoring system as claimed in claim 12, wherein the address is an IP address. A monitoring system as claimed in any preceding claim, wherein the processor comprises a plurality of hardware nodes. A monitoring system as claimed in claim 14, wherein the processor comprises means for ensuring that all records for a particular session are handled by the same node according to a distribution scheme which distributes according to key fields. A monitoring system as claimed in claims 14 or 15, wherein a router node comprises means for routing records according to the keys to minimise a subscriber base of records. A monitoring system as claimed in any preceding claim, wherein the processor comprises means for monitoring a number of sessions up to a maximum threshold. A monitoring system as claimed in any preceding claim, wherein the processor comprises means for monitoring only the most active users. A monitoring system as claimed in claim 18, wherein the processor comprises means for maintaining a database of users detected in a preceding period to determine in real time the most active users. A monitoring system substantially as described with reference to the drawings. A computer readable medium comprising software code for performing operations of a monitoring system of any preceding claim when executing on a digital computer.