KR20070088481A - Rule based data collection and management in a wireless communications network - Google Patents

Abstract

Systems and methods for defining and using a data collection profile in connection with a telecommunications network that includes multiple devices. Initially, a query is defined that concerns one or more aspects of the communications network. The query is then used as a basis for building a data collection profile. Next, a set of target devices is selected based upon criteria relating to the query. The data collection profile is then provided to the target devices which collect data in accordance with the data collection profile. Finally, the collected data is transmitted to a service platform for further processing.

Description

Rule based data collection and management in a wireless communications network}

The present invention relates generally to wireless communication networks and related systems and apparatus. More specifically, embodiments of the present invention relate to systems and methods for using distributed network elements to implement monitoring and data collection in view of selected network parameters.

As a result of the enormous increase in technology development and the number of wireless device users, the size and complexity of wireless communication networks has increased significantly. The inevitable consequence of such an increase in size and complexity has resulted in a relative increase in operational and performance issues associated with communication networks. Reliability issues such as dropped calls, lack of coverage, and poor voice quality are hampering the adoption of wireless technology by end users. These and other quality issues have prevented many end users from relying on wireless voice and data services as their primary means of communication. As new services are introduced that use more complex technologies, perform different forms of usage, and place additional demands on networks that already have problems, network performance will become worse. The quality of service has a direct impact on the cost of reducing user subscription conversions, retention and profits. Therefore, improving service quality is a top priority for service providers.

Although network monitoring solutions are well known in the art and widely adopted by service providers, currently available solutions can only monitor and diagnose a portion of the entire communication system, thus efficiently identifying and solving quality problems. It does not provide the network-wide observation and device performance needed to do so. Typical approaches to network monitoring include "self-monitoring" in which a network element reports its status and performance and reports any errors that occur during operation. Measuring the resulting behavior from a single factor can often be an indicator of a wider system-wide problem than providing extended problems that often involve guessing, problem solving, and wasting valuable resources. Another common approach involves placing probes at various points in the network to determine if network elements are functioning in accordance with the specification. In identifying performance problems with specific network elements, monitor systems, referred to as "sniffers", "log monitors" or "event monitors" are efficient, but interface between network elements Problems coming from cannot be detected. That is, these solutions cannot address cases where a single device is operating in accordance with the specification, and problems arise when these devices interact with each other. A much more complex and subtle set of problems has costly consequences for network operators when services cannot be delivered to end consumers.

Other surveillance approaches known in the art include pre-programmed service monitors in which specific devices to emulate "real world" trading activity perform service transactions. Next, point-to-point performance is monitored and the results are reported. While these solutions detect systemic errors, they cannot detect intermittent or distributed problems, minor failures, or problems specific to the device or end user. In addition, they can only test predicted usage plans and are not designed for interaction between new uses and services.

Specifically, the solutions described above lack the ability to monitor network conditions and intelligently and dynamically define and generate data collection models in response to changing network conditions. Even when employing probes in a communication network, it often means that the probe only provides instructions for the problem, and solving the problem adds a person to be dispatched to a physical location on the network, which adds significant time and cost for problem identification and resolution. Require. In addition, problem solving techniques do not provide a multifaceted view of different network layers, such as the physical layer, the application layer, etc., and do not correlate performance problems across those layers. As a result, numerous quality problems affecting end users are not detected or misunderstood. Thus, such performance arises because the performance data provided by currently available network monitoring solutions lacks the detail and persistence needed to quickly identify, prioritize and solve network problems.

In addition, currently available network surveillance solutions do not adequately monitor and report on the specific end-user experience related to network use, so service providers should rely only on end users who report performance problems to customer service centers. However, that is mostly the case once, and end-user problems cannot be repeated because of the difficulty of describing in detail what he / she experienced, the time of occurrence, and the lack of underlying data that makes the problem useful. Thus, since the service provider cannot historically observe network performance, it may be difficult to understand the practical scope of the reported problem, and the problem cannot be properly prioritized for resolution. Thus, the problem cannot be solved satisfactorily by the reported end user, and can dampen the end user's motivation to faithfully report the problem in the future. In addition, you miss the opportunity to prevent the problem from affecting others and improve the overall quality of your network.

Thus, these situations often occur when the service provider does not realize the cause and reason of the end user's complaint and is away from the company providing the communication service. In addition, since so many end user problems are not reported, it is almost impossible for a service provider to obtain reliable information on the scope of a particular problem. Thus, resources are spent on solving a problem that is largely described but affecting only a small number of end users, and not for problems that are not well described but affect a very large number of end users. As a result, a service provider with a misunderstanding of the scope of the problem may allocate an inappropriate amount of resources for resolution or may fail to recognize priority resolution.

Another surveillance solution known in the art is to gather and analyze information about the performance of the wireless device. Typically, monitoring software can be installed on a wireless device at the time of manufacture or by downloading software on the device. The software can then continue to run in the background to monitor the performance of the device and application. When specific events and errors related to the device occur, the software may collect data related to the error or event and upload the data in real time or later to the data storage device for analysis. However, as will be discussed below, such an approach is not flexible and lacks to be an appropriate or efficient solution.

Because the network monitoring solution described above only monitors the performance of certain devices in the network, wireless device monitoring software can also function in other sets of data points for the performance of the wireless device, but the basic problem is that while device performance information is available, It is disconnected from other performance data from elsewhere, and there is no mechanism for understanding this data in the performance of multiple network devices across the network layer.

In addition, such an approach tends to force the use of preset data collection software. The software can be ordered to collect a specific subset of data, while the software cannot be changed or modified quickly to accommodate rapidly appearing or changing conditions. That is, the software can only collect data that was originally programmed to be collected, and that data can only be collected under the conditions that were originally programmed. Thus, the ability of such software is limited by the programmer's prediction. This lack of flexibility is a significant limitation because it is simply that the programmer cannot predict the events that can occur with a wide variety of conditions of use, problems and communications network operation.

This lack of flexibility is also a problem in the event of temporary network conditions. Specifically, because the software cannot be updated quickly and easily in response to changing network conditions, the window of opportunity to collect data about temporary network conditions may be closed before the software can be modified, and thus the network administrator. Or give others the opportunity to properly diagnose and solve the problem.

Finally, the flexibility of such software can be reduced by the fact that in manufacturing such software, it must be loaded on a mobile phone or downloaded from a website or service. Thus, changes or updates to data collection software are rather cumbersome and time consuming. Since different users can load modified software at different times, there is no quick and reliable way to ensure that a significant number of devices are available to gather the information needed for a particular analysis with the appropriate software. .

In view of the foregoing and other problems in the art, the present invention is directed to the use of a plurality of “end points” or end user devices to obtain “real” usage and performance data across network elements and layers. Systems and methods for implementing a new method for monitoring services in a wireless communication network. Since many errors that can occur with respect to the performance of a particular service can only be seen from the end user's wireless device perspective, the method of the present invention enables the collection of such data that is otherwise not useful. In addition, the data collection methods disclosed herein allow for interactions within and between the plurality of network layers and network elements to be identified and accurately associated with each other. In this way, useful diagnostic data corresponding to the performance of a service can be observed and analyzed within the context of conditions occurring between a plurality of network layers including the physical layer, transport layer and application layer.

While the data collection method is already used in connection with a number of high capacity end points such as networked PCs and unique highly specialized systems with significant limitations such as Mars landing systems, the invention disclosed herein is relatively limited in functionality. (E.g., limited memory capacity and battery life) and interaction with RF networks that provide complex dependencies (e.g., variable capacity and reliability to monitor and acquire network and device performance data according to changing network conditions). It is an attempt to provide a service monitoring technology that depends on a large number of wireless devices. These conditions can sometimes be instantaneous, geographically isolated, and rely on specific interactions between network elements. Thus, the data collection and management system of the present invention addresses a very different set of problems than addressed by data collection systems known in the art.

Embodiments of the system and method of the present invention provide distributed wireless devices, wireless telephones to implement monitoring of one or more wireless communication network parameters, collection of data relating to such parameters, and conditional transmission of collected data to a reporting system. It can be implemented by selecting and using a portable personal digital assistant that can be used as a telephone and a telephone. Regarding the monitoring and data collection activities of wireless devices, in some embodiments, the methods of the present invention may, in addition to wireless devices, include wired devices, application servers, or other servers on a network where data may be collected and processed as described above. It should be noted that it may be implemented using an inclusive network node.

One embodiment of the present invention provides a quality of service platform of a data collection and management system for dynamically generating and downloading rule-based data collection files to a group of wireless devices. The data collection profile can be manually generated by a network administrator, software developer or other parties involved in the operation of the network (hereinafter referred to as "network administrator"), or can be generated offline as part of a data analysis solution. Can occur automatically based on network parameters or other events. The profile defines what conditions and events should be collected on the device in response to the conditions and events that cause the device to upload the collected information. Conditions or events include any event on the network and the device that the device can detect, such as giving up a call or pressing a user button on the device. Conditions and events also include the passage of time or a request from the network administrator to have the device report information back to the server. In this embodiment, a group of wireless devices that receive a data collection profile and perform data collection (hereinafter “wireless device (s)”, “wireless communication device (s)”, target wireless device (s) ”or“ devices ”). (S) "may be selected based on the characteristics, capacity, and usefulness of the end-user wireless device. As part of the target wireless device quality and selection process, it is determined that a particular process overwhelms the end-user devices. In order to prevent the collection of data, it is instructed to implement the data collection rules in the data collection profile. Rules defining the upload of data are defined to avoid adversely affecting network performance, for example collecting in some cases. Uploaded data can only occur at off-peak hours, especially when the device and network activity is low, and in addition to strict device quality processing, network monitoring and data collection activities are expensive in terms of the device and network resources used. As such, the method of the present invention provides the ability to structure a data collection profile to ensure that unnecessary data is filtered during the data collection process, and that only the most relevant and reliable data exists for analysis of a particular problem.

The present invention can be used in connection with a wireless communication network, such as a wireless cellular telephone network, comprising a plurality of wireless devices, such as a wireless telephone, properly established to communicate with a network. The data collection and management system includes a Quality of Service Platform (SQP) that generates and manages data collection activities and processes and Quality of Service Clients (SQCs) present on them to engage target wireless devices or other target network nodes with data collection activities. It consists of

In operation, the data collection and management system of the present invention allows a plurality of wireless devices and network nodes, each with an SQC installed, to provide information about complex network state problems. The following discussion shows an embodiment of the invention in which queries are used to obtain such information, but the invention is not limited to the following modes of operation. According to the present embodiment, complex network state problems may be described in terms of the term "query", ie, the state or performance of a network device or a query about other aspects of network performance. In response to initiating such a query, the SQP determines whether it has data stored in one of the subsystems that meets the conditions specified in the query. If so, the data can be used for analysis and the query gets the answer immediately. If the requested data is not present in the SQP, a collection work request (CTR) will be generated and processed by the SQP's work subsystem. CTRs are typically predefined as part of a data collection and analysis package that addresses the particular industry or technology required. The CTR is then stored in the SQP and called as the request occurs. In some cases, the CTR may be defined at the request of the network administrator. Next, CTR is used as the basis for building the data collection profile. The profiles describe what parameters should be obtained to each target wireless device receiving it and which events will be acquired, stored and uploaded data. Once the data collection profile is created, the set of target wireless devices is selected based on the conditions related to the characteristics of the CTR and available wireless devices. Quality characteristics include usage statistics, including device type such as make and model, available memory capacity and battery life, type of application installed on the device, geographic location of the device, user interaction with the device, and customer It may include a profile.

As discussed above, in some embodiments, the data collection profile is also sent to other network nodes, such as an application server. The use of the term "target devices" herein refers to a broader category of devices, including wireless devices and other network nodes that may be employed by the SQP for data collection. According to the instructions specific to the profile, the SQC on target devices and other network nodes collect, store, package and upload the specified data. Finally, the SQP receives the reported data and processes it to provide information useful for decision support. Based on the reported data, additional queries and data collection profiles can occur and can be performed to further investigate and diagnose the problem. For example, additional queries may be automatically generated based on the set of rules, target wireless devices may be automatically selected, and new queries included in the data collection profile may be sent to the selected devices. In other situations, existing data collection profiles can be maintained on the device for any time and allow the SQC to continue to collect, package and upload data periodically. More generally, data collection profiles can be deactivated, changed and replaced at any time.

These and other aspects of embodiments of the present invention will become more apparent from the following description and the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS In order to obtain the above-mentioned and other advantages and features of the present invention, a more detailed description of the present invention briefly described above will be provided by referring to specific embodiments thereof described in the accompanying drawings. It is to be understood that these drawings are intended to illustrate exemplary embodiments of the invention, and therefore should not be considered as limiting the invention, the invention will be described and described with additional features and details through the use of the accompanying drawings. Will be.

1 illustrates an embodiment of a wireless communication network in which a data collection system may be implemented.

2 schematically illustrates aspects of a data collection and management system suitable for use in connection with wireless network operation.

3 is a schematic diagram illustrating various considerations related to queries that may be performed by a data collection and management system.

4 is a schematic diagram of a data collection profile that may be employed by an embodiment of a data collection and management system.

5 is a flow diagram illustrating aspects of an example process for formulating and providing a query.

6 is a flow diagram illustrating aspects of an example process for generating a data collection profile.

7 is a flow diagram illustrating aspects of an example process for collecting data in connection with a data collection profile.

8 is a flow diagram illustrating aspects of an example process for using collected data in connection with a data collection profile.

9 is a flow diagram illustrating an example process for trigger activating a data collection activity.

10 illustrates an example process for invoking a domino trigger.

Reference will be made to the drawings to describe various aspects of embodiments of the invention. It is to be understood that the drawings are schematic and schematic representations of such embodiments, and therefore, are not intended to limit the scope of the present invention or to be necessarily drawn to scale.

Embodiments of the invention relate to systems and methods of using distributed wireless devices and other network nodes to implement monitoring and data collection on selected communication network parameters. The collected data can then be analyzed and processed by focusing on the data collection results as suggested by the collected data and related directions. Data analysis can be used above all with regard to network failures and outages, the implementation of correct behavior, the rationalization of network behavior, the improvement of customer service and the development of marketing strategies.

I. Example Wireless Communication Networks and Wireless Devices

To illustrate various methods of the present invention, FIG. 1 shows an example of a wireless network 100 that may be practiced with the present invention. Of course, such an apparatus or process described herein is for illustrative purposes only, and other arrangements and elements (e.g., machines, interfaces, functions, order of elements, etc.) may be added or replaced and some elements may be omitted. It should be understood that it may be. In addition, many of the elements described herein to those skilled in the art may be implemented by software, firmware and / or hardware in any suitable combination and position in combination with individual elements or other elements. It will be obvious that it is a functional entity.

The network of FIG. 1 merely illustrates an example of a suitable environment in which the present invention may be implemented, and other network structures are possible. In particular, although wireless network 100 is described and illustrated as a Code Division Multiple Access (CDMA) network, the present invention relates to a Global System of Mobile Communication (GSM), Universal Mobile Communication System (Universal). Mobile Telecommunication System (UMTS), Time Division Multiple Access (TDMA), Wideband Code Division Multiple Access (WCDMA), General Packet Radio Service (GPRS) networks, It can be implemented with other wireless networks, including 802.11 networks and other networks to be developed in the future. More specifically, in an embodiment of the present invention, the wireless network 100 includes a radio frequency (RF) network that provides wireless connection and session management for circuit switched and packet data technology based communications. Accordingly, wireless network 100 includes one or more wireless devices 400, base station 120, base station controller (BSC) 130, mobile switching center (MSC) 150, and packet data support node (PDSN) 140. It includes, or interfaces with, all the elements needed to route circuit switched telephone calls and / or packet data communications over the network. More specifically, the methods of the present invention can be used in connection with a circuit switched network, a packet data network, or both. Although an embodiment of the present invention uses the Internet Protocol (IP) as its transport protocol, the methods of the present invention provide other transport protocols such as short message service (SMS) and short data burst service, which are well known in the art. Can be implemented using Wireless network 100 also includes the infrastructure necessary to use a Home Location Register, Visiting Location Register, billing and allocation system, one or more gateways, servers and short message service messages. And many other network elements not shown in FIG. 1. Application servers or third party wireless systems (shown as network server 102 in this embodiment) may be installed outside network 100 and used in connection with the present invention. The term "wireless network" or "network", which is still referred to herein, should be interpreted to include network infrastructure, servers, end user devices, and applications and servers.

Wireless network 100 may further include third party network monitors, probes, and packet detectors known in the art for use in connection with the present invention. The data collection and management system 200 shown for illustrative purposes operating in connection with a packet data network may receive data from a node on a circuit switched network, or the elements of a system located within the circuit switched network may receive data from the packet data network. Can be received. Gateways, network monitors, and other network elements may be employed to enable data transfer between the network and the data collection and management system 200. In addition, in one embodiment, the elements of data collection and management system 200 may be implemented on one or more network servers within a network operator's network. Optionally, the data collection and management system 200 may be implemented as a service hosted by a service provider other than the network operator, such that the elements of the system are located outside the network operator's network to allow various nodes within the network operator's network. It can be installed to communicate with them.

II. Structure of an exemplary data collection and management system

FIG. 2 shows details of the data collection and management system generally indicated at 200, as well as the implementation of some of the relevant components of the wireless network 100 that may be used with it. The functions disclosed herein may be employed in connection with a wide variety of wireless or wired network devices, including radios, portable personal digital assistants (“PDAs”), network servers, desktop computers, and other devices related to network communications.

In this embodiment, the data collection and management system 200 of FIG. 2 uses one or more target wireless devices 400 that an end user has access to a telephone service and the target wireless device 400 is connected to a packet data network. It is installed to provide access to the Internet or multimedia data through any suitable protocol to participate. The target wireless device 400 can be virtually any mobile wireless device capable of communicating in a wireless network 100 environment, and can be a wireless telephone handset, a wireless portable personal digital assistant (PDA) or other wireless communication device. Wireless devices may also be referred to as mobile stations or mobile devices, hereinafter referred to as wireless devices.

The target wireless device 400 has a local memory that is transmitted over the wireless network 100 and used locally to store data that permits the execution of software such as operating systems and applications, ie to support the operation of the wireless station. . Various known operating systems may exist within the wireless device, such as Symbian, REX or other suitable operating system. In addition, the target wireless device 400 may participate in data collection activities by receiving and executing a data collection profile, in other words by communicating with a quality of service platform (SQP) 201 of the data collection and management system 200. Quality of Service Client Software (SQC) 402 is installed. SQC 402 may be installed on a wireless device by a variety of methods, including various download methods, such as placing the device in a cradle that is installed at the time of manufacture or connected to a computer via a cable, or through a variety of methods over air. Can be installed.

Development environments such as BREW and JAVA may also be present on the wireless device to support such download of SQC 402 software or in other words to support data collection system operation. In addition, typical target wireless device 400 includes a variety of application software that allows a wireless device to access one or more services provided by a network service provider and provides utilities or entertainment for users such as calculators and games. A typical target wireless device 400 also includes a wireless transmitter and receiver, circuits for voice encoding and decoding and call control, display devices, a keyboard, a power supply, and a Subscriber Identity Module (SIM) chip. It may include. Note that in other embodiments, the SQC 402 may be installed on other nodes in the network to receive and execute the data collection profile described above. For example, in addition to data collection activities of wireless devices, the SQC software can be installed on an application service or on a PDSN in the network so that these network nodes can participate in data collection activities.

With continued reference to FIG. 2, the data collection and management system 200 includes an SQP 201 and an SQC 402. The elements of the SQP 201 include a matrix collection subsystem (MCS) 202 consisting of a matrix collector 202A and a matrix analyzer 202B; A job subsystem 204 composed of a job processor 204A, a job database 204B, and a target device database 204C; A profile subsystem 206 composed of a profile builder 206A, a profile editor 206B, and a profile database 206C; A reporting subsystem 208 composed of a reporting engine 208A, a rule engine 206B, and a reporting database 210; And an interface subsystem 212 composed of a user interface (UI) 212A and an API / service interface 212B. The data collection and management system 200 may also include one or more target wireless devices 400 or other network nodes where the SQC 204 is present. Additionally, data collection and management system 200 may communicate with one or more network servers 102, such as a third party reporting system.

As discussed above, the data collection and management system 200 of FIG. 2 may be implemented as a service hosted by a service provider other than the network operator, and in other embodiments some or all of the functions of the data collection and management system may be networked. It may be provided in the form of a standalone software package suitable for installation on a server or on another network system within the network operator's wireless network 100. Thus, the scope of the present invention should not be construed as limited to the implementation of any particular data collection and management system 200.

The SQP 201 of the data collection and management system 200 generally functions to generate and manage various queries (CTRs) and to generate data collection profiles that can be employed for data collection. In addition, the SQP 201 receives the collected data and performs various analysis and other processes on the collected data.

With continued reference to SQP 201 of FIG. 2, matrix collector 202A of MCS 202 receives data collected from a plurality of wireless devices 400 or other network entities. The collected data may be provided to the metrics collector 202A directly or through other network nodes such as PDSN 140 or gateway 160. MCS 202 may communicate with working subsystem 204 and reporting subsystem 208. Matrix analyzer 202B performs verification, augmentation and / or analysis functions such as input matching of received data before passing the received data to other subsystems of SQP 201. The MCS 202 also receives and responds to a request for a collection profile from the SQC 402 operating device. For example, upon initial activation of a device operating as SQC 402, the SQC indicates its presence through the MCS 202 to the SQP 201 where one or more profiles can be downloaded to the device in response to it.

The reporting subsystem 208 stores in the reporting database 208C reports generated as a result of input from the MCS 202 and data from external systems such as third party network monitors. The reporting database 208C may also send or publish report data to other reporting or presentation systems, which are shown as network 102 servers in this embodiment. The reporting database 208C may be integrated into the data collection and management system 200 or other network elements, or may function as an independent database. The reporting subsystem 208 also plays a central role in the collection work process described below.

In one embodiment of the present invention, information about complex network state problems can be obtained using the following process, although the present invention is not limited to the following modes of operation. According to this embodiment, when processing a new query, the reporting subsystem 208 first checks whether the reporting database 208C already has enough and necessary data to answer the query. If you have such data and no additional collection is needed to satisfy the query, the data and / or report is returned to the network administrator. If the reporting database 208C does not contain data to answer the query, a CTR occurs and follows the entire process by the reporting subsystem 208 and processed by the work subsystem 204. The job subsystem 204 selects an existing data collection profile or generates the necessary new data profile, identifies the appropriate set of target wireless devices, sends the data collection profile to the selected target wireless device, Communicate with profile subsystem 206 to communicate with the reporting subsystem to maintain track. If the query requires collection from a non-SQP device, i.e., requires collection from a device that does not have SQC installed to participate in the data collection and management system 200, the reporting subsystem 208 is responsible for the interface subsystem. It will generate and transmit a CTR for the external system via the system 212. While not processed by the data collection and management system 200, in this manner, the required work while not instructed by the data collection and management system 200 is the context of the system and other data collection activities connected by the system. It will be documented, tracked and reviewed in the context. It should be noted that job subsystem 204 and / or reporting subsystem 208 may also consider the priority of the query and / or CTR when generating the data collection profile and identifying the target device number. It is also noted that some of the work functions described above may be replaced by an operator, and in accordance with another embodiment of the present invention the operator directly assigns profiles to the device.

The SQP 201 may communicate with one or more network servers 102 to provide data collection and management services. For example, reporting subsystem 208 of SQP 201 may provide reports, raw data, and other data products to network servers 102. Network administrators and external systems can then access the data by the network server 102. The elements of the data collection and management system 200 may also initiate communication with one or more target wireless devices 400 and inform the network administrator that the request data may be collected and analyzed for a short message service center (SMSC) 180. And communicate with various message services within the wireless network 100, such as:

The SQP 201 is a task subsystem 204, rule engine 208B and target device database 204C for generating or modifying various data collection profiles that control the collection of data regarding various network operations and parameters. It may further include a profile subsystem 206 that cooperates with. For this purpose, profile subsystem 206 includes profile builder 206A, profile editor 206B, and profile database 206C. Note that the profile subsystem 206 can be configured and implemented in a variety of ways. In one embodiment profile subsystem 206 includes elements of SQP 201d. In another embodiment, profile subsystem 206 is an independent system designed to communicate with data collection and management system 200. Thus, the scope of the present invention should not be interpreted solely as an implementation of the profile subsystem 206 shown in FIG.

The data collection profiles generated in association with the profile subsystem 206 may then be downloaded to a designated set of target wireless devices 400. Tools at this time can also be used to identify target devices compatible with the profile and appropriate for the query. The SQC 402 residing on the target wireless device 400 receives the data collection profiles and “triggers” defined in the profile that initiates and terminates the collection activity of the data, as well as other rules within the data collection profile. Perform a data acquisition process in response to commands. Data collection rules, triggers, and other instructions contained within a data collection profile are often referred to herein as "data collection instructions" for convenience. As discussed above, the target wireless device 400 may be a mobile phone or other wireless device. However, as described above, the target device can be any other network element or node that has SQC software installed that allows data to be collected and uploaded to the SQP 201.

Data collected by the target wireless device 400 may be buffered in the SQC 402 while awaiting generation of the metrics package. Once created, the metrics package can be included in the metrics archive of the SQC 402, deleted, and uploaded to the metrics collector 202A of the MSC 202.

The UI 212A of the interface subsystem 212 may be a graphical user interface that enables one or more network administrators to interact with and control the operation of the data collection and management system 200. In this regard, at least some implementations of data collection and management system 200, UI 212A, may be directed to the generation of queries by network administrators regarding data collection relating to one or more aspects of wireless network 100, discussed below. It should be noted that the manipulation is possible. In some cases, such queries may be automatically generated or processed based on reports generated and output by reporting subsystem 208 or as a result of processing by rule engine 208B. In addition, the UI 212A can be used by the network administrator to collect data and reports from the system and the API / Service Interface 212B provides commands and requests to the system and originates from the data collection and management system 200. It can be used by external systems to retrieve reports and data.

Finally, target device database 204C includes various types of information about target wireless devices, such as mobile phones, and other network nodes, including elements of wireless network 100. Target device database 204C may be implemented in many different ways. It may be an existing network element in a service provider network, such as a home location register (HLR) where information about the characteristics and behavior of the wireless device is stored and accessed by the elements of the data collection and management system 200. Optionally, the target wireless database 204C may exist within the data collection and management system 200, which collects device usage information and device characteristics when the device is registered with the wireless network 100. In some cases, a data collection profile specifically for collection information for presenting the target device database may exist on the wireless device, gathering status and usage information about the device, such as the application being used, the opposing base stations, and so forth. This information is reported to the target device database 204C. By providing such data and periodic updates to the target device database 204C, the SQP 201 can fine tune device selection conditions for more efficient device targeting. In many cases, the target device database may make use of data from multiple resources, device characteristics, or network characteristics, including HLR data, data generated by profiles, and data from other databases associated with user accounts. As a result, the data collection profile is downloaded to these target wireless devices 400 that are most suitable for data collection in response to the defined query or queries. It should be noted that the interface to multiple resources described above is well known in the art and therefore is not described herein.

Thus, the target device database 204C may include the model number and the manufacturer of the target wireless device, information about specific hardware features of the target wireless device, such as account type or formats associated with the target wireless device, battery type and memory size, and target wireless device. Information such as billing address, user information, installed applications, favorite geographic areas, and any other information that enables selection of one or more target wireless devices 400 associated with a particular query about the communication network. have. The database may be created from information collected from the device through a data collection and management system or by any other suitable means. In addition, the foregoing are merely exemplary, and any other target wireless device characteristic that is more general and substantial that may be considered desirable and relevant to be employed may be included in the target device database 204C.

Additionally, since the data collection and management system 200 allows multiple data collection profiles and data analysis activities to be performed at the same time, specific devices can be targeted to perform multiple data collection profiles. Thus, the target device database 204C detects data collection activity taking place on the device and maintains detailed information about the specific data collection profile that is activated on the device. By doing so, it is possible to detect and solve any content or priority problem by adjusting the number of selected target devices.

The target device database 204C also provides information about non-SQP entities in the network, including characteristics such as the location of the non-SQP device in the wireless network 100, geographic location serviced, available data, and the like. can do. In general, substantially any other characteristic or attribute of a non-SQP entity that is deemed relevant may be stored.

Finally, it should be noted that FIG. 2 illustrates only one way in which various functions associated with the data collection and management system 200 may be assigned. Thus, the functions shown herein in connection with the data collection and management system 200 include, but are not limited to, various types of distributed systems and devices employed, structures of data and communication networks suitable for collection, or other systems. Depending on the non-limiting considerations, it can be assigned in various other ways. Accordingly, the functional assignment shown in FIG. 2 is merely exemplary and should not be construed as limiting the scope of the invention in any way.

III. Example queries ( Queries )

Referring now to FIG. 3, details are provided regarding methods and systems for defining and querying questions in connection with the data collection and management system 200. Such queries may be defined with reference to a variety of different expectations regarding the operation or function of the wireless network 100 and with respect to target wireless devices 400 or other network nodes. Queries collect performance information about the effects of simple activities such as button presses by the user, or more complex interactions involving multiple network layers such as the physical layer, network layer, transport layer, and application layer. Can be structured in a collected manner. In particular, the target wireless device has software stacks that communicate with various network layers of the communication network.

Once collected, the information related to the device's software stacks is packaged together with information about the software stacks or network layers with which they communicate so that a more accurate analysis of the information and complete observations of interdependence between the different layers Can be correlated to be observed within the context of the appropriate RF state information to provide. This may be merely an example query subject, but such queries may be more generally directed to substantially different aspects of the wireless network 100 associated with the target wireless device 400 or other network nodes that may be of interest to network administrators or others. It may be about.

An example query structure is indicated generally at 500 in FIG. 3. The query structure 500 may be designed in any form in accordance with the functions disclosed herein. As shown in FIG. 3, one way in which the query structure 500 may be defined is through interaction between the network manager and the SQP 201 by the interface subsystem 212. If defined by the network administrator, the query structure 500 may appear as a pop-up window or other form of device that allows the network administrator to make various choices regarding the nature or scope of the appropriate query or queries. . The network administrator has the ability to interfere with the system and write questions in the query field, while the preferred embodiment of the present invention is stored in a number of questions and profiles database 206C that are "predefined" within the reporting subsystem 208. Provide profiles. As commercial and technical demands for data collection arise, these predefined questions are called to generate a complete analysis package that includes one or more data collection profiles and related reports.

The query structure 500 of any number and type may be used to determine the performance of the wireless network 100, one or more related target wireless devices 400, or any other services or entities provided in connection with the wireless network 100; It may be configured to be able to predefine or generate and submit queries concerning substantially all aspects of the operation. Thus, the query structure 500 shown in FIG. 3 is merely illustrative and is not intended to limit the scope of the present invention in any manner.

In yet another embodiment, one or more query structures 500 are defined that do not include any predetermined menu selection. Rather, the network administrator defines a highly personalized query structure by simply dragging and dropping certain items into the basic query structure template. Profile system 206 may be configured to prevent the definition of inconsistent or inconsistent queries and related data collection profiles. In this way, network administrators can ensure to some extent that no “noise” data is collected for a given query or data collection profile.

Also, queries as well as cases other than the data profile disclosed herein may occur manually or automatically in response to the occurrence of one or more system events or as a result of analysis of previously collected data. The data collection profile includes "triggers" that indicate the start and end of the data collection activity. Aspects of the definition and use of triggers are described in more detail below.

3 conceptually illustrates various aspects of a query. In particular, the query structure 500 allows a network administrator to target devices 502, software applications 504, services 506 associated with target devices, and radio frequency system 508 employed by the target device. Information, and any other other considerations or parameters 510 that may be of interest to the network administrator or others, including the various aspects of the data collection and management system itself. Specifically, such queries in any combination are directed to one or more of the above-described aspects of activity taking place on the device or within a network that allows the collection of data that occurs in connection with the activity.

In generating a query regarding aspects 502 of the target device, the developer more specifically targets a query for characteristics 502A, such as hardware, software, and other aspects of the device, such as the target wireless device 400. can do. Similarly, queries relating to applications 504 running on a target device specifically include the execution of applications residing on the target device, usage of each of those applications, and applications associated with or installed on the target device or wireless network 100. May be specifically targeted to characteristics 504A, such as various other features of.

With continued reference to FIG. 3, a query relating to services 506 useful to a target device may be defined for a particular service 506 including IP messages, voice transmissions, and various other services related to the wireless network 100. . In a similar form, the query may relate to RF related to aspects 508 of wireless network 100 and / or target wireless devices 400, such as RF performance parameters, problems, and other considerations.

Embodiments of the data collection and management system 200 may involve one or more network administrators or other parties in relation to target wireless devices of which the manager (s) may be interested in or inquiring about the actual aspects of the wireless network 100. Make it possible to define or implement them. As a result of such functionality, network administrators are not limited to the prescribed selection of query subjects, but can more generally define one or more queries for any subject that may be of interest. In addition, query structures 500 and their related systems allow specific forms of questions to occur or be used that correspond to a particular set of interests.

For example, some question structure 500 considers only information that may be of interest to marketing departments, such as the dissemination of applications used. In another example, an accounting department alone or in association with a marketing department may want to use various service plans relating to a particular group of end users and queries related to the behavior of these end users. In another example, technicians associated with wireless network 100 and target wireless device 400 may wish to invoke queries about the various performance aspects of the network and target devices. Technologists may also be interested in the performance of other network nodes, such as media servers or packet data support nodes. Thus, such parties will have access to a plurality of predefined queries as well as a query structure that allows them to quickly and easily define a plurality of useful queries that can be used to identify problems and improve services.

The set of queries described above may all come from a single query phrase 500, or each may constitute a unique query structure that is accessible only to specific people. However, these are only examples and the scope of the present invention should not be construed as being limited to these examples.

Ⅳ. Example Data Collection Profiles

As noted above, the wide variety of predefined queries for numerous aspects of the wireless network 100, associated target wireless devices 400, and other components that make up the wireless network 100 monitors performance data. Can be used by data collection and management systems to collect. As discussed above, in this case network administrators can create queries for customer needs. Thus, once a query or queries are defined and it is determined that existing data is not sufficient to satisfy the query, the CTR may be associated with the target wireless devices 400 or other network nodes in the collection of data for the query or query. Causes one or more data collection files to be generated or selected.

In this embodiment, data collection efforts are pushed to a group of target wireless devices 400 for data collection activities. The data collection cost imposed on each target wireless device 400 should be minimized since the demand for each device, including any existing work request, is already considered as part of the data collection conditions. In addition, as described herein, the collection capabilities and efforts of the various target wireless devices 400 provide a strong mechanism for collecting data easily and effectively in response to queries.

Aspects of the data collection profile 600 described above are described in FIG. 4. As generally indicated in FIG. 4, the data collection profile 600 includes various parameters 602 that define the set of data to be collected as well as the conditions under which the data collection will be performed and the manner in which the data will be collected and processed. . In this regard, the data collection profile 600 is for specifying the format and quality of the data for the query defined by the query structure 500 when the data is collected and processed. Constitutes an improvement of related queries.

The data collection profile 600 may be set to be activated (ie, starting the collection process) and deactivated after a certain time or period of time. The data collection profile 600 and / or its configuration collection command may also end after a period of time. Upon completion of the data collection profile and its configuration data collection command, the device may return to its fixed profile, or may notify the SQP 201 through the MCS 202 of the expiration, resulting in receiving another profile. Can be. In another case, the termination capability may allow the device to effectively "remove" obsolete profiles or individual collection instructions and be ready to receive new data or collection instructions. The data collection profile can also be set to include a plurality of triggers that start or end data sampling in response to a particular occurrence for a predetermined time. In one case, data collection may occur in response to a trigger defined as the occurrence of the event. In other cases, data collection can begin or end in response to a trigger defined by the lack of event occurrence. For example, a data collection profile specifies that data collection begins when arriving at a particular geographic area and stops if the user does not receive a voice call within an hour after entering the particular geographic area. In this case, entry to a particular geographic location is a start trigger that causes data collection to begin and the data to be buffered, causing the end trigger to stop data collection when the voice call fails to occur within an hour. In this case, the buffered data may not be sent to the metrics package and may or may not be deleted.

In some cases, data collection may be terminated and then restart certain network event occurrences. If this second event must have failed to occur, that is, the trigger will not be activated within the next period and the data collection profile will be terminated. The data collection profile may include instructions regarding the conditions under which the collected data is sent to the metrics package and when the metrics package should be unloaded into the metrics collector 202A. In other cases, data collection profile 600 may indicate whether information about both the occurrence and non-occurrence of a particular event, as well as whether information has been collected for a period of time, is reported to matrix collector 202A. .

In this embodiment, the profile builder 206A of the profile system 206 targets to facilitate the definition of the data collection profile 600 and to define a set of target devices to which the data collection profile 600 will be sent. It operates in conjunction with the device database 204C. As shown in FIG. 2, profile subsystem 206 is a profile editor that allows network administrators to retrieve and edit one or more existing data collection profiles when the environment requires manual change of such data collection profiles. 206B. In addition, once completed, the data collection profile 600 may be stored in the profile database 206C for reuse, backup purposes and further editing, copying, and other processes.

In all cases, the data collection profile 600 shown in FIG. 4 specifies, among other things, the format or formats of data to be collected in relation to the data collection profile 600. As discussed earlier, the data collected with respect to the data collection profile 600 may include substantially all forms of data relating to the wireless network 100 and / or target network devices 400. Thus, the data that must be collected includes the UI of the target wireless device 400, the performance of other network elements such as applications or applications mounted on the target wireless device 400, applications and network servers, and the transport and physical layers. It may be about considerations regarding various network layers as well.

In addition to specifying various data formats, the described data collection profile 600 may also specify a time parameter, ie a time frame or time frames, during which some or all of the specific data is to be collected. For example, as a result of the reporting of call abandoneds in a particular area, a data collection profile for call abandoned analysis may occur. In this case, the data collection profile may specify that data collection should begin as soon as the wireless device enters a particular geographic area. The collected data is entered into a buffer on the device, so it can be included as part of the metrics package if a trigger occurs that the metrics package should be generated. The data collection profile may also specify that as part of the data collection rules, a five second buffer of information about all previous call occurrences and data collected during the call should be stored while the device is within a particular geographic area. If the call is abandoned, an additional five seconds are obtained after the call is abandoned, and the data obtained before, during, and after the call is included in the metrics package.

In some cases, the data collection profile may specify that certain data should be collected according to a predetermined schedule format. In other cases, the data collection profile 600 may specify that some data should be collected regularly or periodically, while other data is collected irregularly, only once or not periodically. In another case, the data collection profile may predefine a start and / or stop date that specifies that data collection should begin on a certain date and end on another particular date, despite any other conditions.

With continued reference to FIG. 4, data collection profile 600 may be used to determine whether a data collection process specified within a data collection profile should be started, stopped, and terminated, and various triggers used to determine when to start, stop, and terminate. Include, imply, or comment on them. For example, one technique known in the art (also known as Push to Talk®) for initiating network-based instant connect communication is to establish a session initiation protocol (request to participate in one or more terminating wireless devices in a communication session). Session Initiation Protocol (SIP) A start wireless device that transmits INVITE to a network-based connection server. The INVITE is received by the instant connection server and sent on the terminating wireless device. If the terminating wireless device has recently been activated, the server may send a "100 attempt" message to the initiating device while continuing to contact the terminating device.

Typically, when a 100 TRYING message is received at the starting device with an indication that the floor is open, the starting device begins to transmit voice data. While the initiating device is busy sending voice data, the instant connection server may fail by actually attempting to reach the terminating device, and a 408 TIMEOUT message may be sent to the initiating device. At this time, the user of the starting device has a very poor use experience. At the same time, the service provider does not have the ability to obtain all of the interactions that occur or fail to occur between the terminating headset and other network elements such as the initiating wireless device, instant connection server, packet data support node (PDSN). Thus, it does not have any data to repair or troubleshoot the fault.

Using the method disclosed herein, one embodiment provides a data collection profile installed on a starting device to begin collecting data when the start trigger is activated—in this embodiment, instant connection communication is attempted. . The reception of the 408 error message described above by the originating device is a terminating trigger condition that causes the SQC 402 installed on the originating wireless device to stop collecting data and to prepare the unloading of the metrics package to the matrix collector 202A. In some cases, the trigger can cause the data collection activity to stop without generating a metrics package. In this embodiment, the data collection profile may also be set to specify that a 402 error should be prepared immediately by the metrics package and unloaded into the metrics collector 202A, thus configuring the conditions under which the metrics package is unloaded. In this example, the terminating wireless device does not recognize that a communication attempt has been made and therefore no trigger is activated to begin collecting data and / or to generate a metrics package on the terminating device. In order to ensure that important data related to the communication attempt is collected from the terminating device, receipt by the metrics collector 202A of the metrics package from the originating wireless device results in a command such as an SMS message being received at the terminating device. This command activates a trigger that causes an "on-demand" data collection activity on the terminating wireless device, forces any additional data collection needed, and unloads the metrics package at a particular point in time. It should be noted that "on-demand" data collection activities can occur in addition to extraneous data collection activities that may already occur on the device as part of normal operation. In other cases, the termination device may have a profile that allows the continuous collection of data related to the problem being specifically investigated. In this case, it will be the time and frequency of INVITE reception. As a result of the on-demand upload, a history related to the particular area under consideration is uploaded to provide full visibility of the events that occur at the terminating device and the data surrounding the 408 problem being investigated. In addition, in some situations, the collected data is processed in a minimum / maximum / average / coefficient format at the device or other calculations that collect or reduce the amount of data to be sent from the device to the SQP. In this case, the data to be sent to the metrics collector has already been processed, not just the data initially collected.

In another embodiment, the SQC can be installed on another network element, such as an instant connect application server, and the data collection profile can be set up immediately to upload the metrics package in the event of a 408 error. In this case, if the 408 error is recognized by the instant connect application server, the SQC causes the metrics package to be generated and uploaded to the data collector 202A. SQP 201 then causes an SMS message to be sent to the terminating device, in which case an "on-demand" data collection activity is initiated on the terminating device and processed as described above. More specific information regarding triggers associated with the data collection profile and various other aspects of the data collection and management system 200 are likewise provided herein.

As soon as the metrics package is prepared on the target wireless devices 400, the SQP 201 receives from the starting and ending wireless devices and from any other network elements with data collection profiles associated with the instant connection communication attempt. Analysis work can be embedded into the reporting subsystem 208 ready to analyze the prepared metrics packages. The metrics package originating at the terminating wireless device is assigned an identifier that associates with the metrics package originating at the originating wireless device and / or at the instant connected application server and any other network elements associated with the interaction. At any time, queries can operate on reporting subsystem 208 to identify metrics packages associated with other metrics packages, thus completing events occurring between all network elements involved in interactions between devices. Order can be observed, tracked and analyzed. Reports originating from reporting subsystem 208 may be stored in reporting database 208C for later use, and basic data may be exported to third party reporting system via interface subsystem 212. Additionally, in some embodiments, notifications regarding data reception and analysis results may be sent to other systems as well as network administrators and other parties.

As another example of the parameters used to guide the data collection behavior, the data collection profile 600 can specify certain geographic limits and restrictions. More specifically, such geographic restrictions may be collected only by target devices where the data is located within or receives calls within a particular geographic boundary, or only by devices in an area proximate to a particular base station. The data collection profile may also specify that only devices that have experienced a particular problem in a particular geographic area collect data. For example, such information may allow network administrators and other stakeholders to isolate repeated problems to specific areas, pinpoint the cause of them, and clarify steps to resolve them. For example, such geographic information may also be used to help identify failed base stations or other devices if there are an excessive number of call abandoneds within a defined geographic limit. The information also helps to identify that a device, such as a signal enhancement device, is located within the network to enhance service. In addition, it enables the development of pictorial descriptions that point to the concentration of geographic information and various conditions and / or problems associated with the wireless network 100.

As further shown in FIG. 4, the data collection profile 600 specifies a specific service to consider for the data to be collected. Some embodiments of the data collection profile 600 may specify that transmit signal (“Tx”) strength data for an Internet Protocol (IP) data call initiated by an instant connect communication session should be collected. In other cases, the data collection profile 600 may specify that data regarding the message service accessed by the target wireless device 400 should be collected. Because wireless devices use software stacks that participate in each service to be delivered to each end user and communicate with each network layer required to transmit that service, collect metrics related to a particular service and when not using that service. The ability to ignore the metrics offers significant benefits over existing solutions. For example, to monitor the transmission signal strength under the conditions described above, network probes known in the art can be used across all transmissions (eg voice or IP) for all wireless terminals, You will have to monitor the transmit power for all services (eg, phone, message, browser and instant communication connections). The size and range of such data collection processes may render data incapable of collection by any method other than those disclosed herein in connection with the present invention.

In some cases, a data collection profile can be defined to collect information about the performance of a service when other aspects of the system are within certain configurations. For example, the query may include analyzing the poor performance of wireless application protocol (WAP) browser pages on the service provider portal from a particular location. WAP browsing sessions use an IP layer that operates on an RF layer that is connected to the portal through large IP networks, WAP gateways, and other IP networks. Some of these systems can cause performance degradation, but RF degradation is a frequently encountered problem. Therefore, it is useful to be able to filter the RF related data to discover other possible resources for poor performance. In this case, the target wireless device 400 is equipped with a data collection profile that is adapted to collect data about very specific network conditions under conditions where the WAP page load is longer than a particular configuration type. The device's request for the WAP page load is a start trigger that starts data collection. The data acquisition profile analyzes RF performance conditions such as frame loss and automatically ignores events that can occur in high loss conditions. Since RF is typically the most unreliable device in the network, this simple filtering ensures that the most frequent and most unrelated causes of delays in WAP page loading are ignored and the focus of the data is on a higher layer. Dramatically increase the value of the reported information. Similarly, the data collection profile can also ignore the results of where IP services are not performing properly, collected within scenarios. In addition, since the wireless device includes a software implementation of all service layers, a data collection profile can be defined to compare information from any layer to modify the analysis operation.

In other cases, the data collection profile may be defined to use the status and parameters of other devices of the wireless device to filter the data collection. The wireless device has many other services that operate on the wireless device, such as voice calls, messages, games, and cameras. These services interact in both devices (competition for resources such as processors and batteries) and networks (competition for bandwidth). The number of such services is growing rapidly, and systems such as BREW and JAVA allow the wireless device to download the services of its choice after the user owns it. Thus, the increasing number of potential interactions between these services and the difficulty of analyzing all possible scenarios actually confirms that many failures due to the interaction of these services will occur.

For example, multimedia message service (MMS) systems use a model in which messages are downloaded "in the background" to a user's wireless device to provide an "instant access" experience. Applications such as instant connect communications, also known as Push to Talk®, have excessive demands and expectations for data channels. If an MMS download occurs during an instant connect communication session, the user experience will be very strong and the instant connect communication session will not be able to run as expected. The cause of this problem is not visible to the user, and therefore no interaction with customer management will be able to solve why an instant connect communication session sometimes exhibits very poor performance. To address this, a data communication profile can be defined as an initial trigger that starts collecting data whenever multiple applications or services require data access at the same time. The data collected according to this profile allows the service provider to evaluate these and any other predicted competitions between the services.

The data collection profile 600 also specifies various parameters related to the manipulation and management of the data collected by the target wireless device 400. In the illustrated embodiment, the data collection profile 600 specifies a schedule in which data collected by the target device will be updated to the SQP 201. In one embodiment, data collection profile 600 may specify a regular upload schedule, while other data collection profiles 600 may specify a one-time upload or a real-time upload of collected data. In accordance with other aspects of the data collection profile 600, the upload schedule may be set to meet the needs of a particular application, operating system, or target device. The upload schedule may also be driven by an event, which may depend on other factors including end user activity, useful device battery life and coverage status. For example, the upload may be ignored or delayed so that if the battery of the target wireless device 400 is low, it will not significantly affect the user's experience by reducing the limited power usage. Another example of how to protect the user experience is that the SQC 402 can build a pattern of behavior for the end user and adjust the upload schedule called in the upload collection profile to a low usage time empirically derived.

As described above, the upload schedule may be based on the state of the target device. This state may relate to an event or condition of the target wireless device itself. In addition, the state of the target device may also be related to the conditions of the network and the interaction of the target device within the network. For example, the conditions of the network that can determine the upload schedule include signal quality, high or low contention, non-roaming service, high speed data service presence, and the like.

In another example, the RF environment is analyzed and the target wireless device 400 can determine whether uploading the metrics package is efficient for both the device and the network. For example, the amount of power the wireless device consumes for data transmission depends on the power level required to achieve the proper signal to noise ratio in the signal tower. Matrix package uploads that do not occur at the threshold time may be set to proceed if the power setting is below a preset maximum. This ensures that only minimal power is consumed for operation. Further modification of this model may include changing the preset maximum power based on the time at which the upload is delayed.

Other aspects of data collection profile 600 may include the ability to specify a device configuration with which the collection profile may be compatible. As more and more user-definable and downloadable features exist for a wireless device, it will make it impossible for the target device database 204C to have all the information about all aspects of the target wireless device 400. For example, a user can download a custom ringtone and use a plurality of different models for playback. If a suspicious problem appears only for attempts to play MIDI-based ringtones longer than 30 seconds in length, a data collection profile can be defined and a target population of wireless devices can be selected, but this data collection profile can Will only be activated when 400) has that particular form. In this case, the data collection profile can be downloaded to a large group of devices that meet certain known conditions, while it will only be activated on certain devices using MIDI-based ring tones greater than 30 seconds in length. Target wireless devices 400 that are not compatible with may ignore the data collection profile or remove the data collection profile from the memory of the target device. Best of all, this feature prevents the collection of irrelevant or unnecessary data.

In some cases, data collection profile 600 incorporates various other related functions. For example, the one or more data collection profiles 600 include instructions that specify what action the target wireless device 400 should take if conflicting data profiles affect its target wireless device 400. . In another example, some data profiles 600 include instructions that specify a priority of a data collection profile specified there relative to one or more others associated with the target wireless device 400. Multiple data collection profiles may be performed simultaneously, but if there is high urgency with respect to the requested data or if the device is faced with multiple query data collection requests, and the capability of the target wireless device 400 is one If simultaneous collection of data streams specified by the above data collection profile 600 cannot be achieved, then a need may be determined to prioritize the performance of the data collection profile. To resolve this potential conflict, data collection profiles can be assigned a priority level, and once downloaded to the device, the relative priority levels are evaluated by the SQC 402 installed on the device and the profiles are assigned their respective priorities. Can be performed according to. Once the data is collected according to the priority specified in the data collection profile, the metrics package can be created and located in the metrics archive of the SQC 402. However, such functions are merely exemplary, and various other rules regarding the management of data collection profile 600 may also be used.

As mentioned above, the data collection profile 600 may specify, include, or incorporate the conditions under which the data will be collected, as well as other information, guidelines, or rules for the data to be collected. Some data collection profiles 600 may include evaluation conditions that enable the target wireless device 400 to self-select a data collection process. The data collection profile 600 may also include or specify various parameters relating to hardware and / or software included in or associated with one or more target wireless devices 400.

As discussed above, data collection profiles may be defined or stored for later use, or may be generated “on-demand” in response to specific business and technical needs. In addition, in many cases data collection profiles are automatically derived from the particular data observed after analysis of the matrix packages. They may also occur when certain thresholds are reached or trends are identified. Thus, data collection profiles are derived from one or more features of the collected information, such as trend information, predefined data points or thresholds. For example, the data collection profile may be set to collect RF performance information such as call abandonment rather than wireless data. In this example, an excessive number of call abandons within a particular geographic area will serve as a basis for the generation of a new data collection file or a modification of an existing data collection profile. That is, more specifically, the wireless data collection is instructed to enable evaluation and diagnosis of a relatively plurality of call abandonment. Thus, even if wireless data is not the initial goal of the data collection operation, trend or threshold analysis of the collected metrics package may provide a basis for additional, related data collection efforts.

Ⅴ. Example Query Definition Process

Referring now to FIG. 5, details are provided regarding a process 700 for defining a query as may be used for the continued development of a data collection profile. As discussed above, a plurality of industrial and technical issues can be identified and completed data collection packages including data collection profiles and reports can be developed and stored for later use. However, in some cases human intervention may be needed to immediately react to a previously unknown sequence of events, and network administrators may wish to create a “on-demand” data collection profile. The mechanism for creating on-demand data collection profiles is shown in FIG. 5. In step 702 of process 700, a critical question that is to be answered by the data collection action is identified. As shown in FIG. 5, the identification of critical questions in the query process may be based on the results of various other reports that have occurred with respect to previous data collection actions. In this way, network administrators can use the previously developed knowledge to clarify the query. This, in turn, improves the development of relevant data collection profiles. In this regard, the threshold question constitutes nothing other than an explicit language description or question for the particular data desired, or a question to be solved.

If the critical questions are identified, process 700 proceeds to step 704 where the network administrator accesses the query screen by the UI. Process 700 then proceeds to step 706 where the network administrator uses a query screen to process the critical question or questions identified in step 702. As discussed above in connection with FIG. 3, such processing of critical questions allows trees, serial menus and / or other structures and apparatus to enable a network administrator to process a problem or problems to be solved by a data collection operation. It can be implemented by a query construct that uses them.

The network administrator can specify queries about the RF performance or the performance of other systems in which the target device is employed, as well as the specific target device or type of target device, applications installed on the target device, and services connected in connection with the target device. Can be. As shown in step 708 of the process 700, development of the query structure may preferably take into account or incorporate aspects of previously developed queries and / or data and results obtained with respect to such queries. .

Next, process 700 proceeds to step 710. In this step, the network administrator completes the query based on the structure and / or results of one or more previously developed queries, as well as the various categories of information specified. As soon as the query is completed, the query is sent to the reporting subsystem 208. Completed queries can be stored in memory for easy retrieval and modification if desired. In addition, stored queries can also be retrieved and copied later to streamline the development of similar queries. In addition, such queries can be stored in a searchable database so that a network administrator can quickly determine whether queries of interest to him or her have already been developed. This arrangement likewise allows for the efficient development of further queries and data collection profiles. If the data needed to satisfy the query has already been collected, appropriate data and / or reports may be returned to the network administrator. If not, a CTR is generated by the reporting subsystem 208 and communicated to the work subsystem 204.

Ⅵ. Example work, building a data collection profile, and target  Device Selection Processes

Referring now to FIG. 6, details of a process 800 for generating or building one or more data collection profiles based on the CTR, selecting target devices receiving the profile, and downloading the profile to the target device. This is provided. At step 802 of process 800, a CTR is received at work subsystem 204. In some cases, step 804 is entered where the development of a data collection profile is initiated in response to receiving trigger information regarding the trigger condition occurrence or setting the trigger condition. Thus, some or all of the data collection profiles may occur automatically.

In another embodiment, a data collection profile occurs when requested by the network administrator through the submission of the completed query structure and the resulting CTR. For a relatively fast development of data collection profiles, the profile system is set up to retrieve data profiles, such as profiles that match or respond to CTRs. In this way, the development of new data collection profiles can proceed fairly quickly. In addition, in some cases, a network administrator may specifically change a particular data collection profile in a particular manner so that the changed data collection profile can be installed on various target devices. In this case, a decision on the use of a previously developed data collection profile is taken from the profile system and is instead given to the network administrator.

In either case, process 800 then proceeds to step 806 where the CTR is provided to profile builder 206A. At step 808, profile builder 206A includes rules and triggers, generates data collection instructions corresponding to the CTR, and retrieves the existing data collection instruction as applicable. Thus, as data collection commands are generated and retrieved, process 800 can change an existing data collection profile as profile builder 206A can build or apply a data collection profile.

After the data collection profile has been selected, constructed or changed, a determination should be made as to which target device or target devices may be used for the collection of data identified in the data collection profile, if possible. Thus, at step 812 of process 800, profile subsystem 206 notifies work subsystem 204 of the device request in the data collection profile. Based on this information, job subsystem 204 accesses job database 204B and target device database 204C, and identifies, at step 814, the target device or devices whose characteristics match or correspond to the data collection profile. As mentioned above, any number of device characteristics and conditions may be considered in the device selection and selection process. In embodiments where wireless devices are used for data collection, the selection process may include a method for determining which wireless device is compatible with the data collection profile and which wireless device should be disqualified from the data collection activity. To further select or disqualify a wireless device, physical characteristics such as the device's configuration, available memory, location, and effective battery life, account status, usage patterns, reporting service issues, other usage histories, Various other factors and characteristics may be given, such as the interaction or physical manipulation of an application, an installed application, and a usage state such as an end user's profile.

The device information and characteristics used in the device targeting and selection process in step 814 are various resources, including a system controlled by a network operator such as an HLR or VLR, or a target device database when the device first operates the SQC 402. May come from information provided to 204C. Additionally, device information may be extracted from previous data collection activities and stored in the target device database 204C, or device “targeting” to the target device database through the use of a data collection profile in which data collection instructions focus on the data collection target. targeting) data "may be collected specifically for the purpose of storing. An example of this is a data collection profile for recording the identification of base stations that each device downloads to all known devices and meets for seven days. On the last day of seven days, the collected data is uploaded to the metrics collector 202A and communicated to the target device database 204C. The targeted data is then stored for later device targeting purposes. Thus, the device targeting and selection process will access the identification of the base station where the device meets and the data regarding the frequency of the particular base station where the device meets. Thus, a particular problem study can include a device that can provide the best performance data about specific base stations and those base stations that can be targeted to collect data for that study. As a result of collecting the targeting data into the target device database 204C and then later using that data for data selection criteria, the set of target devices can be identified and any metrics shown in the SQC 402. Can be selected on the basis of This includes device behaviors such as real-time device configuration, access patterns and application usage patterns included in user download software and content, device characteristics such as battery type, and assistive devices attached or attached between other wireless and network information. can do.

The targeting process shown at step 814 of FIG. 6 may also automatically select devices with certain problems or problems previously encountered. For example, if a group of devices experiences a failure known to data collection and management system 200, reporting subsystem 208 may cause a data collection profile to occur automatically to further investigate the failure, and Devices that have experienced a fault may be selected to receive a new data profile and participate in that data collection activity.

The device selection process may be iterative in that the initial results of the target device search may result in too many or too few compatible devices. In this case, the selection condition may be modified gradually to bring up the appropriate target device population. In the example of colliding base stations as described above, the initial selection condition may be limited to devices selected to collide with a particular base station in the last week. However, if this results in more devices than are required for data collection, then select the devices that have collided with the base station during the last two days so that another iteration of the selection process can bring up a more appropriate number of devices for the data collection activity. It can be specified. In practice, this iteration will typically be performed automatically by the work subsystem in the process of selecting a device.

Another source of data for device targeting and the selection process of step 814 may be the account information of the device user, typically stored as part of the network operator's billing and provisioning system. This information identifies the end user's external assets and the financial relationship between the end user and the network operator. This information may include corporate companies, payment plans and styles, customer care history and events, and user statistics information. An example of the use of this information could be a query to monitor the performance of data related to a particular high value enterprise customer. By tracking specific performance parameters, such as call abandonment vs. call success for devices of a particular corporation, the network operator can provide a guarantee of a specific performance level for the monthly dues repayment rate. In markets where perceived quality is an important key for network operator service selection, such programs can provide significant competitive advantages.

In addition, if the data collection profile specifies that data relating to wireless calls made in the San Francisco Bay area should be collected, the task subsystem 204 may cause the target device database 204C to determine whether the device is currently or recently San Francisco. Even if they were active in the Bay Area, the account information would not select target devices indicating that the device is in New York. In other cases, note that the working subsystem 204 accesses the profile developed by the profile system to determine the target device to which the data collection profile is to be sent. The result of that determination is then sent to profile builder 206A, and the device compatibility information can be included in the data collection profile.

After the target device has been identified, process 800 proceeds to step 816 where a new or updated profile is sent to the identified target device (s) as can be seen. Profile transfers include "pushing" the data collection profile to the target device, sending a message to the target device that prompts the target device to retrieve the data collection profile, such as SMS, and uploading the metrics package. Similarly, the next time the target device connects to the SQP 201 may occur in a variety of ways, including preparing a data collection profile for download. Such profile transfers to the SQC 402 present on the target device (s) include short message service ("SMS"), hypertext transfer protocol ("HTTP"), secure hypertext transfer protocol ("HTTPS"), wireless applications. Use any of a variety of transport mechanisms and standards, including protocol ("WAP") push, IP-based broadcast (IOTA) protocol, OMA / DM, or other protocols known in the art or may be developed in the future. Can be achieved.

Process 800 then proceeds to step 818 where the data collection profile is stored on the target devices. When received by the target device, the collection profile is processed by the SQC 402. In some cases, the data collection profile may be stored in the received state, integrated with previously received data collection profile (s), or may replace previously received data collection profile (s). Factors affecting how the data collection profile is processed by the SQC 402 include, but are not limited to, the suitability of the device to the data collection requirements defined in the data collection profile, the data collection profile, and any The relative priority of previously received profiles, and any explicit processing rules described in that data collection profile. If processing the new profile with SQC 402 in any way results in a different data collection activity than that specified in the data collection profile (for example, if the device selects the data collection activity on its own), then the SQC 402 A description of how and why the data collection activity differs is communicated back to the SQP 201.

The data collection profile can be sent to the target devices via a wireless or wired connection. Since the data collection profile is relatively small, the transmission of the data collection profile proceeds relatively quickly and imposes a minimum processing fee on the target devices. In addition, the population of target devices can be quickly redefined, and the data collection profile can be downloaded quickly and easily to achieve the data collection purpose. Such repetitive data collection processes are particularly useful for understanding transient error conditions because of the speed at which data collection activities can be improved. Other data collection activities can contribute to longer trend analysis. For example, a threshold may be set for performance degradation that results in the generation and download of a data collection profile to a group of wireless devices when the threshold is reached. In this way, additional data collection can take place that allows further study of the problem. As a result, embodiments of the data collection and management system are very flexible and can rapidly process, reconfigure and redirect data collection efforts in response to newly occurring network conditions or transient network conditions. In all cases, statistical analysis related to data collection can quickly focus solutions or responses to questions placed in relation to the query.

Unlike systems known in the art, the data collection and management system does not rely on the end user of the target device to download the data collection profile or, in other words, to take action to enable the data collection process. Rather, as described above, the update of the target device may proceed without a minimal or no association to some of the end users of the target device. In addition, since each target device is carefully selected for participation in data collection activities, the likelihood that any particular target device is not an appropriate candidate for the collection operation is minimized. Thus, as specified in connection with the data collection and management system 200, data collection is performed quickly and easily by the target devices. In addition, since data collection profiles are automatically generated in response to the occurrence of certain network conditions, the flexibility and speed with which the data collection and management system 200 operates is also enhanced.

Iii. Data Acquisition and Processing

The data collection function of the data collection and management system 200 is generated by software or hardware of the wireless device 400 during operation and is based on one or more metrics that are indicators of device and network performance. In one case, during device fabrication, software instructions are integrated into the operating system or other device software that causes the metrics to occur. As the metrics occur, they are sent to the data structure used to call the SQC 402. Since SQC 402 is present with the metrics that occurred, SQC 402 determines, based on the data collection profile (s) under its control, whether a given metric is interested in the time it occurred, and if so You can store the metrics in a buffer for later inclusion in the metrics package. The process of storing metrics in a buffer is referred to as collecting metrics. However, in some cases, the metrics generated may be collected directly in the metrics package without being buffered first. Otherwise, the matrix collection and buffering process may continue as part of the basic operation of the wireless device 400. Control of such data collection logic is further described below in conjunction with FIG.

Another mechanism for generating metrics on the wireless device 400 is to integrate the matrix generation software into application software installed or downloaded on the wireless device. This is in contrast to, for example, integrating matrix-generating instructions into the operating system software of the wireless device during device manufacturing. During application software development, an application programming interface (API) is used to allow an application developer to call the SQC 402 with the metrics generated by the application. In one embodiment of the present invention, when the SQC 402 is present with the generated metrics, the SQC only interprets the metrics within certain fields of the metrics data structure, and the remaining metrics do not need to be further developed by the present invention, without further manipulation by the present invention. Processing using techniques known in the art, such as variable length structures. The APIs reflect in-memory data structure models from XML descriptors shared between SQP operators and application developers through external means. Common fields of the metrics data structure include unique identifiers (assigned by the SQP operator during the XML conversion process) and size fields, allowing new metrics to be generated by the new application without changing the SQC. The rules in the data collection profile dictate the assignment of metrics to the buffer and the assignment of link triggers to the metrics generated by matching identifiers within common aspects of the metrics data structure. Data collection profiles can be implemented to define data collection rules, triggers, and buffers for metrics requests that occur after the creation and implementation of SQC.

A scalable metrics model can be implemented for both third party applications (such as browser and / or messaging applications) integrated in the wireless communication device and for applications downloaded to a virtual environment such as a BREW or JAVA virtual machine. The virtual environment implements a programming interface for invocation by applications developed with SQC enhanced platforms and functions. Every time an interface is called. The matrix data structure is passed to the SQC as a metric.

Impersonation devices as described above provide another important point for the generation of metrics. Specifically, by installing a virtual machine to generate metrics, regardless of whether or not to use the programming interface as described above, all applications running on the virtual machine are therefore installed to some extent to generate metrics. In this embodiment, the following aspects may be set up for matrix generation. Substantially any aspects of wireless device 400 functionality may generate a matrix. Metric generation includes all error display routines that cause an error to be displayed to the user so that it can be obtained by the data collection and management system for later analysis. Exceptions (software that violates basic theories of target device operation) and task timers (timers that ensure that no single function can bring all processing resources and detect errors in the form of an infinite loop) when an application occurs It can generate metrics indicating the state of. When metrics are acquired and uploaded, the metrics can enable application developers to detect errors that occur in fields in their software and enable significant speed problem diagnosis. User interface routines, including screen updates and user input (button press or other direct input), can generate metrics with user input including timestamps. When metrics are uploaded, user interface metrics can be used to understand how users interact with the application or how the device optimizes the application design and detects errors or confusion points. This technique can also be applied to the entire wireless device 400 as the main operating system is installed instead of a virtual machine.

Referring now to FIG. 7, details are provided regarding a process 900 performed by one or more target devices with respect to one or more data collection profiles. Process 900 begins with the generation of metrics as part of target device operation. In step 902 of process 900, data collection begins as specified in the data collection profile. As noted above, such data collection can be as simple as arrival or date time at a geographic location or combined with the use or nonuse of complex security or characteristic services or applications of conditions such as date time, geographic location, certain errors, etc. May be initiated in response to a start trigger, which may be any combination of the above mentioned conditions. Once the profile is activated, data collection is initiated and continues, and it is possible to start, end and restart sampling data based on data collection rules and triggers defined in the data collection profile. As mentioned above, during the data collection process, the metrics may be stored in a matrix buffer or temporary memory and may be used or not used forever. The matrix buffer is a circular buffer that stores N data samples that are useful for use in generating a matrix package if necessary. The use of buffered data allows, for example, the observation of metrics that occurred before certain error conditions occurred. This historical observation, along with other metrics surrounding the error condition, is very important for analyzing the problem. Thus, in response to the trigger, buffered data may be obtained in the matrix package along with any specific data reached after the trigger is activated. In other cases, data may be collected and written directly to the metrics package without first being buffered.

In any case, the target device starts collecting data as specified in the data collection profile. For example, each target device may first examine and verify that the data collection profile matches various characteristics of the target device, such as hardware and software included on the target device, and various services that the target device is authorized for use. Examine the received data collection profile. In this manner, target devices that are unable to collect data specified in the data collection file, or in other words not appropriate for making a response decision to a query, may be dropped from the data collection process themselves. Optionally, depending on the compatibility-related instructions included in the data collection profile, rather than dropping out of data collection, the device can collect only a subset of the requested data, which can be combined with the metric collector (based on the subset collection). 202A). Thus, the network administrator can be assured that the data ultimately collected and transmitted to the matrix collector 202 only includes data that matches the initial imposed query and responds to it.

In any case, once the data has been collected, process 900 proceeds to step 903 where the collected data is sent by the SQC 402 to the metrics package and stored in the metrics archive of the SQC 402. Since device resources are inherently limited, the SQC 402 can employ methods for managing the priority of the metrics packages stored in the device. For example, each time a metrics package is added to the metrics archive of SQC 402, SQC 402 calculates the total nonvolatile memories used. If the total nonvolatile memory required to store the new metrics package exceeds the amount of memory allocated to the metrics archive, the SQC 402 resets the data characteristics in each stored metrics package. In one case, under certain conditions, one or more metrics packages can be uploaded to the metrics collector 202A to create a room for a new metrics package on the device. Under certain conditions, the SQC 402 may employ an algorithm to re-observe the stored metrics packages and select the metrics packages to be deleted. The algorithm may take into account conditions including data storage allocation specified in the data collection profile, characteristics and priorities of the data, schedules of the data, the size of the new package to be stored, and the size of the package to be deleted. The selected metrics packages are then deleted from the device's metrics archive until there is enough memory to store the new metrics package.

At step 904 of process 900, the metrics package is sent to the metrics collector 202A at a time specific to the applicable data collection profile or profiles. As discussed above, the transmission of the metrics package may be a one-time event, performed periodically, or may occur irregularly. It may also be sent as a result of resource management by the QC, or as a result of a "on-demand" command sent to the device by the network administrator or device user by the system.

In one embodiment, the data collection profile may specify that uploading of the metrics package occurs only upon certain event or condition occurrence. When the transmission condition is satisfied, the collected data is sent to the matrix collector 202A. In some cases, receipt of a metrics package can generate a notification, such as an email alert or system alarm, which can trigger a reporting activity and / or further monitoring or data collection activity. In other cases, if the condition is met, the current state of the end user device is further evaluated before the upload activity begins to prevent exceeding the capacity of the device or network or to interfere with the general activity of the end user. Can be. If the condition is met, but the end user is involved in activity on the device to be interrupted by the metrics package transmission, the transmission may be postponed next and uploaded according to the rules of the auxiliary set. The metrics package can also be completely discarded if, for example, the data is time sensitive and the device is unavailable for network connectivity in an extended period, or if SQC resource management enforces a hard delete to make room for the priority metrics packages. Can be.

Finally, as shown in FIG. 7, process 900 proceeds to step 906 to determine if a new or changed data collection profile is available for download by one or more target wireless devices 400. Although this step in the process is shown as part of the continuous operation in FIG. 7, the determination of whether a new or changed profile is available for download is independent of any other step of the data collection process, and such a new or changed data collection profile. Can be created and prepared for download to the selected device at any time in response to changing network conditions, accidental occurrence of specific network conditions, occurrence of a new query, or various other factors. If a new or changed data collection profile is available, the available new or changed data collection profile is downloaded to the target wireless devices at step 908. Otherwise, according to the rules specific to the data collection profile at step 910, it is determined whether the existing data collection profile remains valid. If it remains valid, the data collection activity resumes at step 902. In many cases, an existing data collection profile can be present on the device at all times with or without change, and can perform numerous moments of data collection and upload. However, if it is determined in step 910 that the existing data collection profile has been discarded, the process proceeds to step 912 where the existing data collection profile is invalidated and in some cases automatically deleted from the device.

Referring to FIG. 8, details are provided regarding aspects of process 1000 for processing and managing metrics packages. In step 1002 of process 1000, the metrics package data is accessed from reporting subsystem 20b. Metric packages can be indexed so that all appropriate data can be accessed from the reporting subsystem and can be associated with data collection profiles and other metrics packages.

Process 1000 then proceeds to step 1004 where the metrics package data is processed. Such processing can be in fact any manipulation or analysis of the metrics package data, including statistical analysis. In some cases, the analysis results of the metrics package data are used to develop or process current or future queries, data collection profiles, and / or to improve the targeting accuracy of the work subsystem 204. After collection and before or next to the report generation operation described below, data may be discussed using automated computing and additional sources of information. In addition, the plug-in server architecture may allow other applications to supply the metrics package.

Next, process 1000 proceeds to step 1006 where various reports are generated based on the processing of the metrics package. In a preferred embodiment of the present invention, such reports will be predefined in relation to the data collection analysis package described herein above and can be grouped with the analyzed data. Optionally, in this case a customer report can be generated on demand by the network administrator. Finally, process 1000 stores data and reporting information in reporting database 208C for further use and reference, exports to third party reporting systems, and is useful for queries, discussions, and / or data profile changes and definitions. Proceed to step 1008.

Iii. Of trigger  Definition and use

As mentioned above, the data collection process performed in connection with the network and associated target devices may be based on specific events, network conditions, trends, data points and Involves the use of critical points. Embodiments of the invention also provide for the definition and use of various forms of triggers in connection with a data collection profile.

In this embodiment, the trigger is one or more conditions defined during the integration process of the wireless device 400. Triggers allow the SQC 402 to be invoked for further metric processing purposes. Each trigger defines a set of specific states that may be associated with a particular identifier and visible to the wireless device 400 software. When the SQC function is invoked, the trigger identifier is compared with the triggers included in the downloaded profiles to determine the appropriate course of action. As such, trigger inclusion within a particular data profile effectively defines the conditions under which that data profile can take a particular action.

In this embodiment, the triggers may be included in data collection instructions of the data collection profile, such inclusion of which the SQC starts, stops and ends the data collection activity as appropriate when the relevant trigger condition is called by the wireless device 400. Let's do it. An alternative embodiment of the triggers and data collection function is discussed later in Section VII. The trigger call that matches the start trigger causes the data collection activity to start. Matching the end trigger causes the data collection activity to end, and then the metrics package is ready for upload. The abort trigger causes the data collection activity to stop and the metrics package is not ready. In the example used above, the start of instant connect communication causes the SQC to be called with a "start instant connect communication" trigger event that matches the triggers in the downloaded profile and initiates a data collection activity on the start device. Receipt by the start device of the 408 error will cause other triggers to be activated, and the SQC will match the event with the end trigger in the profile to stop data collection and to prepare and upload the data package. As can be seen, the inclusion of a trigger in a profile effectively selects the conditions under which a particular action associated with that profile should be performed. Triggers are not necessarily in the profile, but rather associate specific profile actions (start, stop, end) with specific events on the device.

In another embodiment, the trigger event may be the start of a voice call. In this case, the start of the voice call is a start trigger call that causes the SQC to execute the data collection rules contained in the data collection profile existing on the device associated with the voice call. In this example, the data collection profile may combine the termination of the data collection activity and the act of terminating the voice call and also cause the preparation of the metrics package for upload to the metrics collector. This data collection activity allows the recording of metrics activity specific to the call and the generation of the metrics package associated with the call taking place on the device.

The use of triggers can provide finely tuned data collection activities. By linking profile actions to events defined in the device, virtually all devices can be efficiently measured, including slowing down measurements and tracking in time. As a result, important trend information or other instructions expressed in the collected data can be quickly identified and studied, thereby improving the quality and usefulness of the collected data. All the various triggers can be defined and used. Thus, the scope of the present invention should not be construed as limited to the particular form, number, combination, implementation or use of triggers. Details regarding some triggers are provided below.

One basic form of such triggers includes the definition or specification of a particular event that causes the SQC to be invoked. As used herein, the term “event” is broadly construed as the occurrence or nonoccurrence of a specified event that defines a particular identification of the trigger. If it is determined that an event defined by the SQC occurs (e.g., by being called by wireless device software with specific trigger identification) or it is determined that the defined event did not occur within the set time frame, The above data collection activities may be performed by the SQC. The SQC also defines specific metrics that cause the start of identification of a specified trigger in a particular state. In virtually all states, every metric or sequence of metrics can be defined as causing the onset of a particular trigger identity. For example, a threshold trigger can be generated by a monitored parameter that falls above or below a certain threshold for a defined time that can range from zero to any large number. One particular example is a threshold trigger that occurs when the frame error rate metric exceeds a predetermined value.

Trigger identification may be included in the data collection profile and cause the occurrence of related behavior. For example, to define a data collection profile that imports a metrics package with information about abandoned voice calls but does not collect information about other types of calls (success or blocking), the following combination of triggers may be used: Can be.

Start Trigger: Start voice call

End Trigger: End a blocked voice call

Abort Tree: End Voice Call

In this example, because the data collected with respect to this trigger is a collection activity target, the end trigger stops data acquisition and allows the metrics package to be ready. Since the data associated with this activity is not focused on the purpose of the collection, it indicates that data collection should be terminated and that no metrics package needs to be prepared.

A related form of trigger is a “domino” trigger that is activated when a command is sent from the SQP 201 present on the network node to the SQC. In the previous example where the error 408 begins to be monitored, the terminating device of this example does not recognize the communication session attempted by the initiating device and therefore data collection activity is not triggered at the terminating device in this case. “Domino” triggers are used to ensure that critical data relating to the condition surrounding the terminating device is obtained during a communication attempt. As described above with respect to the 408 TIMEOUT message, a domino trigger occurs when the SQP 201 sends a command that causes the start trigger to be activated to the SQC 402 on the end device in the form of an SMS or other message, and is “on demand”. The data collection activity begins and the message package is ready for upload. As part of the “on-demand” data collection activity, the metrics can be selected from the metrics present in the terminating device's buffer, and other metrics can be collected to complete the metrics package. Specifically, the data collection profile previously activated on the terminating device in this example is a collection of data that is always useful for "on-demand" data collection activities that allow the domino triggers to gather data that has already been collected on the terminating device before the domino trigger arrives. Can bring

Referring to FIG. 9, details of an example of the process indicated at 1100 for data collection are shown. Process 1100 begins with step 1102 in which the SQC 402 on the wireless device or devices receives a data collection profile specifying triggers. The trigger specified in the data collection profile may be set such that the SQC 402 performs data collection activities upon the occurrence of the specified trigger. In the example shown, the SQC 402 on the wireless device is associated with a monitoring process to identify the activity of the trigger specified in the data collection profile, at step 1104. Data collection begins when the start trigger is activated. Once data collection begins, some or all of the monitored data may be collected or ignored according to instructions contained within the data collection profile.

In the process shown, after the initial trigger initiates data collection, the data collection proceeds according to the instructions contained in the data collection profile up to the EO where the abort or termination trigger will be detected at decision point 1106. For example, data collection instructions may specify sampling data at periodic intervals for a particular period of time. Data collection stops when the abort or shutdown trigger is activated.

Triggers may be defined using any aspect of the state of the wireless device during implementation. By this mechanism, complex dependencies can be implemented, such as collecting data only when certain services are in a particular state. In the previous embodiment, data collection for IP data calls made by an instant connect application can be implemented as a specific trigger definition that can only be activated under a loose and limited set of conditions.

Finally, referring to FIG. 10, details are provided regarding process 1300 related to the definition and use of domino-type triggers. In the case shown, process 1300 is performed in terms of a communication network. This arrangement not only focuses on the portion of the action associated with a particular wireless device, but provides the ability of network stakeholders to observe and evaluate the overall action between wireless devices in a communication network.

Process 1300 begins at step 1302 where one or more metrics packages are received from one or more target wireless devices or other network nodes. Such data generally relates to one or more parameters of a communication network and / or associated wireless devices. Next, a process 1300 is made in which a determination is made as to whether the data received in the metrics package (s) includes another "unreported" network node, such as an end device that does not recognize that the start device has attempted to communicate. Proceed to decision point 1304. If no other network node is included, process 1300 proceeds to step 1306 where further analysis can be performed.

On the other hand, if the received data indicates that a " unreported " network node is included and can provide important data for analyzing the action, process 1200 proceeds to step 1308 instead. In this step, the SQP 201 causes the command to be sent to an included network node "unreported." In this case, the command from SQP 201 is interpreted as a trigger that allows the included unreported network node to collect relevant data and send that data to the matrix package.

Thus, if wireless device "A" sends a page to wireless device "B" and does not receive a response, wireless device "A" receives an error and this error triggers a trigger that causes the metrics package to be uploaded to the matrix collector. Activate it. However, because the wireless device "B" may not recognize that the page has been sent, the wireless device "B" may not be aware that any error or problem has occurred and may not be triggered to upload the metrics package associated with the error. . Thus, in order to achieve a more complete understanding and evaluation of the problem, it is necessary to consider the overall action or attempted action between the devices "A" and "B".

In step 1308, the SQP 201 causes the command to be sent to the wireless device “B”. In step 1310 this command is interpreted as a start trigger that causes data collection activities such as the selection of metrics from the buffer of device “B”, the collection of any additional metrics required, and the preparation and uploading of the metrics package. Next, the metrics package from wireless device “B” may be received by the metrics collector at step 1312 and further analysis may be performed. At this stage the metrics packages uploaded by such other network devices with respect to the data collection profile are received and analyzed. Process 1300 may be performed once or repeatedly as part of an iterative analysis and resolution process.

Iii. Documented ( scripted Device profiles

In another embodiment of the present invention, documented device profiles rather than data collection profiles may be used for the processing of metrics generated by the target device. In this embodiment, instead of a limited set of fixed functions, SQC is "documented" in a full-featured programming language that allows any algorithm for metrics, triggers, and measurement management to be defined by a documented device profile. Additional flexibility is provided by the data collection and management system of the present invention because the details of metrics collection, processing, and management can be defined more dynamically by documented device profile downloads than those coded in the SQC. Significantly increases the number of scenarios that can be done.

In contrast to the previous embodiment, in this embodiment, when the matrix occurs as a result of an activity or action performed on the target device as previously described, the metrics are collected, the metrics are not collected, or the occurrence of specific trigger identification is avoided. Rather than being limited to importing, matrix processing is considerably more flexible. More specifically, embodiments of the present invention allow complex processing in matrix collection to be defined entirely by the downloaded device profile. Matrix parameters and triggered functions that replace the data collection instructions (rules and triggers) described in the previous embodiment are stored in a documented device profile and govern the processing of the matrix. As mentioned above, the Quality of Service Client (SQC) is responsible for receiving and managing profiles.

Referring to FIG. 11, when a matrix occurs on wireless device 400, SQC 403 is invoked, which is specified in device document 404 as the matrix processing element 404A determines the proper management of the matrix. Reference metrics parameters 404B. In many cases, no information may be useful within the metric parameters 404A regarding the metric, such that the metric is ignored, device operation continues, and the metric is part of the normal operation of the wireless device 400. It keeps happening.

However, if the metrics parameters 404A specify management of the matrix, further processing is performed. The additional process may have one of two forms. In a first form, called filtering 404C, the matrix is compared to the parameters specified within matrix parameters 404B. The parameters specify both the comparison form and the values required for the comparison. As a result of this comparison, a binary decision is made whether the matrix should be stored in one or more buffers 406 or ignored. Specific comparisons that are useful are defined as part of the SQC implementation, and substantially any order of comparison can be performed. The comparison specified within the metrics parameters 404B may include a simple binary decision such as "always collect this metric" or "gather if the metric satisfies a binary or numerical comparison." Other comparisons can include comparison of metrics with other metrics or other aspects of the wireless device 400 state. Even more complex comparisons can be specified within the matrix parameters, including multiple conditions and “preprocessing” to determine whether a particular result has been achieved. In general, the comparisons specified within the metrics parameters 404B may be substantially any software function, but may be limited to a relatively small number of commonly used behaviors defined at the time the SQC is implemented.

If the conditions specified in the documented device profile 404 are met, the metrics will be collected in one or more buffers 406. As in the previous embodiment, a common function is fixed size buffers for a certain number of metrics, a simple rotational buffer model in which the new matrix overwrites the oldest stored metric if the buffer is full. Another method is to call the triggered function 404D (discussed in more detail below) if the buffer is full and the metrics collected under a particular condition are prioritized buffers that overwrite the metrics collected under another condition. It may include. Indeed, any buffer management model may be implemented in connection with the present invention and may be used as part of the matrix parameters 404B. Indeed, a limited set of fixed functions may be implemented, and additional algorithms may be implemented with the triggered function 404D. In some cases, it should be noted that the metrics may be included in the metrics package without first being buffered.

The second form of matrix processing also uses the comparison function described instead of collecting metrics on positive results for what is defined for filtering, where the previously downloaded script, referred to as the triggered function 404D, is an additional matrix. Called for processing.

The triggered function 404D is a significant departure from previous embodiments of trigger and data collection profile functions. The triggered function 404D dictates the behavior of a fully characterized computer language, i.e., the wireless device 400 processor, such that any software can be implemented on the wireless device 400 in connection with data collection, management and analysis. It is software that makes it possible. As a result of the call by matrix comparison, the triggered function 404D represents the control of the matrix processing. As previously described, the triggered function can perform virtually any software function on the wireless device 400, call metrics, all buffers in which the metrics can be stored, additional state information on the device, metrics packages. And upload functions. In the previous embodiment using data collection profiles and triggers, instead of a limited set of actions that can be performed in response to a trigger call, the traggers each define the functions performed in response to their activation. In this embodiment, the use of the documented device profile and the triggered function causes the result of any of the metrics comparisons defined above to act as a trigger and to perform sequential actions after the trigger is activated (“triggered function”) Allows full flexibility More specifically, the triggered function 404D may cause a new metric to occur and change the metric parameters 404D to change the conditions and processing instructions contained in the documented device profile 404. . The triggered function 404D can also copy, change, and process buffer 406 content and send the resulting data to the metrics packages 408 for later upload. The triggered function can perform the upload and allow new documented device profiles or other software codes to be downloaded to the wireless device 400 and executed there.

The triggered function 404D is included in the device profile 404 documented in this embodiment. A plurality of models known in the art have been found to be effective as a management model for the triggered function 404D to enable software downloadable models on the wireless device 400. These models include virtual machines (for example, implemented in Java) and real machine languages (for example, implemented by the BREW environment) that operate in a defined environment. Also today a large number of commercial packages can be used to perform firmware upgrades "over the air", in which any software is downloaded, added to existing software on the wireless device 400 and called with sequentially triggered functions.

As a complete computer language, the triggered function 404D can implement virtually any software function possible on the device. In this embodiment, the triggered function 404D is implemented in a native device language and thus has access to all states and functions of the wireless device 400. However, certain implementations may have a more limited access to device state and functionality, and may still play substantially the same role.

In addition to the triggered function called by the matrix comparison, a specific set of events on the wireless device 400 may be defined as special triggers 412 and invoke a similarly triggered function 404D. First of all, when the documented device profile 404D is first downloaded to the wireless device 400, the special trigger 412 is triggered in order to perform a profile start in some cases in order to set the trigger and metrics parameters. 404D) can be activated. When the wireless device 400 is turned on or off, reset, or when a processor exception occurs, the special trigger 412 is activated. These special triggers may be implemented such that the triggered function is invoked by an action, such as an application call, user actions, or other activities that do not cause a natural occurrence of metrics. Special triggers 412 may be invoked by virtually any other event on the device that may prove to be of interest in further analysis.

Because of the general capability characteristics of the triggered function 404D, many other aspects of the data collection function (stop triggers or stop triggers) may be other triggered functions than special functions. As mentioned above, the documented embodiment of the SQC discussed in this section indicates that when one of these events occurs, full flexibility as to what happens can be effectively performed by the data collection and management system of the present invention. This significantly increases the number of measures of interest present.

Iii. Mobile communication device and computing environment

As described herein, embodiments of the present invention may be implemented in connection with dedicated and general purpose mobile communication devices such as wireless and wireline telephones, other wireless communication devices, or dedicated or general purpose computers designed to have similar mobile communication capabilities. have. Embodiments within the scope of the present invention also include computer readable media carrying or having computer executable instructions and electronic content structures stored thereon, such terms being used for any such media or instructions used in mobile communications devices. It is defined to extend to. For example, such computer readable media may include RAM, ROM, flash memory, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage or other magnetic storage devices, or computer executable instructions or electronic content. It may be used to carry or store the desired program code in the form of a structure and may include any other medium that can be accessed by a general purpose or dedicated computer or other computing device.

When information is transmitted to a computer or computing device through a network or other communication connection (wired, wireless, or a combination of wired and wireless), the computer or computing device views the connections as a computer readable medium. Thus, any such connection may preferably be referred to as a computer readable medium. Combinations of the above should also be included within the scope of computer-readable media. Computer-executable instructions include, for example, instructions or content that cause a general purpose computer, dedicated computer, dedicated processing device, or computing device to perform a particular function or group of functions.

Although not required, aspects of the invention are described herein in the general context of computer-executable instructions, such as program modules, being executed by computers in a network environment. Generally, program modules may include routines, programs, objects, syntax elements, and content structures that perform particular tasks or implement particular summary content formats. Computer-executable instructions, related content structures and program modules, are shown as examples of program code for performing aspects of the methods disclosed herein.

The described embodiments are to be considered in all respects only as illustrative and not restrictive. Accordingly, the scope of the invention is indicated by the appended claims rather than the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Included in this specification

Claims (59)

Obtaining a data collection profile defining data to be collected and an environment in which the data is to be collected; Identifying a target subset of the plurality of devices in the communication network; Distributing the data collection profile to the target subset; And And a plurality of devices comprising receiving data generated in response to the data collection profile from at least a portion of the target subset. The method of claim 1, Identifying the target subset includes connecting a database to a target device to select devices having properties corresponding to the data collection profile. The method of claim 2, Characteristics corresponding to the data collection profile relate to the usage history of the individual devices. The method of claim 2, Wherein the characteristics are determined using data previously collected from devices in the communication network using a data collection profile. The method of claim 2, Properties corresponding to the data collection profile, Environmental factors that the device encounters; And At least one of the specific software settings of the device. The method of claim 5, The characteristics correspond to the specific software configuration, The specific software setting, Downloaded software; Data entered by a user of the device; User preferences; Downloaded ringtones; or At least one of other downloaded data. The method of claim 2, Characteristics corresponding to the data collection profile relate to physical characteristics of the devices. The method of claim 2, Properties corresponding to the data collection profile, Area code; Local postal code; Billing address; And And at least one of other data identifying the location of the device. The method of claim 2, Properties corresponding to the data collection profile, Services subscribed to by the user; And At least one of the performance of the device. The method of claim 2, Properties corresponding to the data collection profile, The ability of the device to collect metrics specific to the profile; The ability of the device to respond to triggers specified in the profile; RAM storage on the device available for matrix collection; Permanent storage on the device available for storage of a matrix package; Processing power of the device; And At least one of the general purpose capabilities of the device for performing the particular data collection profile. The method of claim 1, And wherein said data is generated in response to detecting a triggering condition by individual devices in said target subset. The method of claim 1, Said data relating to a service of a communication network provided to users via said devices. The method of claim 1, And the data relates to characteristics of a software stack of a device in communication with various network layers associated with the communication network. The method of claim 1, Wherein said data collection profile defines a filter for said data by identifying data to be collected. The method of claim 1, Using the received data to enhance the data collection profile. The method of claim 1, The data is, An application layer of the communication network; A transport layer of the communication network; And And communication by a software stack of a target device having at least one of the physical layers of the communication network. The method of claim 1, The data is, A history of use of the target device; Interaction with the device or physical manipulation of the device by the end user; Hardware associated with the target device; And And at least one of the software associated with the target device. The method of claim 1, Identifying a subset of a plurality of target devices in the communication network so that the number of target devices is within a selected range. The method of claim 1, Distributing the data collection profile to the target subset includes: Pushing the data collection profile into devices in the target subset; Sending a message to devices in the target subset, instructing the device to retrieve the data collection profile from a server in the communication network; And One of preparing a data collection profile to be obtained by the device when the devices next contact the servers, without sending a message to the devices in the target subset. Way. The method of claim 1, Receiving data generated by the target subset in response to the data collection profile from at least a portion of the target subset includes: receiving data in at least one metrics package and transmitting the metrics package on the communication network. Receiving an identifier associated with the interaction. The method of claim 20, Receiving a metrics package from a plurality of nodes in the communication network; And Correlating a plurality of metrics packages received from different nodes of the communication network based on the identifiers of the metrics packages, Wherein the plurality of matrix packages allow a sequence of events related to the interaction of the communication network to be analyzed. The method of claim 1, Identifying the target subset includes selecting devices with characteristics corresponding to device characteristics specified in or implied within the data collection profile. The method of claim 1, Receiving data obtained from a collection buffer in response to a trigger defined by the data collection profile from at least a portion of the target subset; And said data is stored in said acquisition buffer prior to said trigger. The method of claim 1, Identifying the target subset of devices includes considering any existing collection task to which the device may be associated. The method of claim 1, The target devices, Minimizing the collision of the communication network; Compensating for attempts to perform a previously failed function; Responding to a battery condition of the device; Minimizing a negative impact on the user experience of the device user; or And local adjustment of operation so that the step of compensating for the recognized load of the communication network can be performed. The method of claim 25, The regional adjustment may be performed to compensate for an attempt to perform the function that previously failed, The function includes reporting of metrics packages, And said regional adjustment comprises reporting the first available metrics package after failing previous reporting. The method of claim 1, Data received from at least a portion of the target subset is Locally written as a metrics package to a metrics package archive on a device within the target subset; to the next A data collection method characterized by being reported to a network server. The method of claim 27, The data recorded in the archive is reported in response to an event defined in the data collection profile. The method of claim 28, The event is a device-selected reporting time within a time range defined in the data collection profile. The method of claim 29, The selection of the reporting time by the device is based on a special number associated with the device, the special number being An electronic serial number associated with the device; A mobile device identifier associated with the device; And And one of the telephone numbers associated with the device. The method of claim 28, And the event is a metric corresponding to a characteristic set of conditions. The method of claim 28, The report above Battery status of the reporting device; Network connectivity associated with the reporting device; And Delayed by a condition that can be measured from the reporting device comprising at least one of the measurable conditions defined in the data collection profile. The method of claim 27, And the metrics package archive has a maximum size specified in the data collection profile. The method of claim 33, wherein Data stored in the archive is reported in response to an event defined in the data collection profile, The event, A metrics package archive of a reporting device that reaches a specified percentage of the specified maximum size; And And at least one of the metric package archives up to a particular size. The method of claim 33, wherein And the reporting device comprises logic defining how the reporting device should respond to a full metrics package archive using a priority design. Receiving a data collection profile defining data to be collected at a third-party device and an environment in which the data is to be collected; Determining whether a triggering condition defined by the data collection profile occurs; In response to the triggering condition, collecting data at the target device as specified by the data profile; And Sending the data to a remote server, The target device is selected among a plurality of devices to receive the data profile based on a determination of whether the target device has at least one characteristic corresponding to the data collection profile. The method of data collection associated with. The method of claim 36, The triggering condition comprises one or more events in the communication network. The method of claim 36, And the data relates to a characteristic of a software stack of the target device in communication with various communication layers of the communication network. The method of claim 36, The target device receives a multiple data collection profile, And the method further comprises adjusting the performance of the multiple data collection profile based on priorities associated with the multiple data collection profiles. The method of claim 36, Collecting data in the target device is performed in response to the configuration of the target device compatible with the collection of data. The method of claim 40, A characteristic indicating that the target device is compatible with the collection of data, The ability of the device to collect metrics specific to the profile; The ability of the device to respond to the triggers specified in the profile; RAN storage on the device usable for collecting metrics; Permanent storage on the device usable for storage of matrix packages; Processing power of the device; And At least one of the general capability of the device to perform the specified data collection profile. The method of claim 40, Later when the setting of the target device is not compatible with the collection of data, The triggering condition occurs; The data is not collected or only a subset of the data is collected. The method of claim 36, Removing the data collection profile from the target device upon expiration of the data collection profile. The method of claim 36, Removing the data collection command in the data collection profile upon expiration of a particular data collection command. The method of claim 36, The specified collection profile or data collection instructions in the collection profile are effective or disabled for one or more effective times. The method of claim 36, The specified collection profile or data collection commands in the collection profile are effective or disabled after a predetermined period of time. The method of claim 36, The triggering condition comprises a specific time. The method of claim 36, The target device selects between the plurality of devices to receive the data collection profile by accessing a target device database to determine if the target device has at least one characteristic corresponding to the data collection profile. Data collection method characterized in that. The method of claim 49, Collecting the data at the target device comprises placing the data in matrix packages stored at the target device. The method of claim 49, Storing a plurality of matrix packages on the target device; And In preparation for saving another metrics package, Evaluating memory resources of the target device to determine whether there is enough memory available to store the other metrics package; And If not enough memory is available, performing an action on a previously stored matrix package to make sufficient memory available. Obtaining a data collection profile defining data to be collected and conditions under which the data is to be collected; Identifying a targeted subset of the plurality of devices in the communication network by selecting the devices; Distributing the data collection profile into targeted subsets; And Receiving data generated in response to the data collection profile from at least a portion of the targeted subset, Wherein said data is related to a service of a communication network and is collectable at a device other than other components of said communication network. The method of claim 51, The data is related to at least two services in the communication network and an interaction between the at least two services. The method of claim 52, wherein Selecting a device having characteristics corresponding to the data collection profile comprises determining if the services are available in the targeted subset. The method of claim 53, Selecting the device further comprises determining usage or format of use of the services by end users. Receiving at a server providing a communication service a data collection profile defining data to be collected and conditions under which the data is to be collected; Determining whether a triggering condition defined by the data collection profile occurs; In response to the triggering condition, collecting data at the server as specified by the data collection profile; And Transmitting the data to another server in the communication network; The server is selected to receive the data collection profile based on a determination of whether the server has a characteristic corresponding to the data collection profile, the server providing a communication service and a plurality of devices. How to collect data related to your network. The method of claim 55, And the server is an instant connection application server that provides a network based instant connection service for devices in a communication network. The method of claim 56, wherein In the event of an error in the network-based instant connection services, the data collection profile causes the server to send the data to a matrix collector at another server in the communication network. The method of claim 57, In response to the occurrence of the error, sending a message to the terminating device in the communication network causing the terminating device to collect and send data to another server in the communication network. In response to an indication of the query, determining whether the data required to answer the query already exists; If data already exists to answer the query, no collection activity is performed on the wireless device, If there is no data to answer the query, the data, Generating a collection job request for use in generating a data collection profile distributed to the target wireless device; Maintaining the collection job request state for the entire lifetime; And Receiving data generated from the target wireless device in response to the data collection profile and answering the query, Processing the data in response to the query; answering a query relating to a communication network comprising a plurality of wireless devices.

Images