JP4663726B2 - Data collection and management based on rules in wireless communication networks - Google Patents

Description

  The present invention relates generally to wireless communication networks and related systems and devices. More particularly, exemplary embodiments of the invention relate to systems and methods for using distributed network components to implement monitoring and data collection of selected network parameters.

  As technology has advanced and the number of wireless device users has increased enormously, the scale and complexity of wireless communication networks has increased significantly. As an inevitable consequence of such increases in scale and complexity, operational and performance problems associated with communication networks have increased relatively. Reliability issues such as dropped calls, lack of coverage, and poor voice quality prevent end users from accepting wireless technology. These and other quality issues have prevented many end users from trusting wireless voice services and wireless data services as their primary means of communication. As new services are introduced that use even more complex technologies, adopt different usage patterns, and place additional demands on networks that already have many problems, the network performance will be further degraded. Service quality has a direct impact on the complexity and cost issues that drive away customers and reduce profitability. Therefore, improving the quality of service is a top priority for service providers.

  Network monitoring solutions are well known and widely used by service providers, but currently available solutions only monitor and diagnose a subset of the overall telecommunications system, thus efficiently addressing quality issues. It does not consider the overall view of network and device performance needed to identify and resolve. Typical approaches to network monitoring include “self-monitoring” where network components report on their own status and performance and report any errors that occur during their operation. The resulting behavioral metrics from a single element can indicate a wider system-wide problem, but problem solving can involve guesswork and expanded troubleshooting rather than providing an answer. Along with that, waste valuable resources. Another common approach involves installing probes at various points in the network to determine whether the network components are functioning according to the specification. While these monitoring systems, sometimes referred to as “sniffers”, “log monitors”, or “event monitors”, are effective in identifying performance problems with specific network components, they are You can't catch problems that come from the interfaces inside. That is, these solutions are problematic when a single component is performing according to the specification, but those components interact with each other. This collection of extremely difficult and subtle problems results in increased costs for network operators when services cannot be delivered to end users.

  Another known monitoring technique is a pre-programmed service monitor where certain components execute service transactions to mimic "real world" trading activity, end-to-end performance is monitored, and results are reported. Need. These solutions catch systematic failures, but cannot detect intermittent or distributed problems, subtle malfunctions, or problems specific to the device or end user. Furthermore, they can only be verified for the predicted use cases and cannot adapt to new usage and interactions between services.

  In particular, the solutions described above lack the ability to monitor network conditions and intelligently and dynamically define and generate data collection models in response to changing network conditions. Even if a probe is used in a communication network, the probe only shows the appearance of the problem, and actually troubleshooting the problem requires that personnel be dispatched to a physical location on the network, Often a significant amount of time and expense is added to the solution. Furthermore, these troubleshooting techniques do not provide a multifaceted view of different network layers, such as the physical layer, application layer, etc., and do not correlate performance issues across these layers. As a result, a number of quality issues affecting end users remain undetected or misunderstood. As a result, performance data provided by currently available network monitoring solutions lacks the details and timeliness required to quickly identify, prioritize, and resolve network problems. become.

  In addition, service providers report performance issues to customer service representatives because currently available network monitoring solutions cannot properly monitor and report on specific end-user experiences with network usage. You must rely on the end user. However, once reported, it reproduces end-user problems due to the difficulty of detailing what they have experienced, the timing of their occurrence and the lack of fundamental data to confirm the problem. There are often cases where you can't. In addition, service providers cannot view network performance holistically, so they may not understand the true scope of reported problems and may not be able to properly prioritize problems for resolution. is there. Thus, the problem is not resolved satisfactorily by the reporting end user, reducing the willingness of the end user to report the problem eagerly in the future. In addition, the opportunity to prevent problems from affecting others and improve the overall quality of the network is lost.

  Therefore, a situation frequently arises where the service provider is not even aware of the cause or origin of the end user's dissatisfaction and the end user is alienated from the company providing the communication service. In addition, since many end-user issues remain unreported, it is almost impossible to obtain reliable information about a particular problem scope. Thus, resources are often spent solving problems that only affect a small number of end users, for problems that affect a much larger number of end users. As a result, a service provider with a wrong understanding of the scope of the problem may allocate an inappropriate amount of resources towards the solution, or it cannot recognize the value of prioritizing the solution.

  Yet another known monitoring solution is aimed at collecting and analyzing information about the performance of the wireless crisis. Typically, monitoring software is installed on the wireless device at the time of manufacture or by downloading the software to the device. The software then runs continuously in the background to monitor device and application performance. When an error associated with a particular event or device occurs, the software collects the data associated with the error or event and uploads it to a data repository for analysis in real time or after some time. However, as will be described later, such an approach is not flexible and is far from a suitable or effective solution.

  The network monitoring solution just described above monitors the performance of specific components in the network, and the wireless device monitoring software provides another set of data points regarding the performance of the wireless device. However, while device performance information is useful, it is disconnected from other performance data being generated elsewhere in the network and is used to understand this data in relation to the performance of multiple network components across the network layer. There is no mechanism, and the fundamental problem remains.

  Furthermore, such approaches tend to emphasize the use of pre-configured data collection software. Although the software can be instructed to collect a specific subset of data, the software appears rapidly and cannot be quickly revised or modified to address changing conditions. That is, the software can only collect data that was programmed to be collected initially, and such data can only be collected according to the initially programmed state. Thus, the functionality of such software is constrained by programmer insight. This lack of flexibility is a serious limitation, as it is completely impossible for a programmer to anticipate a wide variety of usage conditions, problems and events that may occur in connection with the operation of the communication network.

  This lack of flexibility is a problem even in situations where transient network conditions occur. In particular, because software cannot be updated quickly and easily to accommodate changing network conditions, the window of opportunity to collect data on transient network conditions is closed before the software modifies the data to the target value. . Thus, network managers and other personnel may not be given the opportunity to properly diagnose and resolve problems.

  Finally, the flexibility of such software is further compromised by the fact that such software is either loaded onto the mobile phone at the time of manufacture or must be downloaded from a website or service. Therefore, revisions or updates to the data collection software are cumbersome and take time to obtain. Because a variety of users can load the revised software at various times (even if it were), a statistically significant number of devices can capture the data required for a particular analysis. There is no quick and reliable way to ensure that it can be used by appropriate software to collect.

  In view of the problems in the above technology, the present invention provides a wireless communication network by utilizing multiple “endpoints” or end-user devices to capture “real world” usage and performance data across network components and layers. The present invention relates to a system and method for realizing a new method for monitoring a service. Since many errors that occur in relation to the performance of a particular service are only visible from the perspective of the end user's wireless device, the method of the present invention collects this data that would otherwise be unavailable. Enable. Further, the data collection methods described herein can identify interactions within and between multiple network layers and network components and accurately associate them with each other. In this way, valuable diagnostic data corresponding to service performance can be displayed and analyzed within a state frame that occurs between multiple layers of the network including the physical layer, transport layer, and application layer. it can.

  Data collection methods are already used in connection with many high-capacity endpoints such as networked PCs and only one highly specialized system with great constraints such as the Mars Lander system. However, the methods described herein provide relatively limited functionality (eg, constrained memory) to monitor and capture network and device performance data in response to changing network conditions. It is intended for service monitoring techniques that rely on a vast number of wireless devices with capacity and battery life) and complex dependencies (eg, interaction with RF networks that provide variable capacity and reliability). These states are often transient, geographically isolated and / or dependent on special interactions between network components. Thus, the data collection management system of the present invention addresses a different set of problems than the problems that known data collection systems address.

  Exemplary embodiments of the systems and methods of the present invention provide for monitoring one or more wireless communication network parameters, collecting data regarding such parameters, and conditional transfer of the collected data to a reporting system. Therefore, it can be realized by selecting and using a distributed wireless device, a wireless telephone, and a personal digital assistant (PDA) for telephone. In connection with wireless device monitoring and data collection activities, in some embodiments, the methods of the present invention use network nodes other than wireless devices, including wired devices, application servers, or other servers on the network. Data may be processed after collection as described above.

  One embodiment of the present invention provides for a quality of service platform of a data collection management system for dynamically generating and downloading a population of data collection profiles based on wireless device rules. Data collection profiles are offline as part of a data analysis solution that may be manually generated by a network administrator, software developer, or other person involved in the operation of the network (hereinafter referred to as the “network administrator”) Or may be automatically generated based on network parameters or other events. The profile defines not only conditions and events that cause the device to upload the collected information, but also what information and information that must be collected on the device in response to the conditions and events. A condition or event includes any occurrence in or on the network that the device can detect, such as a call interruption or a user pressing a button on the device. Conditions and events also include the passage of time or a request from the network administrator that the device returns a report of information to the server. In an exemplary embodiment, the purpose is to receive a data collection profile and perform data collection (“wireless device (s)”, “wireless communication device (s)”, The population of wireless devices (sometimes referred to herein as “target wireless device (s)” or “device (s)”) is the characteristics, capacity of the end user's wireless device And selected based on availability. Particular attention has been directed to implementing data collection to avoid overwhelming end-user devices as part of the quality qualification and selection process for subject wireless devices. Similarly, when defining data collection rules in a data collection profile, the impact of the desired data collection activity on device and network resources, the download of the profile that defines the rule, the data collection being considered, and the upload of the collected data Is defined to avoid adversely affecting network performance. For example, in some cases, upload of collected data may only occur during off-peak hours when device and network activity is particularly low. In addition, because network monitoring and data collection activities are expensive in terms of the devices and network resources utilized, in addition to a rigorous device quality qualification process, the method of the present invention allows irrelevant data during It provides the ability to structure the data collection profile to ensure that only the most relevant and reliable data that is filtered is presented to the analysis for a particular problem.

  The present invention may be practiced in conjunction with a wireless communication network such as a wireless cellular telephone network including a plurality of wireless devices, a wireless telephone suitably equipped to establish a connection to the network and communicate with the network. A data collection management system resides in a quality of service platform (SQP) that generates and manages data collection activities, processes and stores collected data, and a target wireless device or another target network node, which collects data It consists of a quality of service client (SQC) that allows them to participate in activities.

  In operation, the data collection and management system of the present invention allows multiple wireless devices and network nodes, each with SQC, to provide information regarding complex network status issues. In the following description, the present invention is not limited to the following modes of operation, but presents exemplary embodiments of the present invention in which queries are used to obtain such information. In accordance with this embodiment, complex network status problems can be described by “queries”, ie questions regarding the status or performance of network devices, or other aspects of network performance. In response to the initiation of such a query, SQP determines whether it is already storing data in one of its subsystems that meets the criteria specified in the query. If so, the data is available for analysis and the query is answered immediately. If the required data does not already exist in the SQP, a collection task request (CTR) is generated and processed by the SQP task subsystem. CTR is typically predefined as part of a data collection and analysis package that addresses a specific business or technical need. The CTR is then stored in the SQP and called when needed. In some cases, the CTR can be defined on demand by the network administrator. CTR is therefore used as the basis for building a data collection profile. The profile describes which parameters must be captured for each target wireless device that receives it, and which events cause the data to be captured, stored, and uploaded. Once the data collection profile is created, the target wireless device is selected based on criteria relating to CTR and available wireless device characteristics. Qualification features include device type such as manufacturer and model, usable memory and battery life, type of application residing on the device, geographical location of the device, usage statistics including what characterizes the user's interaction with the device , And customer profiles. The data collection profile is then provided to the SQC on the target wireless device.

  As noted above, in some embodiments, the data collection profile is also sent to other network nodes, such as application servers. As used herein, the term “target device” refers to a broader category of devices including wireless devices and other network nodes that SQP may utilize in data collection. According to the instructions specified in the profile, the SQC on the target device and other network nodes is collected, stored, implemented, and uploaded. Finally, SQP receives the reported data and processes it to provide useful information for decision support. Based on the reported data, additional queries and data collection profiles may be generated and executed to further explore and diagnose the problem. For example, based on a set of rules, additional queries are automatically generated, target wireless devices are automatically selected, and new queries included in the data collection profile can be sent to the selected device. In other situations, an existing data collection profile may exist on the device for any amount of time and instruct the SQC to continue to collect, implement, and upload data periodically. More generally, the data collection profile may be deactivated, refined, or replaced at any time.

These and other aspects of the present invention will become more fully apparent from the following description and appended claims.
For further advantages and features of the invention, a more detailed description of the invention may be understood by reference to the embodiments illustrated in the accompanying drawings. With the understanding that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope, the invention will be more specifically and specifically described by using the accompanying drawings. Explained.

  Reference will now be made to the drawings to describe various aspects of exemplary embodiments of the invention. It can be understood that the drawings are schematic representations of diagrams of such embodiments and thus do not limit the scope of the invention and are not necessarily drawn to scale.

  Embodiments of the present invention relate to systems and methods for using distributed wireless devices and other network nodes to provide monitoring and data collection for selected communication network parameters. The collected data is then analyzed, and the data collection operation can be further targeted and refined as suggested by the collected data and the associated trends. Data analysis is used in conjunction with, among other things, detecting network malfunctions and outages, implementing corrective actions, streamlining network operations, improving customer service, and developing sales strategies.

<I. Typical Wireless Communication Network and Wireless Device>
To illustrate the various methods of the present invention, FIG. 1 shows an embodiment of a wireless network 100 of the present invention. Of course, this arrangement and the arrangements and processes described herein are described by way of example only, and other arrangements and elements (eg, machine, interface, function, element order, etc.) are added. It is understood that some elements may be used altogether and some elements may be omitted altogether. Further, those skilled in the art will recognize that many of the elements described herein may be software, firmware, and / or in any suitable combination and location, either as separate components or in conjunction with other components. It can be understood that it is a functional entity that can be realized by hardware.

  The network of FIG. 1 is only an example of a suitable environment in which the present invention can be implemented and other network architectures are possible. In particular, wireless network 100 is described and illustrated as a code division multiple access (CDMA) network, while the present invention includes Global System for Mobile Communications (GSM), Universal Mobile Telephony, including those developed in the future. It can be practiced with communication systems (UMTS), time division multiple access (TDMA), wideband code division multiple access (WCDMA), general packet radio service (GPRS) networks, 802.11 networks and other networks. More specifically, in an exemplary embodiment of the invention, wireless network 100 provides radio connectivity and session management for radio frequency (RF) that provides circuit switched communication and communication based on packet data technology. Includes network. Accordingly, the wireless network 100 is through a network that includes one or more wireless devices 400, a base station 120, a base station controller (BSC) 130, a mobile switching center (MSC) 150, and a packet data serving node (PDSN) 140. Includes all elements necessary to send circuit-switched calls and / or packet data communications, or these elements are interconnected. More particularly, the method of the present invention can be used in conjunction with a circuit switched network, a packet data network, or both. Although the exemplary embodiment of the present invention uses the Internet Protocol (IP) as its transport protocol, the method of the present invention is a short message service (SMS) and the well-known short data burst service. Or other transport protocols may be used. Wireless network 100 includes a home location register (HLR), a visiting location register (VLR), a billing and issuing system, one or more gateways, servers and infrastructure required to use short messages and service messages. And may include many other network components not depicted in FIG. An application server or third party reporting system (depicted as network server 102 in the exemplary embodiment) may reside external to network 100 and can be used in conjunction with the present invention. References herein to the term “wireless network” or “network” are to be interpreted to include network infrastructure, servers, end-user devices, applications and services.

  The wireless network 100 may further include known third party network monitors, probes, and packet sniffers for use in conjunction with the present invention. Although the data collection management system 200 is depicted for purposes of illustration as functioning in conjunction with a packet data network, it may receive data from nodes on a circuit switched network, or system elements may be Located and may receive data from the packet data network. Gateways, network monitors, and other network components may be utilized to facilitate the transmission of data between the network and the data collection management system 200. Furthermore, in one embodiment, the components of the data collection management system 200 may be implemented on one or more network servers that reside within the network of the network operator. Alternatively, since the data collection management system 200 may be realized as a service hosted by a service provider other than the network operator, the components of the system reside outside the network operator's network, and the network operator's network It may be in the form of communicating with various nodes.

<II. Structure of a typical data collection management system>
FIG. 2 shows details regarding an embodiment of a data collection management system, indicated generally at 200, along with some of the relevant components of the wireless network 100. The functionality described herein can vary widely, either wireless or wired, including wireless, personal digital assistant (“PDA”), network servers, desktop computers, and other devices associated with the network. May be used in conjunction with a wide range of network devices.

  In an exemplary embodiment, the data collection and management system 200 of FIG. 2 allows any end user to access telephone services and any suitable device that allows the target wireless device 400 to participate in the packet data network. One or more target wireless devices 400 are configured to provide access to the Internet or multimedia data through various protocols. Target wireless device 400 communicates in the environment of wireless network 100 and may be virtually any mobile wireless device that is a wireless telephone handset, a wireless personal digital assistant (PDA), or another wireless communication device. The wireless device may be referred to as a mobile station or a mobile device, and is hereinafter referred to as a wireless device.

  The target wireless device 400 is transmitted over the wireless network 100 to locally store data that enables execution of software such as operating system and applications, and to otherwise support the operation of the wireless station Has local memory used. Various known operating systems such as Symbian, REX, or any other suitable operating system may reside on the wireless device. Further, the target wireless device 400 receives and executes the data collection profile, otherwise communicates with the quality of service platform (SQP) 201 of the data collection management system 200 so that it can participate in data collection activities. Enabled by Quality of Service client software (SQC) 402. The SQC 402 can be installed in a variety of ways, including installation during manufacture, various download methods such as installing the device in a cradle connected to a computer via cable, or using any number of wireless methods, etc. It may be installed on the wireless device.

  Development environments such as BREW and JAVA may also reside on the wireless device and support such download of the SQC software 402 or otherwise support the support operation of the data collection system. In addition, the exemplary target wireless device 400 may provide various access to one or more services provided by a network service provider or provide utilities or entertainment to users such as calculators and games. Including application software. A typical wireless device 400 also includes a wireless transmitter and receiver, voice encoding and decoding, and circuitry for call control, a display device, a keyboard, a power supply, and a subscriber identity module (SIM) chip. It's okay. In other embodiments, the SQC software 402 may also be installed on other nodes in the network to receive and execute data collection profiles as noted above. For example, in addition to wireless device data collection activities, SQC software may be installed on an application server in the network or on a PDSN so that these network nodes can participate in the data collection activities.

  With continued reference to FIG. 2, the data collection management system 200 includes SQP 201 and SQC 402. The elements of the SQP 201 include a metric collection subsystem (MCS) 202 composed of a metric collector 202A and a metric analyzer 202B, a task processor 204A, a task database 204B, and a task composed of a target device database 204C. A reporting subsystem comprising a profile subsystem 206 comprising a subsystem 204, a profile builder 206A, a profile editor 206B, and a profile database 206C, a reporting engine 208A, a rules engine 308B, and a reporting database 210. 208, and an interface subsystem 212 composed of a user interface (UI) 212A and an API / service interface 212B. The data collection management system 200 further includes one or more target wireless devices 400 or other network nodes in which the SQC 402 resides. Further, the data collection management system 200 may communicate with one or more network servers 102, such as a third party reporting system.

  As described above, the data collection management system 200 of FIG. 200 may be implemented as a service hosted by a service provider other than a network operator, or in other implementations, some of the functionality of the data collection management system 200 or All may be provided in the form of a stand-alone software package suitable for installation on a network server and / or other network systems in the network operator's wireless network 100. Accordingly, the scope of the present invention is not limited to a particular type of implementation of the data collection management system 200.

  The SQP 201 of the data collection management system 200 is generally useful for creating and managing various queries and creating data collection profiles such as those utilized in data collection. In addition, SQP 201 receives the collected data and performs various analyzes and other processes on the collected data.

  With continued reference to SQP 201 of FIG. 2, metric collector 202A of MCS 202 receives data collected from multiple wireless devices 400 or other network entities. The collected data may be provided either directly to the metric collector 202A or via other network nodes such as PDSN 140 or gateway 160. The MCS 202 may communicate with the task subsystem 204 and the reporting subsystem 208. The metric analyzer 202B may perform validation functions, such as incident matching of received data, augmentation functions and / or analysis functions before passing it to other subsystems of the SQP 201. Further, the MCS 202 may receive and respond to a collection profile request from the SQC 402 compatible device. For example, upon initial startup of a device enabled with SQC 402, the SQC indicates its presence to SQP 201 via MCS 202 in response to one or more profiles that may be downloaded to the device.

  The reporting subsystem 208 stores a report generated as a result of the input from the MCS 202 and data from an external system such as a third party network monitor in the reporting database 208C. The report database 208C may export or publish the report data to another report or presentation system depicted in the exemplary embodiment as the network server 102. The reporting database 208C may be integrated with the data collection management system 200 or another network component, or it may function as a stand-alone database. In addition, the reporting subsystem 208 plays a central role in the collection performed as described below.

  In one embodiment of the invention, the invention is not limited to the following modes of operation, but information regarding complex network status issues can be obtained using the following procedure. When processing a new query in accordance with this embodiment, the reporting subsystem 208 first checks whether the reporting database 208C already has sufficient data necessary to answer the query. If no additional collection is required to satisfy the query, data and / or reports are returned to the network administrator. If the reporting database 208C does not contain data to answer the query, a CTR is created, tracked by the reporting subsystem 208 throughout its life cycle, and processed by the task subsystem 204. The task subsystem 204 selects an existing data collection profile or generates a new data collection profile as needed, identifies the appropriate set of target wireless devices, and sets the data collection profile to the selected target wireless Communicate with the profile subsystem 206 for transmission to the device. If the query requires collection from a non-SQP device, that is, a device that is not enabled in SQC to allow it to participate in the data collection management system 200, the reporting subsystem 208 is externally connected through the interface subsystem 212. Create and send a CRT for the system. In this way, the tasks required are not directed by the data collection management system 200, but are documented and tracked within the system and other data collection activities being performed by the system. Can be displayed. It should be noted that the task subsystem 204 and / or the reporting subsystem 208 may consider the query and / or CTR priority when generating the data collection profile and identifying the target device population. Any of the task functions can be replaced by the operator, and in another embodiment of the invention, the operator assigns the profile directly to the device.

  SQP 201 may communicate with one or more network servers 102 to provide data collection management services. For example, the reporting subsystem 208 of the SQP 201 can provide the network server 102 with reports, raw data, and other data products. Network administrators and external systems can then access the data via the network server 102. The components of the data collection management system 200 indicate that the requested data is collected and can be used for analysis in order to start communication with one or a plurality of target wireless devices 400 or to the network administrator. To notify, it may communicate with various messaging services in the wireless network 100, such as a short message service center (SMSC) 180.

  SQP 201 uses a profile subsystem 206 that cooperates with task subsystem 204, rules engine 208B and target device database 204C to generate or modify various data collection profiles that govern the collection of data regarding various network operations and parameters. In addition. To achieve this goal, the profile subsystem 206 includes a profile builder 206A, a profile editor 206B, and a profile database 206C. The profile subsystem 206 is configured and realized by various methods. In one implementation, the profile subsystem 206 comprises SQP 201 components. In another implementation, the profile subsystem 206 is a stand-alone system configured to communicate with the data collection management system 200. Accordingly, the scope of the present invention is not limited to the implementation of the profile subsystem 206 shown in FIG.

  Next, the data collection profile generated in connection with the profile subsystem 206 is downloaded to the specified set of target wireless devices 400. The tool can also be used at this point to identify target devices that match the profile and are suitable for the query. The SQC 402 resident in the subject wireless device 400 receives the data collection profile and not only in response to other rules and instructions of the data collection profile but also “triggers” that start and end the data collection activities set in the profile. In response to the data capture process. The data collection rules, triggers, and other instructions included in the data collection profile are sometimes collectively referred to herein as “data collection instructions” for convenience. As suggested above, the target wireless device 400 may be a mobile phone or other wireless device. However, as described above, the target device may be any other network component or node that comprises SQC software that allows it to collect and upload data to SQP 201.

  Data collected by the target wireless device 400 may be buffered at the SQC 402 and awaiting creation of a metric package. Once created, the metric package may be placed in the metric archive 402 of the SQC 402, deleted, or uploaded to the metric collector 202A of the MCS 202.

  The UI 212A of the interface subsystem 212 may be a graphical user interface that allows one or more network administrators to interact with and control the operation of the data collection management system 200. In this regard, at least some implementation of the data collection management system 200, i.e., UI 212 </ b> A, facilitates generation and manipulation by the network administrator of the queries described below with respect to data collection for one or more aspects of the wireless network 100. To do. There are cases where such queries may be generated or refined automatically based on reports generated and output by the reporting subsystem 208, or as a result of processing by the rules engine 2080B. In addition, the UI 212A may be used by a network administrator to retrieve data and reports from the system, and the API / service interface 212B is generated from the data collection management system 200 for submitting commands and requests to the system. It may be used by external systems to retrieve reports and data.

  Finally, the target device database 204C includes various types of information related to the target wireless device, such as a mobile phone, and other network nodes that comprise the components of the wireless network 100. Target device database 204C may be implemented in many different ways. It may be an existing network component in the service provider's network, such as a home location register (HLR), where information about the characteristics and operation of the wireless device is stored and accessed by components of the data collection management system 200 . Alternatively, the target device database 204C may reside in the data collection management system 200, in which case the target device database 204C collects device usage information and device characteristics when the device registers with the wireless network 100. In other cases, a data collection profile specifically aimed at collecting information with the target device database is a status and usage information about the applications such as applications, base stations encountered, etc. that are resident and used in the wireless device Can be reported to the target device database 204C. By providing such data and periodic updates to the target device database 204C, the SQP 201 can fine tune its device selection criteria for more effective device targeting. In many cases, the target device database is data from multiple sources, including HLR data, data generated by profiles, and data from other databases associated with user accounts, device characteristics, or network characteristics, among others. Is used. As a result, the data collection profiles are downloaded to those target wireless devices 400 that are most suitable for collecting data in response to one or more configured queries. Since the above-described interfaces to a plurality of sources are known, description thereof will be omitted in this specification.

  Thus, the target device database 204C includes information regarding specific hardware features of the target wireless device, such as the model and manufacturer of the target wireless device, one or more account types associated with the target wireless device, battery type and memory size, One or more target wireless devices 400 in connection with specific queries relating to billing issue addresses, usage information, installed applications, frequent geographic regions, and communication networks associated with the target wireless devices Other information may be included to facilitate selection. The database can be created from information collected from the device itself via a data collection management system or by any other suitable means. In addition, the foregoing is merely an example, and more generally, virtually any other target wireless device characteristic that is desired to be utilized or considered relevant is the target device database. 204C may be included.

  Furthermore, certain devices may be targeted to perform multiple data collection profiles in order to allow the data collection management system 200 to perform multiple data collection profiles and data analysis activities simultaneously. Thus, the target device database 204C tracks data collection activity that occurs on the device and maintains detailed information about the specific data collection profile that is active on the device. In doing so, it can detect and resolve conflicts or priorities by adjusting the population of selected target devices.

  The target device database 204C may store information about non-SQP entities, such as the location of non-SQP devices in the wireless network 100, the geographical area in which the service is provided, and available data, in the network. In general, virtually any other characteristic or attribute of a non-SOP entity that may be considered relevant may be stored.

  Note that FIG. 2 illustrates one method of assigning various functionalities associated with the data collection management system 200. Accordingly, the functionality described herein in connection with the data collection management system 200 is intended for the various types of distributed systems and devices utilized, the data desired to be collected, and the communication networks of other systems. It may be assigned in various other ways including, but not limited to, structures. Accordingly, the functionality assignment shown in FIG. 2 is only an example and does not limit the scope of the present invention.

<III. Typical query>
Referring to FIG. 3, details are shown regarding a method and system for setting up and presenting a query in connection with the data collection management system 200. Such a query may be established with reference to any of a variety of different ideas regarding the operation or functionality of the target wireless device 400 or other network node associated with the wireless network 100. Queries collect performance information about the impact of simple activities such as button presses by the user, or the information is more complex including multiple network layers such as physical layer, network layer, transport layer and application layer It may be structured so that it may be collected for transactions. In particular, the target wireless device has a software stack that communicates with various network layers of the communication network.

  Once collected, the information associated with the device's software stack can then be correlated, so that information about the software stack or the network layer with which it communicates is implemented along with the corresponding RF status information. RF status information may be displayed, providing a complete display of interdependencies between different layers and a more accurate analysis of the information. However, these are only typical query subjects, and such queries may more generally be important to the wireless network 100, the associated target wireless device 400, and the network administrator or other personnel. May be directed to substantially any aspect of any other network node.

  A typical query structure is shown generally at 500 in FIG. Query structure 500 is configured to have any of the functionality described herein. As illustrated in FIG. 3, one way in which the query structure 500 can be set is by interaction between the network administrator and the SQP 21 via the interface subsystem 212. When configured by the network administrator, the query structure 500 may be a pop-up window or other type that allows the network administrator to make various choices regarding the nature and scope of the desired query or queries. Appears as a device. Although the network administrator has the ability to interfere with the system and write queries on demand, the preferred embodiment of the present invention provides a large number of “preconfigured” query and profile databases 206C in the reporting subsystem 208. Provide a stored attachment profile. As business and technical needs for data collection arise, these pre-configured queries require to generate a complete analysis package that includes one or more data collection profiles and associated reports Is done.

  Any number and type of query structures 500 may be any of the services or entities provided by the network administrator in connection with the wireless network 100, one or more associated target wireless devices 400, or the wireless network 100. Queries can be constructed to allow pre-configuration, creation, and submission for virtually any aspect of the performance and operation of the system. Thus, the query structure 500 shown in FIG. 3 is merely exemplary and is in no way intended to limit the scope of the present invention.

  In yet other implementations, one or more query structures 500 are set that do not include predetermined menu options. Rather, the network administrator sets up a highly customized query structure by simply dragging and dropping the desired items into the basic query structure template. The profile system 206 can be configured to prevent the setting of queries and related data collection profiles that are internally inconsistent, i.e., inconsistent. In this way, the network administrator receives some indication of a guarantee that no “noise” data will be collected in connection with a predefined query or data collection profile.

  In addition, the queries as well as the data collection profiles described elsewhere in this specification can be manually performed in response to the occurrence of one or more system events or as a result of analysis of previously collected data. Or it may be generated either automatically. The data collection profile includes “triggers” that command the start and end of data collection activities. Hereinafter, the manner of setting and using the trigger will be described in more detail.

  FIG. 3 conceptually illustrates various aspects of the query. In particular, the query structure 500 allows the network administrator to provide information about the target device (502), software application (504), services associated with the target device (506), radio frequency system (508) utilized by the target device, And a query related to any of a variety of other considerations or parameters (510) that may be important to the network administrator or other personnel, including various aspects of the data collection management system itself. Show. Significantly, these queries, in any combination, are directed to one or more of the aforementioned aspects of activity occurring at the device or within the network and generated in connection with the activity. The captured data may be captured.

  When creating a query related to the target device aspect (502), the creator targets the feature 502A, such as hardware, software, and other aspects of the device, such as the target wireless device 400, more specifically. be able to. Similarly, a query for an application (504) running on the target device may include the performance of the application residing on the target device, the usage of each such application, and otherwise residing on the target device. Features 504A, such as various other aspects of applications associated with wireless network 100, can be specifically targeted.

  With continued reference to FIG. 3, queries regarding services (506) available to the target device are set for a particular service 506A, including IP, messaging, voice transmission, and various other services associated with the wireless network 100. Similarly, the queries can be directed to RF-related aspects (508) of the wireless network 100 and / or the target wireless device 400, such as RF performance parameters, issues and other considerations.

  Embodiments of the data collection management system 200 include related wireless devices that one or more network administrators or other personnel may be of interest to the wireless network 100 and / or the administrator (s). One or more queries directed to substantially any aspect of 400 can be set up and implemented. As a result of such functionality, the network administrator is not limited to a predetermined choice of query subjects, but more generally one or more directed to any subject that may be of interest. You may set a query. In addition, the query structure 500 and its associated systems allow the generation and use of special types of queries that correspond to a particular set of interests.

  For example, some query structures 500 relate only to information that may be of interest to the sales department, such as the dissemination of applications being used. As another example, the accounting department, either alone or in conjunction with the sales department, utilizes queries that correlate various service plans with specific groups of end users and their actions. It may be desired. As another example, a technician associated with wireless network 100 and associated target wireless device 400 may wish to invoke queries regarding various performance aspects of the network and target device. The technician may also be interested in the performance of other network nodes such as media servers or packet data supply nodes. As a result, such personnel are not limited to a number of query structures that allow them to quickly and easily set up multiple useful queries that can be used to identify problems and improve services. Access to pre-configured queries.

  Each of the query sets may flow out of a single query structure 500, or each may constitute a separate query structure that can be accessed only by certain personnel as described above. These are merely exemplary implementations, but the scope of the invention is not limited to these examples.

<IV. Typical data collection profile>
As described above, a wide variety of pre-configured queries regarding a number of aspects of the wireless network 100, the associated target wireless device 400, and other components that make up the wireless network 100 can be generated by the data collection management system. Can be used to monitor and collect. As described above, the network administrator can also create a custom query upon request. Once one or more queries have been established and it is determined that the existing data is not sufficient to satisfy the query, the CRT can collect the data in response to the one or more queries with the target wireless device 400 or One or more data collection profiles are generated or selected to direct other network nodes.

  In an exemplary embodiment, data collection operations are pushed to a population of target wireless devices 400 for data collection activities. Since the desires at each device, including any existing task requests, are already considered as part of the device selection criteria, the data collection overhead imposed on each of the target wireless devices 400 is minimal. Furthermore, as described herein, the collection functions and operations of various target wireless devices 400 provide a powerful mechanism for easily and effectively collecting data that responds to queries.

  FIG. 4 illustrates an aspect of a data collection profile 600 as described above. As generally shown in FIG. 4, the data collection profile 600 includes various parameters that define not only the set of data that must be collected, but also the conditions under which the data collection takes place and how the data must be collected and processed. 602. In this regard, the data collection profile 600 is directed to a specific type and amount of data corresponding to a query set via the query structure 500 when the data collection profile 600 is collected and processed. Configure query refinements associated with.

  The data collection profile 600 can be configured to become active (ie, initiate the collection process) and / or become inactive at a certain time or after a certain period of time. The data collection profile and / or its component collection instructions may also expire after a certain period of time. When a data collection profile or its component data collection instruction expires, the device may return to its default profile, or it notifies SQP 201 via MCS 202 that it has expired, so that it is instructed to receive another profile. The In either case, the expiration function allows the device to effectively “purge” profiles or individual collection instructions that are no longer supported and ready to receive new profiles or collection instructions. The data collection profile may be configured to include a plurality of triggers that begin and end data sampling over a period of time and in response to certain occurrences. In some cases, data collection may occur in response to a trigger set as the occurrence of an event. In other cases, data collection may be initiated or terminated in response to a trigger set as a lack of event occurrence. For example, a data collection profile may specify that it starts on arrival at a certain geographic region and aborts if the user cannot receive a voice call within an hour after entering the specified geographic region. In this case, entering the designated geographic area is a start trigger where data collection begins and data is buffered, and when a voice call cannot occur within one hour, the abort trigger stops data collection. In this case, the buffered data is not converted into a metric package, which may or may not be deleted.

  In some cases, data collection may be terminated and resumed if a designated network event occurs. In the unlikely event that this second event occurs, i.e., the trigger is not activated within a subsequent period, the data collection profile may expire. The data collection profile may also include instructions for the conditions under which the collected data is converted into a metric package, and whether and when the metric package is uploaded to the metric collector 202A. In other cases, the data collection profile 600 reports information about both the occurrence and non-occurrence of specified events to the metric collector 202A, as specified, as well as whether data was collected over a period of time. You may instruct it to be done.

  In the exemplary embodiment, a target device database for profile builder 206A of profile system 206 to facilitate setting up data collection profile 600 and to set a set of target devices to which data collection profile 600 is sent. Operates in connection with 204C. As shown in FIG. 2, the profile subsystem 206 retrieves one or more existing data collection profiles when the network administrator requires manual changes to those data collection profiles. A profile editor 206B that allows editing. Further once completed, the data collection profile 600 may be stored in the profile database 206C for reuse, backup, and / or for additional editing, copying, or other processes.

  In any case, the data collection profile 600 depicted in FIG. 4 specifies, among other things, one or more types of data that must be collected in connection with the data collection profile 600. As suggested above, the data collected in connection with the data collection profile 600 may comprise substantially any type of data regarding the wireless network 100 and / or the target wireless device 400. Thus, the data that is desired to be collected includes the UI of the target wireless device 400, one or more applications resident on the target wireless device 400, network services accessed by the target wireless device 400, applications, network servers, etc. It may relate to the performance of other network elements and considerations for various network layers such as the transport and physical layers.

  In addition to specifying various data types, the illustrated data collection profile 600 can also specify time parameters, ie one or more time frames during which some or all of the specified data must be collected. . For example, as a result of a report of calls interrupted in a particular area, a data collection profile intended to analyze the interrupted call is generated. In this case, the data collection profile may specify that data collection should begin as soon as the wireless device enters a certain geographic region. The collected data enters a buffer on the device so that it can be included as part of the metric package when a trigger occurs that creates the metric package. The data collection profile must store a 5-second buffer of information that precedes any call that occurs while the device is in the specified geographic area as part of the data collection rules, and the data is You may further specify that they must be collected. If the call is interrupted, another 5 seconds are captured after the call is interrupted, and data is captured before, during, and after the call is placed in the metric package.

  In some cases, the data collection profile may specify that certain data is collected on some type of predetermined schedule. In other cases, the data collection profile 600 specifies that some data is collected regularly, i.e. periodically, while other data is collected irregularly, only once or some other non-periodic. You can do it. In yet other cases, the data collection profile specifies a start date and / or specifies that data collection must begin on one particular date and end on another particular date, regardless of any other conditions. Or it may have specified the stop date in advance.

  With continued attention to FIG. 4, the data collection profile 600 determines whether and when the data collection process specified in the data collection profile should start, abort, and end, and when it should start, abort, and end. Includes, refers to, or relates to various triggers used to determine what is not. For example, one known technique for initiating network-based immediate connection communication (also known as Push-to-Talk ™) is that the originating wireless device is one or more terminating parties. Sending a Session Initiation Protocol (SIP) INVITE to a network based instant connection server to request a wireless device to participate in a communication session. INVITE is received by the immediate connection server and transmitted to the terminating wireless device. If the terminating wireless device has been recently active, the server may send a “try 100 times” message to the originating device while it continues trying to contact the terminating device.

  Normally, when a 100 TRYING message is received at the originating device with an indication that it will receive a question, the originating device starts transmitting voice data. While the calling device is busy sending voice data, the immediate connection server tries and the terminating device may not actually be reached and a 408 TIMEOUT message is sent to the calling device. At this point, the user of the originating device has a very poor user experience. At the same time, the service provider captures all unsuccessful transactions that occurred between the originating wireless device, the instant connection server, other participating network components such as the packet data serving node (PDSN), and the terminating handset. Does not have the ability to Therefore, it does not have data that can handle and solve the problem.

  One embodiment using the method described herein is resident at the originating device to initiate data collection when the start trigger is activated (in this example, an immediate connection communication attempt originates). Provide a data collection profile to The termination trigger that the originating device receives the aforementioned 408 error message causes the SQC 402 residing on the originating wireless device to stop collecting data and create a metric package for upload to the metric collector 202A. It is a condition. In some cases, the trigger causes the data collection activity to abort without creating a metric package. In an exemplary embodiment, the receipt of a 408 error causes the data to specify that a metric package is created immediately and uploaded to the metric collector 202A, resulting in a condition that the metric package is uploaded accordingly. A collection profile is further configured. In this example, the terminating wireless device is not aware that a communication attempt has been made, so the trigger is activated at the terminating device to initiate data collection and / or cause the creation of a metric package. It has not been. Commands such as SMS messages are terminated when a metric package is received by the metric collector 202A from the originating wireless device to ensure that important data related to the communication attempt is collected from the ending device. Will be sent to the other device. This command triggers “on demand” data collection activity at the terminating wireless device, driving additional data collection required, and activating a trigger to upload the metric package at a specified time. It should be noted that “on-demand” collection activities may occur in addition to unrelated data collection activities that may have already occurred on the device as part of normal operation. In another case, the terminating device has a profile that instructs it to continually collect data relating specifically to the problem under investigation. In this case, it is the timing and frequency of INVITE reception. As a result of the on-demand upload, the history associated with this particular region of interest is uploaded to provide full visibility of the events occurring on the terminating device and the data around the 408 issues under investigation. In addition, in some situations, the collected data can be min / max / average / count type processing, or add or otherwise reduce the amount of data that must be sent from the device to the SQP. It may be exposed to processing on the device such as calculation. In this case, the data sent to the metric collector has already undergone some processing and is not simply unprocessed collected data.

  In another embodiment, the SQC may reside on another network component, such as an instant connection application server, and the data collection profile is configured to instantly create and upload a metric package when a 408 error occurs. You can. In this case, when the 408 error is recognized by the immediate connection application server, the SQC causes a metric package to be created and uploaded to the metric collector 202A. The SQP 201 then causes an SMS message to be sent to the terminating device, so that “on demand” data collection activity at the terminating device begins and proceeds as described above. More specific information regarding the trigger when the trigger is associated with the data collection profile and various other aspects of the data collection management system is provided elsewhere herein.

  At the same time that the metric package is being created on the target wireless device 400, the SQP 201 may receive any other network component that includes a data collection profile associated with the originating wireless device and terminating wireless device and with an immediate connection communication attempt. The analysis task of the reporting subsystem 208 that is created to analyze the metric package received from may be queued. The metric package generated at the terminating wireless device includes the metric package generated at the originating wireless device and / or the measurement generated by the instant connection application server and any other network components associated with the transaction. An identifier associated with the reference package is assigned. At any time, a query may be executed against the reporting subsystem 208 to identify a metric package that is associated with other metric packages, so between all of the network components involved in a device-to-device transaction. You can view, track and analyze the complete sequence of events that occurred in Reports generated from the reporting subsystem 208 may be stored in the reporting database 208C for later use, or the underlying data may be exported to the third party reporting system via the interface subsystem 212. Furthermore, in some embodiments, notifications regarding the results of data reception or analysis can be sent to other systems as well as to network administrators and other personnel.

  As another example of parameters used to guide a data collection task, the data collection profile 600 can specify certain geographic constraints and restrictions. More specifically, such geographical constraints are such that the data is only within the specified geographic boundary range by the target device that makes or receives calls, or within a certain proximity range to a particular base station. May specify that it must be collected by the target device. The data collection profile may further specify that only those devices experiencing a particular problem in a certain geographic region should collect data. Such information may allow, for example, network administrators and other personnel to isolate problems that recur in a particular area and clarify the steps for correcting the problems. For example, such geographic information is used to help identify malfunctioning base stations or other devices when there are a significant number of dropped calls within established geographic limits. May be. The information may further help identify where a device, such as a signal enhancement device, must be installed in the network to improve service. Further, the geographic information allows for the creation of a graphical depiction that indicates the location and concentration of the problem with various conditions and / or wireless network 100, such as a map.

  As further shown in FIG. 4, the data collection profile 600 specifies a particular service as to what data must be collected. Some implementations of the data collection profile 600 may specify that transmission signal (“Tx”) strength data is collected for Internet Protocol (IP) data calls initiated by an instant connection communication session. In other cases, the data collection profile 600 may specify that data is collected for a messaging service accessed by the target wireless device 400. The wireless device participates in each service delivered to the end user and collects metrics related to the specific service to take advantage of the software stack that communicates with each network layer required to deliver the service, The ability to ignore those metrics when not doing the service provides an important advantage over existing solutions. For example, to monitor transmitted signal strength under the conditions described above, known network probes are used for all services (eg, telephone, messaging, browsing) for all wireless terminals across all transports (eg, voice or IP). , And immediate connection communication), the total transmission power must be monitored. In other methods than those described herein with respect to the present invention, the scale and scope of this data collection process effectively renders this data uncollectable.

  In some cases, only when other aspects of the system are in a fixed configuration, a data collection profile can be set up that collects information about the performance of the service. For example, a query may include analyzing the low performance of a Wireless Application Protocol (WAP) browser page at a service provider portal from a particular location. A WAP browsing session takes advantage of the IP layer running on the RF layer to connect to the large IP network, then to the WAP gateway, and to the portal in question through another IP network. While any one of these systems can cause performance degradation, RF degradation is a frequently encountered problem. It is therefore useful to be able to remove RF related data so that other possible causes of poor performance can be found. In this case, the target wireless device 400 includes a data collection profile that is tailored to collect data for very special network conditions where loading a WAP page takes longer than a certain set time. The request by the device for WAP page load is an initiator trigger that initiates data collection. The data collection profile analyzes RF performance such as frame loss and automatically ignores events that occur in high loss situations. Because RF is usually the most unreliable element in the network, this simple filter ignores the most frequent and irrelevant cause of delay in loading WAP pages in this case, and the focus of the data It dramatically increases the value of the information reported by ensuring that it is possible to be at a higher layer. Similarly, the data collection profile can ignore results collected in scenarios where the IP service is not performing properly. In addition, since the wireless device includes all service layer software implementations, the data collection profile can be set to compare information from any layer to qualify the analysis operation.

  In other cases, the data collection profile can be configured to utilize parameters and status of other services on the wireless device to filter data collection. Wireless devices have many different services running on them, such as voice calls, messaging, games and cameras. These services interact both on the device (contention for resources such as processors and batteries) and within the network (contention for bandwidth). The number of these services increases rapidly and systems such as BREW and JAVA allow users to download their own selection of services after the wireless device is owned by the user. Thus, due to the increasing number of potential interactions between these services and the problems in analyzing all possible scenarios, a substantial number of failures will occur for these service interactions. There is no doubt.

  For example, a multimedia messaging service (MMS) system uses a model in which messages are downloaded “in the background” to a user's wireless device to provide an “instant access” experience. Applications such as Instant Connect communications, also known as Push to Talk ™, have very demanding requirements and expectations for data channels. If an MMS download occurs during an instant connect communication session, the user experience can be greatly affected and the instant connect communication session may not function as it should work. Because the cause of this problem is invisible to the user, the amount of interaction with customer care does not resolve why an instant connection communication session sometimes delivers very poor performance. To address this, a data collection profile can be set with an initiator trigger that initiates data collection when multiple applications or services are requesting data access simultaneously. Data collected according to this profile allows service providers to evaluate this and other unforeseen conflicts between services.

  The data collection profile 600 also specifies various parameters relating to the processing and management of data collected by the target wireless device 400. In the depicted embodiment, the data collection profile 600 specifies a schedule in which data collected by the target device should be uploaded to the SQP 201 accordingly. In one implementation, the other data collection profile 600 may specify a one-time upload, that is, a real-time upload of collected data, but the data collection profile 600 specifies a regular upload schedule. Similar to other aspects of the data collection profile 600, the upload schedule may be set to suit the requirements of a particular application, operating system or target device as needed. The upload schedule may also be event driven, which may depend on other factors including end user activity, available device battery life and coverage status. For example, the upload may be omitted or delayed when the battery of the target wireless device 400 is low so that it does not significantly affect the user experience by reducing the limited available power. Another example of how to protect the user experience is that the SQC 402 builds the end user activity pattern and matches the upload schedule that is invoked in the collection profile to the experimentally derived low usage time. Can be adjusted.

  The upload schedule can be based on the status of the target device. This status can relate to the event or condition of the target device itself. Furthermore, the status of the target device may be related to network conditions and target device interactions within the network. For example, network conditions that can determine the upload schedule include signal quality, high or low contention, non-roaming services, presence of high-speed data services, and the like.

  In another example, the RF environment is analyzed and the target wireless device 400 can determine whether it is efficient for both the device and the network to upload the metric package. For example, the amount of power that a wireless device consumes for data transmission depends on the power level required to achieve an appropriate signal to noise ratio with the tower. A non-time-critical metric package upload may be set to proceed only if the power setting is below a preset maximum. This ensures that only minimal power is consumed for operation. Additional refinements of this model may include changing the preset maximum power based on the amount of time that the upload has been delayed.

  Additional aspects of the data collection profile 600 include the ability to specify device configurations for which the collection profile is compatible. As more and more user definable and downloadable features are available for wireless devices, it becomes impossible for the target device database 204C to have all the information about every aspect of the target wireless device 400. . For example, the user can download a customized ringtone that utilizes multiple different models for playback. If a suspicious issue appears to occur only on devices attempting to play a MIDI-based ringtone that is 30 seconds or longer in length, set up a data collection profile and select the target population for the wireless device Although the data collection profile can be activated only if the target wireless device 400 has that particular configuration. In this case, the data collection profile can be downloaded to a large population of devices that meet certain known criteria, while it can only be activated on certain devices that use MIDI-based ringtones that are longer than 30 seconds in length . The target wireless device 400 that is not compatible with the data collection profile 600 can either ignore the data collection profile 600 or purge it from the memory of the target device. Among other things, this functionality helps to eliminate irrelevant, ie foreign data accumulation.

  Data collection profile 600 incorporates various other related functionality in some cases. For example, one or more data collection profiles 600 may indicate instructions that the target wireless device 400 should take if a conflicting data profile is in effect for the target wireless device 400. Including. As another example, several data collection profiles 600 specify the priority of data collection activities specified therein relative to one or more others associated with the target wireless device 400. Instructions to do. While multiple data collection profiles may be run simultaneously, there is a higher urgency associated with the required data, or when the device meets the data collection requirements of multiple queries and the target wireless device There may be a need to prioritize data collection profile execution when the capacity of 400 is such that simultaneous collection of data streams specified by two or more data collection profiles 600 cannot be achieved. . To address this potential discrepancy, the data collection profile may be assigned a priority level, and once downloaded to the device, the relative priority level is evaluated by the SQC 402 residing on the device, and the profile is It may be executed according to its respective priority. Once data is collected according to the priority specified in the data collection profile, a metric package may be created and placed in the metric archive of SCQ 402. Although such functionality is exemplary only, various other rules for data collection profile 600 may also be used.

  As mentioned above, the data collection profile 600 specifies, includes, or otherwise specifies not only the conditions under which such data is collected, but also other information, guidelines or rules regarding the data that must be collected. Can be incorporated. Some data collection profiles 600 include revocations that allow the target wireless device 400 to voluntarily select itself from the data collection process. The data collection profile 600 may include or specify various parameters regarding one or more target wireless devices 400 or associated hardware and / or software.

  As noted above, data collection profiles may be set up and stored for later use, or they may be created “on demand” in response to certain business and technical needs. Furthermore, in many cases, the data collection profile is automatically derived from specific data observed after analysis of the metric package. They may be guaranteed when a certain threshold is reached or when a trend is identified. Thus, the data collection profile is derived from one or more characteristics of the collected data such as trend information, pre-set data points or thresholds. As an example, the data collection profile is not wireless data, but may be configured to collect RF performance information, such as an interrupted call. If there are a significant number of interrupted calls in a particular geographic area, this example is more specific to allow for existing data collection profile modifications, or evaluation and diagnosis on a relatively large number of interrupted calls. It serves as the basis for generating new data collection profiles for wireless data collection purposes. Thus, even if the wireless data is not the initial target of the data collection operation, the trend or threshold analysis of the collected metric package is the basis for additional related data collection operations.

<V. Typical query setup process>
FIG. 5 now shows details regarding the process 700 for setting the query used in subsequent creation of the data collection profile. As noted above, numerous business and technical issues can be identified and a complete data collection package including data collection profiles and reports can be created and stored for later use. However, in some cases, human intervention may be required to respond immediately to a previously unknown set of events, and network administrators can create “on-demand” data collection profiles. May wish. A mechanism for creating a data collection profile on demand is shown in FIG. At step 702 of process 700, a threshold question that is intended to be answered by the data collection operation is identified. As shown in FIG. 5, the identification of the threshold question stage of the query process may be based on the results of various other reports generated for previous data collection operations. In this way, the network administrator may be able to sharpen the query using previously built knowledge. This also enhances the construction of associated data collection knowledge. At this point, the threshold question may be nothing more than a plain language statement or a question about the specific data desired, or a question to be addressed.

  Once the threshold question is identified, process 700 proceeds to step 704 where the network administrator accesses the query screen via the UI. Process 700 then proceeds to step 706 where the network administrator uses the query screen to narrow down the threshold questions, ie, the questions identified in step 702. As described above in connection with FIG. 3, such refinement of the threshold question can address one or more problems that the tree, cascading menu, and / or network administrator must address through data collection operations. This can be accomplished via a query structure that uses devices and other structures that allow narrowing.

  The network administrator can relate not only to queries related to a particular target device or type of target device, applications installed on the target device, services accessed in relation to the target device, but also to RF performance or target device. You can also specify the performance of other systems that are used. As indicated at stage 708 of process 700, this construction of the query structure takes into account aspects of previously constructed queries and / or data and results obtained in connection with such queries, if desired. In other cases, it may be incorporated.

  The process 700 then proceeds to step 710. At this stage, the network administrator finalizes the query, if applicable, based on the various information categories specified, as well as the structure and / or results of one or more previously constructed queries. At some point, such as when the query is finalized, the query is then sent to the reporting subsystem 208. The finished queries can be stored in memory so that they can be easily retrieved and modified when desired. In addition, stored queries may be retrieved and copied to streamline the construction of additional similar queries. In addition, such queries can be stored in a searchable database so that a network administrator can quickly determine whether a query of interest is already constructed. This arrangement also allows a streamlined construction of additional queries and data collection profiles. If the data necessary to satisfy the query has already been collected, appropriate data and / or reports are returned to the network administrator. If not, a CRT is generated by the reporting subsystem 208 and communicated to the task subsystem 204.

<VI. Typical tasks, data collection profile construction and target device selection process>
FIG. 6 shows details regarding a process 800 for generating, i.e. building, selecting a target device to receive a profile and downloading the profile to the target device based on the CTR. . At stage 802 of process 800, the CTR is received at task subsystem 204. In some cases, the construction of a data collection profile enters a stage 804 that begins in response to the occurrence of a trigger condition, i.e., receipt of trigger information for a set of trigger conditions. In this way, some or all of the data collection profiles may be automatically generated.

  In yet another implementation, a data collection profile is generated when requested by a network administrator with a finished query structure and resulting CTR submission. In order to facilitate the relatively quick construction of the data collection profile, the profile system can be configured to search for the data collection profile in a profile database that matches, for example, or substantially corresponds to the CTR. In this way, the construction of a new data collection profile can proceed fairly quickly. In addition, in some cases, the network administrator explicitly commands that a particular data collection profile is modified in a particular way so that the modified data collection profile can be entered into various target devices. Good. In this case, discretionary rights to use previously constructed data collection profiles are retrieved from the profile system and given to the network administrator instead.

  In either case, process 800 proceeds to stage 806 where CTR is provided to profile builder 206A. At step 808, profile builder 206A generates data collection instructions including rules and triggers corresponding to the CTR and / or retrieves existing data collection instructions as specified. Once the data collection instructions are generated and / or retrieved in this manner, process 800 proceeds to step 810 where profile builder 206A creates a data collection profile or modifies an existing data collection profile as specified.

  Once a data collection profile has been selected, created, or modified as specified, it is then relevant to which one or more target devices are used to collect the data specified in the data collection profile A decision must be made. Accordingly, at step 812 of process 800, profile subsystem 206 notifies task subsystem 204 of device requirements in the data collection profile. Based on this information, the task subsystem 204 accesses the task database 204B and the target device database 204C and either matches one of the data collection profiles at step 814 or otherwise corresponds to the data collection profile. Identify multiple target devices. As described above, any number of device features and conditions may be considered in the device quality qualification selection process. In one embodiment where wireless devices are used for data collection, the quality certification process determines which wireless devices are compatible with the data collection profile and which wireless devices must be disqualified from the data collection activity. Including a method of determining. In addition, to qualify or qualify wireless devices, physical properties such as device configuration, available memory, location, usable battery life, account status, usage patterns, reported services Consider various other factors and characteristics such as usage characteristics such as problems and other usage history, device interaction or physical operation of the device by the end user, installed applications and end user profile Good.

  Device information and features used in the device target selection process in step 814 can be obtained from a number of sources, including systems controlled by network operators such as HLR or VLR, or when a device first activates SQC 402. This may be caused by information provided to the target device database 204C. In addition, device information can be extracted from previous data collection activities and stored in the target device database 204C, or the device “in the target device database by using a data collection profile whose data collection instructions are focused on this data collection goal. It can be specifically collected to input "target data". An example of this is a data collection profile that is downloaded to any known device and aims to record the identity of the base station encountered by each device over a period of 7 days. At the end of the seven days, the collected data is uploaded to the metric collector 202A and communicated to the target device database 204C. The target data is then stored for later device targeting purposes. In this way, the device target selection process may access data regarding the identities of the base stations that the device encounters and the frequency with which those devices have encountered a particular base station. Thus, investigations of a particular problem can involve certain base stations, and devices that can provide the highest performance data for those base stations can be targeted to collect data for that investigation. Identify and select a set of target devices based on any metrics that are visible to SQC 402 as a result of collecting target data into target device database 204C and then later using that data for device selection criteria Can do. These include, among other things, wireless and network information, real-time device configuration including software and content downloaded by the user, user operation history including access patterns and application usage patterns, battery type or connected accessory devices, etc. May include features and configurations.

  The target process depicted in step 814 of FIG. 6 may also automatically select a particular problem or device that has previously encountered a problem. For example, if the device population experiences a fault known to the data collection management system 200, the reporting subsystem 208 can automatically generate a data collection profile to further explore the fault and report the reported fault. Experienced devices may receive new data collection profiles and be selected to participate in data collection activities.

  The device selection process may be iterative in that, for example, there are too many or too few compatible devices resulting from the initial results of the target device search. In this case, the selection criteria can be incrementally modified to produce an appropriate target device population. In the aforementioned encountered base station example, the initial selection criteria may limit the selected devices to those that have encountered a particular base station in the previous week. However, if this yields more devices than are required for data collection, another iteration of the selection process specifies that devices that have encountered the base station in the last two days should be selected for data collection activities. Furthermore, a desired number of devices can be generated. In practice, this iteration is usually done automatically by the task subsystem in the process of selecting a device.

  Another source of data for the stage 814 device target selection process may be device user account information that is typically stored as part of the network operator billing issuance provisioning system. This information identifies the external characteristics of the end user and the financial relationship with the end user's network operator. This information may include business partnerships, billing issuance plans and styles, customer care history and events, and user demographic information. An example of how this information is used is a query that monitors the performance of a device associated with certain expensive corporate customers. By tracking certain performance parameters, such as interrupted calls versus successful calls, for a particular corporate alliance device, network operators can achieve a certain level of performance against a refund of a certain percentage of the monthly fee. Guarantee can be provided. In markets where perceived quality is an important factor for network operators' service choices, such programs can offer significant competitive advantages.

  In addition, if the data collection profile specifies that data is collected for wireless calls placed in the San Francisco Bay Area, the target device database 204C indicates that the device is active or recently active in the San Francisco Bay Area. Unless otherwise indicated, task subsystem 204 probably does not select the target device whose account information indicates that the device resides in New York. In other cases, the task subsystem 204 accesses a profile established by the profile system, and the task subsystem 204 makes a decision regarding the target device to which the data collection profile is sent. The result of this determination is then sent to the profile builder 206A so that device compatibility information is included in the data collection profile.

  After the target device is identified, the process 800 proceeds to step 816 where a new or updated profile is sent to the target device (s) identified as specified. Profile transfer is “pushing” the data collection profile to the target device, prompting to retrieve the data collection profile, sending a message such as SMS to the target device, and the metric It can occur in various ways, including creating a data collection profile for download the next time the target device contacts SQP 201, such as when uploading a package. Such profile transmission to the SQC 402 residing on the target device (s) can be short message service (“SMS”), hypertext transport protocol (“HTTP”), hypertext transport protocol secure (“HTTPS”), wireless application protocol (“WAP”) push, IP-based radio (IOTA) protocol, OMA / DM, or other protocols known or may be developed in the future. This may be accomplished using any of the port mechanisms and standards.

  The process 800 then proceeds to step 818 where the data collection profile is stored on the target device. Once received by the target device, the collection profile is processed by the SQC 402. In some cases, the data collection profile is stored as received, integrated with previously received data collection profile (s), or previously received data collection profile (s) In some cases. Factors that affect how the data collection profile is processed by the SQC 402 are the suitability of the device for the data collection requirements defined in the data collection profile, the relativeness of the data collection profile and any previously received profile. Including, but not limited to, priority and explicit processing rules set forth in the data collection profile. When SQC 402 processes a new profile, when a data collection activity occurs that is slightly different from that specified in the data collection profile (eg, if the device voluntarily selects from among the data collection activities), SQC 402 Details of how and why the data collection activities are different may be communicated back to SQP 201.

  The data collection profile can be transmitted to the target device by wireless connection or wired connection. Because the data collection profile is relatively small, the transmission of the data collection profile proceeds relatively quickly and imposes minimal processing overhead on the target device. In addition, the population of target devices can be quickly reset and the data collection profile can be downloaded quickly and easily to achieve the data collection goals. Such an iterative data collection process is particularly useful in understanding transient error conditions because of the speed with which data collection activities can be refined. Other data collection activities may contribute to longer-term trend analysis. For example, when that value is reached, a threshold may be set for performance degradation that will cause the population of wireless devices to generate and download data collection profiles. In this way, additional data collection occurs that allows additional exploration of the problem. As a result, the data collection management system is extremely flexible, and data collection operations can be quickly refined, reconfigured and redirected in response to sudden or transient network conditions. In any case, the statistical analysis performed in connection with the collected data can quickly focus on the answers or answers to the questions raised in connection with the query.

  Unlike known systems, the data collection management system does not rely on the end user of the target device to download the data collection profile or otherwise take action to enable the data collection process. Rather, as indicated above, the upload of the target device proceeds with minimal or no involvement on the end user side of the target device. Furthermore, since each target device is carefully quality certified for participation in data collection activities, the likelihood that any particular target device is not a valid candidate for the collection task is minimized. Thus, data collection as specified in connection with the data collection management system 200 is performed quickly and easily by the target device. Further, the flexibility and speed with which the data collection management system 200 operates is further enhanced because the data collection profile is typically automatically generated in response to the occurrence of certain network conditions.

<VII. Data collection and processing>
The data collection function of the data collection management system 200 is generated by the software or hardware of the wireless device 400 during its operation and is based on one or more metrics that indicate device and network performance. In some cases, during the device manufacturing process, software commands are integrated into the operating system or other device software that causes a metric to be created. Once the metrics are created, they are converted into data structures that are used to invoke SQC 402. Since the generated metric is presented to the SQC 402, it is based on its managed data collection profile (s) whether the default metric is of interest when it is generated If it is of interest, it may store the metric in a buffer that it can later use to place in the metric package. The process of storing metrics in the buffer is called a collection metric. However, in some cases, the metrics that are generated are not directly buffered but collected directly in the metrics package. Otherwise, metric generation and buffering can occur continuously as part of the basic operation of the wireless device 400. This control of data collection logic is further described below in FIG.

  Another mechanism for generating a metric at the wireless device 400 includes integrating application software that is embedded in or downloaded to the wireless device and software for creating the metric. This is in contrast to integrating the metric generation command during the device manufacturing process, for example with the operating system software of the wireless device. During application software development, an application programming interface (API) is used that allows the application developer to call the SQC 402 with metrics generated by the application.

  In one embodiment of the present invention, when the SQC 402 is presented with the generated metrics, the SQC only interprets the metrics in certain fields of the metrics data structure, and the remaining metrics are in accordance with the present invention. Processed by known techniques such as markup languages or variable length structures without additional processing. The API reflects the in-memory data structure model generated from the XML description shared between the SQP operator and the application developer via external means. Common fields of the metric data structure include a unique identifier field and a size field (assigned by the SQP operator during the XML exchange process) so that new metrics can be created by new applications without modification to the SQC. To. Rules in the data collection profile direct the assignment of metrics to buffers and link triggers for metrics generated by matching identifiers in a common aspect of the metrics data structure. A data collection profile can be implemented that defines triggers and buffers for data collection rules, metric requirements that occur after SQC generation and implementation.

  The extensible metric model is for third party applications integrated with wireless communication devices (such as browsers and / or messaging applications) and for applications downloaded into virtual environments such as BREW or Java virtual machines. realizable. The virtual environment, as described above, implements a programming interface for calls by applications developed for SQC-enabled platforms and functions. Each time the interface is invoked, the metric data structure is passed to the SQC as a metric.

  A virtual machine as described above provides another important point for the generation of metrics. Thus, by instructing the virtual machine specifically to generate metrics, all applications that run on the virtual machine will have some degree of metric generation, regardless of whether they utilize the programming interface as described above. Instructed. In the exemplary embodiment, the following aspects are instructed to generate a metric, but virtually any aspect of wireless device 400 may generate a metric. Metric generation includes all error display routines so that errors displayed to the user can be captured by the data collection management system for later analysis. Exceptions (software that violates the fundamental principles of target device operation) and task timers (timer that guarantees that no single function takes up all processing resources and detects infinite loop type errors) A metric can be generated that indicates the state of the application when it occurs. Once captured and uploaded, metrics allow application developers to find errors in their software that occur in the file and greatly accelerate problem diagnosis. User interface routines that include screen refreshes and user inputs (button pressed or other direct inputs) can generate metrics using user inputs including timestamps. Once uploaded, user interface metrics understand how the user interacts with the application or device, help optimize the application design, and understand error or confusion points Can be used for. These techniques can also be applied to the entire wireless device 400 where the primary operating system is commanded instead of the virtual machine.

  FIG. 7 shows details regarding a process 900 performed by one or more target devices in connection with one or more data collection profiles. Process 900 begins with the generation of a metric as part of target device operation. In stage 902 of process 900, data collection begins as specified in the data collection profile. As mentioned above, such data collection begins in response to an initiating trigger that is simple enough to arrive at a date or a geographic location, or it encounters a date, a geographic location, a specific error, etc. Or a combination of any of the above conditions associated with the use or non-use of a particular service or application. Once the profile is activated, data collection begins and can proceed continuously, which starts, stops and resumes data sampling based on data collection rules and triggers set in the data collection profile be able to. As described above, during the data collection process, metrics can be stored in a metrics buffer or temporary memory, which may or may not be used. The metric buffer is a circular buffer that stores samples of N data available for use in creating a metric package when needed. The use of buffered data allows, for example, display of metrics that are generated before a specific error condition occurs. This history display, along with other metrics surrounding the error condition, can be extremely important in analyzing the problem. Thus, in response to a trigger, the buffered data can be captured in the metric package along with any specified data that arrives after trigger activation. In other cases, data can be collected without first being buffered and written directly to the metric package.

  In any case, the target device starts collecting data as specified in the data collection profile. For example, each of the target devices initially matches the data collection profile with the various characteristics of the hardware and software contained in the target device and the various services that the target device is authorized to use. The received data collection profile may be examined to check and verify whether it is present. In this way, target devices that cannot collect the data specified in the data collection profile or otherwise are not suitable to help determine the response to the query are voluntarily selected from the data collection process. You can. Instead, rather than choosing voluntarily from data collection, in response to compatibility-related instructions contained in the data collection profile, the device collects only a subset of the requested data, A report may be returned to the metric collector 202A along with the basis of the subset collection. In this way, the network administrator is assured that the data finally collected and sent to the metric collector 202a matches only the initially proposed query and comprises only corresponding data. .

  In any case, once the data is collected, process 900 proceeds to stage 903 where the collected data is converted into a metric package by SQC 402 and stored in the metric archive of SQC 402. Since device resources are inherently constrained, the SQC 402 may utilize a method for managing the priority order of metric packages stored in the device. For example, each time a metric package is added to the SQC 402 metric archive, the SQC 402 calculates the total non-volatile memory used. If the total non-volatile memory required to store a new metric package exceeds the amount of memory allocated to the metric archive, the SQC 402 re-evaluates the nature of the data in each stored metric package. To do. In certain cases, under certain conditions, one or more metric packages may be uploaded to the metric collector 202A to make room for a new metric package on the device. Under certain conditions, the SQC 402 may review an stored metric package and utilize an algorithm to select a metric package for deletion. The algorithm may consider criteria including the data storage allocation specified in the data collection profile, the nature and priority of the data, the timeliness of the data, the size of the new package being stored, and the size of the package being deleted. The selected metric package is then deleted from the metric archive on the device when there is sufficient memory to store the new metric package.

  In stage 904 of process 900, a metric package is sent to metric collector 202A when specified in one or more application data collection profiles. As described above, transmission of the metric package may be a one-time event, or may be performed periodically or recurring but irregularly. It may also be sent by the network administrator of the device as a result of an “on-demand” command sent by the system to the device or as a result of resource management by SQC.

  In one embodiment, the data collection profile may specify that the upload of a metric package occurs only when a specific event or condition occurs. When the conditions for transmission are met, the collected data is sent to the metric collector 202A. In some cases, when a metric package is received, notifications such as email alerts or system alerts may be generated, which may initiate reporting activities and / or additional monitoring or data collection activities . In other cases, once the condition is met, upload to avoid the current state of the end user's device overwhelming the device or network capacity or otherwise preventing the end user's normal activity It may be further qualified before the activity begins. If the condition is met, but the end user engages in activities on the device that are disturbed by the transmission of the metric package, the transmission can be postponed until later and uploaded according to a secondary set of rules. A metric package is forced to be deleted, for example if the data is time-dependent and the device cannot be used for long-term network connectivity, or SQC resource management practices make space for the priority metric package If you do, you can discard it completely.

  Finally, as shown in FIG. 7, process 900 determines whether a new or modified data collection profile is available for download by one or more target wireless devices 400. The process proceeds to step 906 in which is determined. Although this stage of the process is depicted in FIG. 7 as part of a continuous operation, the determination of whether a new or modified profile is available for download is not associated with other stages of the data collection process, Such new or modified data collection profiles are created and selected at any time in response to the generation of new queries or the occurrence of various other factors in response to changing network conditions, the sudden appearance of specific network conditions May be provided for download to a connected device. If a new or modified data collection profile is available, at step 908, the available new or modified data collection profile is downloaded to the target wireless device. Otherwise, in step 910, it is determined whether the existing data collection profile remains active according to the rules specified in the data collection profile. If it remains active, it resumes data collection activity at step 902. In many cases, the existing data collection profile may remain on the device for any amount of time, whether or not it changes, and many examples of data collection and upload may be performed. However, if it is determined that the existing data collection profile is no longer supported at stage 910, the process proceeds to stage 912 where the existing data collection profile becomes inactive, and in some cases it automatically Deleted from.

  FIG. 8 shows details regarding aspects of a process 1000 for processing and managing metric packages. In step 1002 of process 1000, metric package data is accessed from reporting subsystem 208. Metric packages can be indexed or otherwise associated with data collection profiles and with other metric packages so that all relevant data is accessible from the reporting system.

  Next, the process 1000 proceeds to step 1004 where the metric package data is processed. Such processing may be virtually any type of manipulation or analysis of metric package data including statistical analysis. In some cases, the results of the analysis of the metric package data are used to build or refine existing or future queries, data collection profiles, and / or to improve the target accuracy of the task subsystem 204 used. After collection and either before or in parallel with the report generation operation described below, the data may be expanded using automatic calculations and additional sources of information. Also, the plug-in server architecture allows other applications to have a direct feed to the metrics package.

  The process 1000 then proceeds to step 1006 where various reports are generated based on the processing of the metric package. In a preferred embodiment of the present invention, such reports can be input with pre-defined and analyzed data in conjunction with a data collection and analysis package as previously described herein. Alternatively, sometimes custom reports may be created on demand by the network administrator. Finally, the process 1000 stores data and report information in the reporting database 208C for additional use and reference and is exported to a third party reporting system, or otherwise queried, augmented, and / or data profile modified. And go to step 1008 where it is enabled for definition.

<VIII. Setting and using trigger>
As described above, the data collection process performed in connection with the network and associated target devices may include the use of certain events, network conditions, trends, data points and threshold usage, data collection profile generation, Needed as a basis for the performance of other operations such as download or refinement. In addition, embodiments of the present invention provide for the setting and use of various types of triggers in connection with data collection profiles.

  In the exemplary embodiment, the trigger is one or more conditions set during the integration process with wireless device 400. The trigger allows SQC 402 to be invoked for additional selection criteria processing. Each trigger is associated with a specific identifier and sets a specific set of states that are visible to the wireless device 400 software. When the SQC functionality is invoked, the trigger identifier is compared to the trigger included in the download profile to determine the appropriate sequence of actions. In this way, including a trigger in a particular data collection profile effectively sets the condition that the data collection profile is to take the specified action.

  In an exemplary embodiment, triggers may be included in the data collection instructions of the data collection profile, and when they are included, the SQC will take appropriate data collection activities when the associated trigger condition is invoked by the wireless device 400. Start, abort, and end. Alternative embodiments of trigger and data collection functionality are described below in Section IX. A trigger call that matches the initiating trigger initiates the data collection activity. A termination trigger match ends the data collection activity and then a metric package is created for upload. The abort trigger stops the data collection activity and no metric packet is created. In the example used above, the start of an immediate connection communication causes the SQC to be matched with a trigger on the downloaded profile and called the “instant connection communication start” trigger event that causes the data collection activity to be initiated at the originating device. . When the originating device receives a 408 error, another trigger is activated and the SQC matches the event to the ending trigger in the profile, stops data collection, causes the metric package to be created and uploaded. As can be seen, including a trigger in a profile effectively selects a condition under which a particular action associated with that profile is performed. The trigger is not strictly within the profile, rather it ties a specific profile action (start, stop, abort) to a specific event on the device.

  In another example, the trigger event may be the start of a voice call. In this case, the start of the voice call is an initiator trigger call that causes the SQC to execute the data collection rules contained in the data collection profile that resides on the device involved in the voice call. In this example, the data collection profile may combine the action of terminating the voice call with the end of the data collection activity and may also cause the creation of a metric package for upload to the metric collector. This data collection activity allows the recording of metric activity specified for the call and the generation of a metric package associated with each call that occurs on the device.

  Using triggers provides fine-tuned data collection activities. By linking profile actions to events set on the device, it is possible to effectively measure virtually any event that follows in the previous measurement and time. As a result, important trend information and other indicators embodied in the collected data can be quickly identified and utilized, thereby enhancing the quality and practicality of the collected data. Any of a variety of triggers may be set and used. Accordingly, the scope of the present invention should not be construed as limited to any particular type, number, combination, implementation or use of triggers. Details regarding some triggers are given below.

  One basic form of such a trigger requires a fixed occurrence setting or designation that causes the SQC to be called. As used herein, the term “occurrence” extends to the occurrence or non-occurrence of specified things that set a particular identity for the trigger. When the SQC determines that a set occurrence has occurred or could not occur within a set time frame (eg, by being called by a wireless device with a specified trigger identity), one or Multiple data collection activities may be performed by the SQC. In addition, the SQC can set certain metrics in certain conditions to cause a specified trigger identity call. At the same time, any metric in any state, or any sequence of metrics, can be set to trigger a particular trigger identity call. For example, a threshold trigger can be invoked by a monitored parameter that exceeds or falls below a certain threshold for a set period of time and can range from zero to an arbitrarily large number. One special example is a threshold trigger that fires when the frame error rate metric exceeds a predetermined value.

  The trigger identity can be included in the data collection profile and causes the associated action to occur. For example, to set up a data collection profile that yields a metric package that contains information about a suspended voice call, but does not collect information about other types of calls (successful or blocked): An associated trigger may be used.

Start Trigger: Voice Call Start End Trigger: Stop Voice Call End Abort Trigger: Voice Call End In this example, the data collected for this trigger is the goal of the collection activity, so the end trigger stops data capture and measures Create a reference package. The abort trigger indicates to the SQC that data collection must be terminated, but the metric package should not be created because the data associated with this activity is not the focus of the collection goal.

  An associated type of trigger is a “domino” trigger that is activated when a command is sent from the SQP 202 to the SQC residing on the network node. In the example described above where 408 errors are being monitored, the terminating device in the example is not aware of the communication session attempt by the originating device, so in this case, data collection activity is not triggered by the terminating device in this case. . A “domino” trigger is used to ensure that important data related to the condition surrounding the terminating device during the communication attempt is captured. As previously described in the example for the 408 TIMEOUT message, a Domino trigger occurs when the SQP 201 causes a command in the form of SMS or other message type to be sent to the SQC 402 at the end device that activates the start trigger, An “on-demand” data collection activity begins and / or a metric package is created for upload. As part of the “on-demand” data collection activity, metrics may be selected from existing ones in the terminating device's buffer and other metrics may be collected to complete the metrics package. In particular, in this example, the data collection profile previously activated on the terminating device causes the collection of data that is valid for the upcoming “on-demand” data collection activity, and the Domino trigger is terminated before the arrival of the Domino trigger. Enable the collection of data that was already collected on the other device. The metrics package may be uploaded immediately upon creation or at some other time specified in the data collection profile.

  FIG. 9 shows details regarding one example of a process indicated at 1100 for data collection. Process 1100 begins at step 1102, where SQC 402 on one or more wireless devices receives a data collection profile that specifies a trigger. The trigger specified in the data collection profile may be configured to instruct the SCQ 402 to perform data collection activities upon invocation of the specified trigger. In the depicted example, the SQC 402 on the wireless device is involved in the monitoring process at step 1104 to identify the activation of the trigger specified in the data collection profile. Data collection begins when the initiator trigger is activated. Once data collection has begun, some or all of the monitored data may be collected in response to instructions included in the data collection profile, or it may be ignored.

  In the described process, after the initiating trigger initiates data collection, data collection proceeds according to the instructions included in the data collection profile until an abort or end trigger is detected at decision point 1106. For example, the data collection instruction can specify sampling of data at periodic intervals for a specified period. Data collection stops when an abort, i.e. termination trigger, is activated.

  Triggers can be set that take advantage of any aspect of the wireless device status in its implementation. This mechanism makes it possible to realize complicated dependencies such as collecting data only when a certain service is in a certain state. In the above example, collecting data only for IP data calls placed by an instant connection application can be implemented as a specific trigger definition that activates only under a specific set of constrained conditions.

  Finally, FIG. 10 shows details regarding a process 1300 that requires the setup and use of domino-type triggers. In the illustrated case, process 1300 is performed from a communications network perspective. This arrangement provides the ability for network personnel to see and evaluate the entire transaction between wireless devices in a communication network, rather than focusing only on some of the transactions associated with a particular wireless device.

  Process 1300 begins at step 1302 where one or more metric packages are received from one or more wireless devices or other network nodes. Such data generally relates to one or more parameters of the communication network and / or associated wireless device. The process 1300 then proceeds to another “non-reporting” such as an ending device whose data received in the metric package (s) is unaware that the originating device has attempted to communicate with it. Proceed to the decision point at step 1304 where a decision is made regarding whether to involve the network node. If no other network node is involved, the process 1300 proceeds to stage 1306 and additional analysis may be performed.

  On the other hand, if the received data is related to “non-reporting” network nodes and may contribute data that is important to the analysis of the transaction, the process 1300 proceeds to stage 1308 instead. At this stage, the SQP 201 sends a command to the “non-reporting” related network node. In this case, the command sent from SQP 201 is interpreted as a trigger that allows the relevant non-reporting network node to collect relevant data and send it in the metric package.

  Thus, if wireless device “A” sends a page to wireless device “B” but does not receive a response, wireless device “A” will generate an error that activates a trigger that causes the metric package to be uploaded to the metric collector. Receive. However, since the wireless device “B” may not be aware that the page has been transmitted, the wireless device “B” does not recognize any errors or problems that have occurred and is associated with an error metric package. Will not be triggered to upload. Thus, consideration of the entire transaction, that is, the transaction attempted between devices “A” and “B”, is necessary to achieve a more complete understanding and recognition of the problem.

  In step 1308, the SQP 201 causes the wireless device “B” to send a command. At step 1310, the command also initiates generating data collection activities such as selection of metrics from device “B” buffer, collection of additional metrics needed, and creation and upload of metrics packages. Interpreted as a side trigger. The metric package from wireless device “B” may then be received by the metric collector at step 1312 and additional analysis performed. At this stage, metrics packages uploaded by such other network devices in relation to the data collection profile are received and analyzed. Process 1300 may be performed once or repeatedly as part of an iterative analysis resolution process.

<IX. Device profile to be scripted>
In another embodiment of the invention, a scripted device profile is used rather than a data collection profile to direct the processing of metrics generated by the target device. In this embodiment, instead of a limited set of fixed functions, SQC is “scripted” in a fully functional programming language to script metrics, triggers, and arbitrary algorithms for measurement processing. It can be set by a customized device profile. It adds this additional flexibility to allow even more of the metrics collection, processing, and management details to be dynamically set by a scripted device profile that is downloaded rather than coded into the SQC. However, it greatly increases the number of scenarios that the data collection and management system of the present invention can address.

  In this embodiment, and in contrast to the previous embodiment, when a metric is generated as a result of an activity or transaction performed on the target device as described above, the metric processing collects the metric. Rather than being limited to doing, not collecting metrics, or invoking a specified trigger identity, the metric processing is much more flexible. More specifically, this embodiment of the present invention allows complex processing for metric collection to be fully configured by a downloaded scripted device profile. Metric parameters and triggered functionality that replace the data collection instructions (rules and triggers) described in the above embodiments are set in the scripted device profile and govern the metrics processing. As described above, the quality of service client (SQC) is responsible for receiving and otherwise managing profiles.

  Referring to FIG. 11, once a metric is generated at the wireless device 400, the metric specified in the scripted device profile 404 is determined by the metric processing element 404A to determine the appropriate processing of the metric. SQC 403 is called to consult with parameter 404B. In many cases, since no information is available in the metric parameters 404A for the metric, the metric is ignored, device operation continues, and the metric continues to be generated as part of the normal operation of the wireless device 400.

  However, if the metric parameter 404A specifies metric processing, then additional processing is performed. The additional processing can take one of two forms: In the first form, called filtering 404C, the metric is compared to the parameter specified in the metric parameter 404B. The parameter specifies both the type of comparison and the value required for the comparison. As a result of this comparison, a binary decision is made as to whether the metric should be stored in one or more buffers 406 or ignored. The special comparisons that can be used are defined as part of the SQC implementation, and virtually any type of comparison can be performed. The comparison specified in the metric parameter 404B may include a simple binary decision such as “always collect this metric” or “collect if this metric satisfies this binary or numeric comparison”. Other comparisons may include comparing the metric with another metric or with another aspect of the wireless device 400 status. Even more complex comparisons that require multiple criteria and “pre-processing” to determine whether a particular result has been achieved may be specified in the metric parameters. In general, the comparison specified in the metric parameter 404B may be virtually any software functionality, but a relatively small number of generalities set when the SQC is implemented. May be limited to actions used.

  Since the criteria specified in the scripted device profile 404 are met, the metrics are collected in one or more buffers 406. As in the previous embodiment, a common feature is for a specific number of metrics in a simple rotating buffer model where a new metric may overwrite the oldest stored metric when the buffer is full. Is a fixed size buffer. Another strategy is to call triggered functionality 404D (described further below) when the buffer is full and to have metrics collected under different conditions collected under certain conditions. May include a priority setting buffer to be overwritten. Virtually any buffer management model can be implemented in connection with the present invention and can be specified as part of the metric parameter 404B. In practice, a limited set of fixed functions may be implemented and additional algorithms may be implemented as triggered functionality 404D. In some cases, the metric may be included in the metric package without first being buffered.

  The second form of metric processing also takes advantage of the comparison functionality described, but is triggered here instead of collecting metrics when a positive result comes out as set for filtering. A previously downloaded script, called the functionality 404D, is called to command additional metric processing.

  The triggered functionality 404D departs significantly from the previous embodiment of triggering and data collection profile functionality. The triggered functionality 404D is a full-featured computer language. That is, it is software that directs the actions of the wireless device 400 processor, so software functionality can be implemented in the wireless device 400 in connection with data collection, management, and analysis. As a result of being invoked by the metric comparison, the triggered functionality 404D takes control of the metric processing. As described above, the triggered functionality can perform virtually any software function on the wireless device 400, which includes caller metrics, all of the buffers in which metrics are stored, and device-related. Access to additional status information, metric packages and upload functionality. In the embodiment utilizing the data collection profile and the trigger, there is a limited set of actions that can be executed in response to the trigger call, and each trigger has a configured function executed in response to its activation. It was. In this embodiment, by using a scripted device profile and triggered functionality, the result of the metric comparison as set forth above can act as a trigger and the trigger is activated. Allows complete flexibility ("triggered functionality") for subsequent actions to be performed after More specifically, triggered functionality 404D can cause the generation of new metrics, which can modify metric parameters 404B to change the criteria and processing instructions included in scripted device profile 404. . The triggered functionality 404D can further copy, modify and otherwise process the content of the process buffer 406 and convert the resulting data to a metric package 408 for later upload. It's okay. The triggered functionality can perform an upload, causing a new scripted device profile or other software code to be downloaded to the wireless device 400 and executed there.

  The triggered functionality 404D is included in the scripted device profile 404 of this embodiment. Any of a number of known models for enabling a software downloadable model on wireless device 400 will prove effective as a management model for triggered functionality 404D. These models include a virtual machine (e.g., realized by Java) and an actual machine language (e.g., realized by a BREW environment) running in a given environment. In addition, many commercial packages are available today to perform “wireless” firmware updates, effectively downloading any software and adding it to existing software on the wireless device 400, Allows subsequent calls as triggered functions.

  As a complete computer language, triggered functionality 404D can implement virtually any software functionality possible on the device. In this embodiment, triggered functionality 404D is implemented in native device code so that it has access to all states and functionality of wireless device 400. However, certain implementations may have limited access due to device state and functionality and still perform substantially the same role.

  In addition to the triggered functionality invoked by the metric comparison, a set of special events on the wireless device 400 can be defined as the special trigger 412, and the triggered functionality 404D can be invoked as well. In particular, when the scripted device profile 404 is first downloaded to the wireless device 400, a special trigger 412 is used to perform profile activation (possibly to set trigger and metric parameters). Activate triggered functionality 404D. The special trigger 412 is activated when the wireless device 400 is switched on or off and reset, or when a processor exception occurs. These special triggers may be implemented to cause triggered functionality to be invoked by actions such as application calls, user actions, or other activities that do not naturally cause the generation of metrics. Special trigger 412 may be invoked by virtually any other event at the device that may prove interesting for later analysis.

  Because of the general nature of the capabilities of triggered functionality 404D, many other aspects of data collection functionality (such as abort triggers or termination triggers) become separate triggered functions rather than special functions. As mentioned above, the scripted embodiment of the SQC described in this section allows full flexibility of what action to take, if any, when one of these events occurs, Significantly increases the number of measurements of interest that can be effectively performed by the data collection and monitoring system of the present invention.

<X. Telecommunications Equipment and Computer Environment>
As shown herein, embodiments of the invention include wireless and wired telephones, other wireless communication devices, or special purpose or general purpose computers adapted to have comparable telecommunications capabilities, It may be implemented in connection with a special purpose or general purpose telecommunication device. Embodiments within the scope of the present invention also include computer-readable media for carrying or having computer-executable instructions or electronic content structures stored thereon, these terms being used with telecommunication devices. Is defined to cover all such media or instructions.

  By way of example, such computer readable media is desired in the form of RAM, ROM, flash memory, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or computer-executable instructions or electronic content structure. Other mediums that can be used to carry or store program code, and that can be accessed by a general purpose or special purpose computer or other computing device.

  When information is transferred or provided to a computer or computing device over a network or another communication connection (either embedded, wireless, or a combination of embedded and wireless), the computer or computing device The device displays the connection appropriately as a computer readable medium. Accordingly, such a connection is properly named a computer readable medium. Combinations of the above should also be included within the scope of computer-readable media. Computer-executable instructions comprise, for example, instructions and content which cause a general purpose computer, special purpose computer, special purpose processing device, or computing device to perform a certain function or group of functions.

  Aspects of the invention are described in the general form of computer-executable instructions, such as program modules, being executed by computers in network environments. Generally, program modules include routines, programs, objects, components, and content structures that perform particular tasks or implement particular abstract content types. Computer-executable instructions, associated context structures, and program modules represent examples of program code for executing aspects of the methods described herein.

  The above embodiments should be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

1 shows an example of a wireless communication network that implements a data collection system. 1 is a schematic diagram illustrating a data collection management system suitable for use in connection with communications network operations. FIG. It is the schematic which shows the various consideration regarding the query performed by a data collection management system. 1 is a schematic diagram of a data collection profile used by a data collection management system of one embodiment. FIG. 2 is a flow diagram illustrating an exemplary process for formulating a query and rendering the query. 2 is a flow diagram illustrating an exemplary process for creating a data collection profile. 2 is a flow diagram illustrating an exemplary process for collecting data in connection with a data collection profile. 2 is a flow diagram illustrating an exemplary process for using data collected in conjunction with a data collection profile. 2 is a flow diagram illustrating an exemplary process for triggering activation of a data collection activity. A typical process for invoking a Domino trigger is shown.

Claims (49)

A method for collecting data associated with a communication network including a plurality of devices, comprising:
Obtaining a data collection profile that defines the data to be collected and the conditions under which the data is collected;
Identifying a target subset of the plurality of devices in the communication network;
Distributing the data collection profile to a data collection agent of each device in the target subset, wherein the data collection profile is configured to perform data collection upon receipt of the data collection profile; Distributing,
Receiving the data generated in response to the data collection profile as a metric package from at least a portion of the target subset, the identifier associating a specified transaction on the communication network with the metric package Receiving further including receiving ,
Receiving the metric package from a plurality of nodes of the communication network;
Correlating a plurality of metric packages received from different nodes of the communication network based on the identifier of the metric package;
Equipped with a,
A method allowing a series of events associated with the transaction in the communication network to be analyzed by the plurality of metric packages .   The method of claim 1, wherein identifying the target subset comprises accessing a target device database to select a device having characteristics corresponding to the data collection profile.   The method of claim 2, wherein features corresponding to the data collection profile are associated with usage history of individual devices.   The method of claim 2, wherein the characteristics are determined utilizing data previously collected from devices in the communication network using the data collection profile. Features corresponding to the data collection profile are:
Environmental factors encountered by the device,
The specific software configuration of the device,
The method of claim 2, wherein the method is associated with at least one of the following. The features correspond to the specific software configuration;
The software configuration is
With downloaded software,
Data entered by a user of the device;
User priority,
Downloaded ringtones,
6. The method of claim 5, comprising at least one of other downloaded data.   The method of claim 2, wherein characteristics corresponding to the data collection profile are related to physical properties of the device. Features corresponding to the data collection profile are:
Area code,
With the area code,
Billing address,
Other data identifying the expected location of the device;
The method of claim 2, wherein the method is associated with at least one of the following. Features corresponding to the data collection profile are:
The service to which the user subscribes;
The function of the device;
The method of claim 2, wherein the method is associated with at least one of the following. Features corresponding to the data collection profile are:
The device's ability to collect the metrics specified in the profile;
The device's ability to respond to a trigger specified in the profile;
A RAM storage area on the device that can be used to collect the metrics;
Fixed storage on the device available for storing metric packages;
Processing power of the device;
The general ability of the device to perform the specified data collection profile;
The method of claim 2, wherein the method is associated with at least one of the following.   The method of claim 1, wherein the data is generated in response to detection of a trigger condition by an individual device of the target subset.   The method of claim 1, wherein the data relates to a service of the communication network provided to a user through the device.   The method of claim 1, wherein the data relates to software stack features of a device that communicates with various network layers associated with the communication network.   The method of claim 1, wherein the data collection profile defines a filter for the data by identifying the data to be collected.   The method of claim 1, further comprising using the received data to narrow the data collection profile. The data is
An application layer of the communication network;
A transport layer of the communication network;
A physical layer of the communication network;
The method of claim 1, wherein the method is in communication with at least one of the devices and associated with a software stack of a target device. The data is
Usage history of the target device,
Interaction with or physical manipulation of the device by an end user;
Hardware associated with the target device;
Software associated with the target device;
The method of claim 1, wherein the method is associated with at least one of the following:   The method of claim 1, wherein identifying a target subset of the plurality of devices in the communication network is repeatedly performed such that the number of target devices falls within a selected range. Distributing the data collection profile to the target subset includes
Transferring the data collection profile to devices in the target subset;
Sending a message to a device in the target subset to instruct the device to retrieve the data collection profile from a server in the communication network;
Preparing the data collection profile to be acquired by the device the next time it contacts the server without sending a message to the devices in the target subset;
The method of claim 1, comprising one of:   The method of claim 1, wherein identifying the target subset comprises selecting devices having characteristics that are specified by the data collection profile or that correspond to device characteristics indicated by the data collection profile. Method.   Receiving data from at least a portion of the target subset that is acquired from a collection buffer in response to a trigger defined by the data collection profile and stored in the collection buffer prior to the trigger The method of claim 1, further comprising:   The method of claim 1, wherein identifying a target subset of the plurality of devices includes reviewing any prior collection tasks with which the devices can be associated. The target device is
Minimize the impact on the communication network,
Compensate for previous failed attempts to perform the function,
Respond to the battery status of the device,
Minimize the negative impact on the user experience of the user of the device, or
The method of claim 1, wherein local adjustments of the operation of the target device can be made that are performed to compensate for the sensed load of the communication network. The local adjustment is performed to compensate for previous failed attempts to perform the function;
The function includes reporting of a metric package;
24. The method of claim 23 , wherein the local adjustment includes reporting a first available metric package after a previous report failure. The data received from at least a portion of the target subset is:
The method of claim 1, wherein the metric package archive on the devices in the target subset has been archived locally as a metric package and reported to a network server. 26. The method of claim 25 , wherein the archived data is reported in response to an event defined in the data collection profile. 27. The method of claim 26 , wherein the event is a reporting time selected by a device within a certain number of times defined in the data collection profile. The selection of the reporting time by the device is based on a unique number associated with the device;
The unique number is
An electronic serial number associated with the device;
A mobile phone identifier associated with the device;
A phone number associated with the device;
28. The method of claim 27 , wherein the method is one of: 27. The method of claim 26 , wherein the event is a metric that matches a specified set of conditions. The report is delayed by a measurable condition from the reporting device;
The measurable conditions are:
The battery status of the reporting device;
Network connectivity associated with the reporting device;
Measurable conditions defined in the data collection profile;
27. The method of claim 26 , comprising at least one of: 26. The method of claim 25 , wherein the metric package archive has a maximum size defined in the data collection profile. The archived data was reported in response to an event defined in the data collection profile;
The event is
The metric package archive of a reporting device that reaches a certain percentage of the defined maximum size;
Said metric archive reaching a certain size;
32. The method of claim 31 , comprising at least one of: The reporting device includes logic that defines how to respond to the metric package archive that is full using a priority scheme;
The priority scheme is:
Determine whether the total non-volatile memory required to store the new metric package exceeds the amount of memory allocated to the metric archive, and if so, one or more of the lower priority 32. The method of claim 31 , wherein the archived metric package is replaced with a higher priority metric package.   The method of claim 1, wherein the data collection profile allows at least one device in the target subset to initiate data collection without a requested action from an external device for the at least one device.   The method of claim 1, wherein the data collection profile is configured to initiate data collection at any time after the data collection profile is received by the target subset.   The method of claim 1, wherein the data collection profile allows at least one device in the target subset to independently initiate data collection.   The data collection profile of claim 1, wherein the data collection profile is configured to allow data collection by the at least one device at any time after the data collection profile is received by the at least one device. Method.   The method of claim 1, wherein obtaining the data collection profile comprises automatically generating a data collection profile. 40. The method of claim 38 , wherein the data collection profile is generated upon the occurrence of a network event. 40. The method of claim 38 , wherein the data collection profile is generated in response to analysis of previous collected data.   The method of claim 1, wherein devices in the target subset are automatically selected from the plurality of devices.   The method of claim 1, further comprising one of inactivating, modifying, and replacing the data collection profile in one or more devices of the plurality of devices.   The method of claim 1, wherein the plurality of devices includes a plurality of devices configured for wireless communication, and at least one of the plurality of devices configured for wireless communication includes a mobile device. 44. The method of claim 43 , wherein the plurality of devices configured for wireless communication includes one or more of a computer, a phone, a PDA, a wireless device, and a data card for use in connection with the computer.   The method of claim 1, wherein one or more of the plurality of devices are configured for operations associated with one or more of telephony, messaging, browsing, and instant connect services.   The method of claim 1, wherein the data collection profile is dynamically changeable or replaceable upon occurrence of a network event or in response to the presence of a predetermined condition.   Further comprising distributing at least one additional data collection profile to one of the devices in the target subset such that two or more data collection profiles are simultaneously present on the at least one device in the target subset. Item 2. The method according to Item 1.   The method of claim 1, wherein obtaining the data collection profile comprises selecting an existing data collection profile from a data collection profile database.   The method of claim 1, wherein obtaining the data collection profile comprises generating a data collection profile.

Images