CN102457893A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN102457893A CN102457893A CN2010105273172A CN201010527317A CN102457893A CN 102457893 A CN102457893 A CN 102457893A CN 2010105273172 A CN2010105273172 A CN 2010105273172A CN 201010527317 A CN201010527317 A CN 201010527317A CN 102457893 A CN102457893 A CN 102457893A
- Authority
- CN
- China
- Prior art keywords
- service
- data
- service data
- terminal
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000010219 correlation analysis Methods 0.000 claims abstract description 21
- 238000004458 analytical method Methods 0.000 claims description 95
- 238000000034 method Methods 0.000 claims description 54
- 238000012544 monitoring process Methods 0.000 claims description 34
- 238000012098 association analyses Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 11
- 230000010354 integration Effects 0.000 claims description 10
- 238000005457 optimization Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 230000003370 grooming effect Effects 0.000 claims 1
- 230000006399 behavior Effects 0.000 description 26
- 230000008569 process Effects 0.000 description 17
- 238000012360 testing method Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 8
- 230000011664 signaling Effects 0.000 description 7
- 238000007418 data mining Methods 0.000 description 6
- 238000007781 pre-processing Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 238000005065 mining Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method and a device. The data processing method comprises the following steps: receiving a first service data reported by a terminal and a second service data reported by a service device, and then performing correlation analysis on the service according to the first service data and the second service data. According to the data processing method and device disclosed by the invention, the service data reported by the terminal and the service data reported by the service device are subjected to the correlation analysis and a more powerful data basis for optimizing a data service and analyzing a user action is supplied.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a data processing method and device.
Background
With the development of mobile data services, especially the large-scale use of third generation (3G) mobile communications, how to rapidly improve the quality of data services and meet the increasing demands of users on diversity and high reliability of data services while ensuring network construction becomes a problem that operators need to solve. For a data service, whether the subscription and unsubscription mode is convenient, whether the user experience of using the service is friendly, whether the client response of the data service is rapid and the like all influence the use of the service by the user, so that the service utilization rate is influenced.
However, due to the fact that the types of data services are various, the quality of the data services is not uniform, and the client side measured by the terminal has multiple versions due to customization and the like; therefore, it is relatively difficult to perform data service quality analysis and user behavior analysis.
In the prior art, the methods for analyzing the quality of data service and user behavior include:
(1) the current user behavior analysis system carries out mining analysis according to various demographic information, historical package, telephone rate conditions of users, telephone bills provided by various service platforms and other information by means of a background support system.
(2) The service platform performs correlation analysis on a client side and a User Interface (UI) measured at a terminal through modes such as customer research and the like.
(3) In the field of network monitoring and testing, a mode of 'test terminal + drive test software' can be used, in the mode, the test terminal is used for testing the network and the signal strength, and a background analyzes data collected by the test terminal.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
the behavior of the user measured at the terminal cannot be combined with background data, so that the analysis result is inaccurate.
Disclosure of Invention
The embodiment of the invention provides a data processing method and equipment, which are used for analyzing a service by combining data on a terminal and data on a service device.
In order to achieve the above object, an embodiment of the present invention provides a data processing method, including:
receiving first service data reported by a terminal and second service data reported by a service device; and analyzing the service according to the association of the first service data and the second service data.
An embodiment of the present invention provides a data processing apparatus, including:
the terminal parameter collection module is used for receiving first service data reported by a terminal;
the platform parameter collection module is used for receiving second service data reported by the service device;
and the association analysis module is used for associating and analyzing services according to the first service data received by the terminal parameter collection module and the second service data received by the platform parameter collection module.
An embodiment of the present invention provides a terminal, including:
the clock synchronization module is used for synchronizing the time of the terminal with the time of the service device;
the monitoring execution module is used for acquiring first service data corresponding to a service and stamping a time stamp on the first service data;
and the parameter reporting module is used for reporting the first service data acquired by the monitoring execution module.
An embodiment of the present invention provides a service apparatus, including:
the clock synchronization module is used for synchronizing the time of the service device with the time of the terminal;
the parameter acquisition module is used for acquiring second service data corresponding to the terminal service; and time stamping the second service data;
and the parameter reporting module is used for reporting the second service data acquired by the parameter acquisition module.
Compared with the prior art, the embodiment of the invention at least has the following advantages:
by performing correlation analysis on the service data reported by the terminal and the service data reported by the service device, the service on the terminal can be optimized, and a more powerful data basis is provided for data service optimization and user behavior analysis.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a system architecture diagram of a data processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a data processing method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart illustrating a process in which a terminal collects service data corresponding to a service used by a user and reports the service data to a service analysis device according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of the Aho-Corasic method using multi-pattern recognition in the multi-pattern matching method according to the embodiment of the present invention;
fig. 5 is a schematic flow chart illustrating a process of analyzing, by a service analysis device according to service data reported by a terminal and service data reported by a service device, a service on the terminal according to an embodiment of the present invention;
fig. 6 is a schematic diagram of system acquisition in an application scenario provided by the embodiment of the present invention;
fig. 7 is a device structure diagram of a data processing method according to an embodiment of the present invention;
fig. 8 is a structural diagram of a terminal according to an embodiment of the present invention;
fig. 9 is a structural diagram of a service device according to an embodiment of the present invention.
Detailed Description
In the prior art, when data service quality analysis and user behavior analysis are performed, instantaneity is poor (when a problem occurs in the use process of a data service, whether a problem occurs in a client or a problem occurs in a background server cannot be determined), and a fault report is realized in a mode of relying on manual complaint of a user, so that the situation when the fault occurs cannot be reproduced in real time, accuracy of an analysis result is reduced, problems (such as low network rate, bug of the client, slow background processing flow and the like) cannot be found comprehensively, and user behavior cannot be analyzed correctly. In view of the above problems, embodiments of the present invention provide a data processing method and device, in which a service analysis device performs association analysis by receiving information reported by a terminal and a service device and combining the information reported by the terminal and the information reported by the service device, analyzes related services by using a data mining technology, and optimizes the related services according to an analysis result, thereby achieving an objective of intelligently analyzing services and user behaviors.
The technical solutions in the present invention will be described clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the present invention provides a data processing method, which is applied to a system including a terminal, a service device, and a service analysis device, as shown in a schematic diagram of a system architecture shown in fig. 1. Wherein:
(1) a terminal: the service data required to be collected can be customized according to different service test tasks and reported to the service analysis device. The service data includes, but is not limited to, service behavior data, service exception data (exception in service use process), user behavior data (user behavior related to service), context information, and the like.
Specifically, by monitoring and recording the behavior of the user using the service on the terminal, the service data can be collected, and the collected service data is reported through a wireless network.
In the embodiment of the present invention, the functional modules located on the terminal include, but are not limited to: the system comprises a protocol analysis and multi-mode matching module, a monitoring configuration module, a parameter reporting module, a monitoring execution module, a clock synchronization module and the like. Each of the functional modules may be a component of a terminal monitoring program, and in practical application, the functional modules may be combined or further split.
a. A monitoring configuration module: namely, the configuration module of the terminal side monitoring scheme can be customized in advance or updated dynamically. The monitoring configuration module is used for determining the service data to be collected by the monitoring program and the mode of reporting the service data. In practical application, the monitoring configuration module may further determine a monitoring scheme at the terminal side by analyzing the configuration mode of the service analysis device, which is not described herein again.
The collected service data specifically includes: collecting parameters of a terminal such as service, signaling, events, user behavior types and the like; the method for reporting the service data specifically comprises the following steps: and reporting the collected service data of the terminal (for example, reporting frequency).
b. A parameter reporting module: after acquiring various parameters (service data), the terminal needs to report the service data to the service analysis device in a wireless network or a data line or other manners under specified conditions.
The specified condition is a reporting mode configured by the monitoring configuration module, and includes but is not limited to: specifying a reporting time, specifying a reporting frequency, specifying a reporting event (trigger event), etc.
c. The monitoring execution module: the monitoring and configuration module is used for collecting various parameters (service data) according to the collection mode configured by the monitoring and configuration module and stamping the time stamp for the collected service data.
Specifically, when the monitoring configuration module configures parameters such as a service, a signaling, an event, and a user behavior type of the terminal to be acquired, the monitoring execution module needs to acquire parameters such as a service, a signaling, an event, and a user behavior type of the terminal.
d. A clock synchronization module: for synchronizing the time of the terminal with the time of each service device.
Specifically, the clock synchronization module may perform synchronization by using a GPS (Global Positioning System) alignment mode; or, synchronization is performed using alignment with the same clock alignment system.
e. The protocol analysis and multi-mode matching module: the method is used for acquiring various data streams by adopting a protocol analysis-based mode, scanning the data streams at one time by adopting a multi-mode matching mode, matching the data strings of preset events and outputting matching results.
(2) An access mode is as follows: the terminal (e.g., a terminal monitoring program on the terminal) may report the service data to the service analysis apparatus via the wireless network.
Specifically, the terminal may report the collected service data to the service analysis device through the wireless network according to a preconfigured manner; or,
the terminal can report the collected service data to the service analysis device in real time (or according to a test plan) through a wireless network; or,
for the specified service abnormality, the terminal can report the collected data related to the service abnormality to the specified service device or service analysis device according to the requirement.
(3) The service device: the service analysis device is used for collecting service data of the terminal (for example, service data of the terminal, user behavior and the like) and reporting the collected service data to the service analysis device.
In the embodiment of the present invention, the functional modules located on the service device include, but are not limited to: the device comprises a parameter acquisition module, a parameter reporting module, a clock synchronization module and the like. Of course, in practical applications, the functional modules may be combined or further split.
a. And the parameter acquisition module is used for acquiring the service data of the terminal.
After the service data of the user on the service device is collected, a timestamp needs to be stamped on the corresponding service data, and the corresponding service data needs to be stored.
b. And the parameter reporting module is used for reporting the collected service data (parameters) to the service analysis device under the specified condition.
Wherein the specified conditions include, but are not limited to: specifying a reporting time, specifying a reporting frequency, specifying a reporting event (trigger event), etc.
c. And the clock synchronization module is used for synchronizing the time of the service device.
Specifically, the clock synchronization module may perform synchronization by using a GPS alignment method; or, synchronization is performed using alignment with the same clock alignment system.
(4) The business analysis device: and the system is used for comprehensively analyzing related services according to the service data reported by the terminal and the service data reported by the service device, and optimizing the related services according to the analysis result.
Specifically, according to the characteristics of the current telecommunication service development, in the embodiment of the invention, the behavior of the user using the service is monitored and recorded on the terminal, the collected service data is reported to the service analysis device through a wireless network, and the correlation analysis is carried out by combining the service data collected by the service device, so that the related services are comprehensively analyzed, and the goal of intelligently analyzing the service and the user behavior is achieved.
In the embodiment of the present invention, the functional modules located on the service analysis device include, but are not limited to: the system comprises a terminal parameter collection module, a platform parameter collection module, an association analysis module and an output module. Of course, in practical applications, the functional modules may be combined or further split.
a. A terminal parameter collection module: the terminal is used for receiving the information reported by each terminal, and the information reported by each terminal comprises the service data collected by the terminal.
b. A platform parameter collection module: the system is used for receiving the information reported by each service device, and the information reported by each service device comprises the service data collected by the service device.
c. A correlation analysis module: and the method is used for performing association and combination according to the service data reported by the terminal and the service data reported by each service device, and taking the processed information as the basis for real-time alarm, service optimization and user analysis.
d. An output module: and the output device is used for outputting the analysis result of the correlation analysis module to the output device so as to display the analysis result more intuitively. For example, to a screen display device for screen display, or to a printing device for printout, etc., in order to more intuitively display the analysis result.
It should be noted that, in the embodiment of the present invention, the service analysis device may be integrated as a functional module on the service device, and comprehensively analyze the related services for the service data reported by the terminal and the service data reported by the service device based on the service corresponding to the service device; the service analysis device may also be an individual entity, and comprehensively analyze the related services based on the service data (data of multiple services or data of an individual service) reported by the terminal and the service data (of an individual service device or multiple service devices) reported by the service device.
Based on each device in the system and the functional module in each device, as shown in fig. 2, the method provided in the embodiment of the present invention includes the following steps:
In this step, the process of acquiring, by the terminal, service data corresponding to a service used by a user and reporting the service data to the service analysis device further includes the following steps, as shown in fig. 3:
Specifically, after the terminal is started and initiates a service execution process, the terminal monitoring program will run in the background and monitor the whole service execution process. At this time, the terminal may collect various parameters (e.g., collect various parameters according to the relevant configuration of the monitoring configuration module), and timestamp the collected parameters. For the user service in the monitoring range, the terminal monitoring program may record parameters of the user service (for example, parameters specifying a signaling interaction process, time when a user key-press behavior occurs, and the like) according to the configuration of the monitoring configuration module.
In the embodiment of the present invention, the parameters to be collected include, but are not limited to:
specifying parameters of a signaling interaction process;
the time when the user's key press behavior occurs;
information on the network layer, such as CELL ID (CELL identity), LAC ID (Location Area Code), neighbor relation information, and the like;
information about the service layer, including short messages, multimedia messages, etc.;
the location information of the terminal includes GPS (Global Positioning System) geographical location information and the like.
It should be noted that the parameter to be collected is the service data that the terminal needs to collect and report to the service analysis device. For example, the information about the network layer is context information in the traffic data.
And step 302, the terminal generates report information according to the monitoring scheme. The reporting information is service data to be reported.
In the embodiment of the present invention, the report information may be generated by using a multi-mode matching algorithm based on protocol analysis, and in practical applications, the report information may also be generated by using other algorithms, which is not described in detail in the embodiment of the present invention.
In the process of monitoring the execution of the service used by the user, a large amount of data is monitored, and in the monitoring scheme (the monitoring configuration module determines that the monitoring program needs to collect information such as the service, signaling, event, user behavior type and the like of the terminal), the service data needing to be reported to the service analysis device is pointed out, so that the service data needing to be reported to the service analysis device can be screened from the large amount of monitored data by using a multi-mode matching algorithm based on protocol analysis, and the screened service data is generated into reporting information.
In summary, due to the start of the multi-pattern matching algorithm based on the protocol analysis, in the execution process of monitoring the service used by the user, the data obtained in the monitoring process recorded each time can be directly screened, so as to finally determine the service data required to be sent to the service analysis device.
Specifically, a reporting mode (which may be set by the monitoring configuration module) after the service data is collected is set at the terminal side, and according to the reporting mode, the terminal may determine whether a trigger event occurs. For example, when the specified reporting time is set, the terminal may determine whether the specified reporting time is reached; when the specified reporting event is set, the terminal can judge whether the specified reporting event occurs.
In practical applications, the triggering event includes but is not limited to: the method comprises the steps of specifying reporting event triggering (such as triggering after abnormal acquisition), specifying reporting time (such as timing triggering), specifying reporting frequency triggering, manually triggering by testers and the like.
And step 304, the terminal sends the service data to be reported to the service analysis device.
The terminal can send the service data to be reported to the service analysis device through a wireless network, a PS (packet switched domain) or an Internet.
In the embodiment of the invention, after the service data needing to be reported is sent to the service analysis device, the terminal can also clear the stored related information. Preferably, the terminal may erase the stored related information after receiving the confirmation information returned by the service analysis device.
It should be noted that, in order to ensure the accuracy of the service data, the time of the terminal may be synchronized with the time of each service device, and in this case, the synchronization may be performed by using a GPS alignment method or a method of aligning with the same clock alignment system.
In the embodiment of the invention, when the terminal is tested, a plurality of protocols and events and a plurality of protocols related to application need to be acquired and analyzed. Therefore, the collected data has a lot of interactive data at the application level except for signaling, and the data does not need to be stored. In addition, considering that there is a large amount of PS domain data in the air interface data acquired by the terminal, all the data streams to be analyzed cannot be stored at the terminal side, and the detection of the trigger event may affect the accuracy of the test parameters. In order to solve the above problem, in the embodiment of the present invention, a multi-pattern matching mode may be adopted to perform real-time protocol analysis on the acquired data.
Specifically, by using the multi-mode matching mode, various data only need to be rapidly scanned from left to right during monitoring and testing, data does not need to be stored, so that a test monitoring program can rapidly find a trigger event and rapidly search energy, and the influence on a testing task is reduced.
In addition, in the embodiment of the present invention, it is considered that the terminal has various events, for example, there are 20 abnormal events caused by the chip, and these events may be the identifier of one or several fields, and the length is very short; or the protocol analysis event of only part of the application layer can be analyzed, and the length is slightly longer; therefore, the multi-pattern matching method in the embodiment of the present invention may also use an Aho-coral method in multi-pattern recognition, as shown in the flowchart of fig. 4, including the following steps:
step 401, setting a protocol event character string.
Specifically, before using the Aho-coral method in the protocol analysis, it is necessary to extract a symbolic identification string (i.e., an event string) from the protocol corresponding to the event, and to indicate the occurrence of the event by the occurrence of the event string.
In addition, in practical application, the program at the terminal side can also preset n event character strings to be matched according to the requirement of the test.
Preprocessing generates a state function, step 402.
By preprocessing the set of event strings, 3 functions are generated: goto (transition) function, failure (failure) function and output (output) function. The transition function goto indicates the next state reached after the reading in of the character of the next data stream to be compared in the current state. The failure function failure is used to indicate the next state to which a transition should be made when the read characters do not match in a certain state. The output function output is used for outputting matched events when matching occurs in the matching process.
Through the generated state function, a multi-mode state machine can be generated, and then a multi-mode matching algorithm can be used for protocol analysis and finding a corresponding character string.
Step 403, scanning the acquired data stream.
Specifically, after 3 functions are constructed, the data stream can be scanned sequentially, and the input characters can be read one by one.
Step 404, judging whether the input function is empty, if so, going to step 403, otherwise, going to step 405.
Step 405, output matching event.
Specifically, starting from state 0, according to the current state and the input characters, and adopting the goto and failure functions to transition to the next state, when the output function of the reached state is not empty. A matching event may be output.
In the embodiment of the invention, the service device can report the collected service data to the service analysis device under the specified condition. Wherein the specified conditions include, but are not limited to: specifying a reporting time, specifying a reporting frequency, specifying a reporting event (trigger event), etc.
Specifically, after receiving the service data reported from the terminal and the service data reported by the service device, the service analysis device can comprehensively analyze the related services and optimize the related services according to the analysis result.
In this step, the service analysis device analyzes the service on the terminal according to the service data reported by the terminal and the service data reported by the service device, as shown in fig. 5, the method further includes the following steps:
Specifically, the service analysis device establishes a database according to the service data reported by the terminal and the service data collected and reported by each service device.
Specifically, the process of data integration includes, but is not limited to: data source selection, data cleaning, denoising, filling in vacant data and the like. In the embodiment of the invention, taking analysis of an unsubscribe event of a certain service as an example, data source selection refers to taking out parameters related to analysis of the unsubscribe event from a database. Some of these parameters may be incomplete, erroneous, or repeated (e.g., several unsubscribe events in a short time period, etc.), and in this case, data washing is required for the relevant parameters, and those parameters that do not meet the requirements are filtered. In addition, for some data which is too far from the normal value (for example, a traffic event with the call duration less than 1 second, etc.), the relevant data also needs to be removed as noise. Filling up the vacant data mainly aims at the problem that some information which should be in the data is missing and needs to be filled up.
It should be noted that in practical applications, common problems of data include, but are not limited to: data is incomplete (some attributes of interest lack attribute values, or contain only aggregated data); the data contains noise (contains errors or "outliers", e.g., negative delays at a certain point); data inconsistency, etc.
Specifically, the service analysis device can subdivide the well-managed data according to specific service analysis purposes and tasks and eliminate redundant data so as to ensure that the theme of the database is clear, improve the efficiency of a data mining algorithm and reduce the consumption of system resources.
Taking the unsubscribe event as an example, the parameters reported by the terminal and the service platform within 24 hours before the unsubscribe event occurs are emphatically concerned during analysis, and at the moment, the original database can be extracted from the organized partial data to form unsubscribe subject data possibly related to the event.
Specifically, the service analysis device may perform data mining according to the service data reported by the terminal and the service data reported by the service device (the related data after the processing procedure is performed).
In practical application, the business analysis apparatus may adopt an FP (frequency pattern) -Growth method to mine the data with strong association rules, where the FP-Growth method mines the frequent item sets by gradually generating a conditional pattern base and a conditional frequent pattern tree, and does not generate candidate sets.
Taking a streaming media service as an example, when the network condition is not good, the video cannot be played normally, and situations such as too long waiting time or frequent disconnection occur, and at this time, the situation that the user is unsuccessfully subscribed to the service will probably occur.
In addition, there is no difference in the service analysis apparatus, and in order to know which case will have a greater influence on the user experience, a correlation analysis method (for example, an analysis method similar to FP-Growth, etc.) may be adopted to obtain the support and reliability that when a user complaint or unsubscribe event occurs, the terminal has too long play wait or a drop, and the higher the support and reliability, the greater the influence on the user experience.
For example, for the association relationship between two events a, B, the support and the credibility can be considered, the support: probability of simultaneous occurrence of events A, B; reliability: when event A occurs, event B occurs with probability. For the streaming media service, taking the relationship of the terminal side event a (broadcast waiting timeout), the event B (drop) and the service device side event C (user unsubscribe) in a certain period of time as an example, the indexes of calculating the association rule a- > C and B- > C are shown in table 1.
TABLE 1
| Association rules | Degree of support | Degree of confidence |
| A→C | n1 | n2 |
| B→C | n3 | n4 |
When the support degree n1 > n3 and the reliability n2 > n4 indicate that the probability of the event C is greater when the event A is generated, and the event A has a greater influence on the user experience, attention should be paid to reducing the occurrence of the event A in the actual service operation.
And step 505, outputting the analysis result of the business analysis device. I.e. outputting the analysis results to an output device for more intuitive display of the analysis results.
In summary, in the embodiment of the present invention, the service analysis device performs association analysis on the service data reported by the terminal and the service data reported by the service device, so as to optimize the service on the terminal and provide a more powerful data basis for data service optimization and user behavior analysis.
Furthermore, by installing the monitoring client on the terminal, the collected parameters can be reported through a wireless network, and the test can be carried out anytime and anywhere. The terminal side adopts a multi-mode matching method based on protocol analysis to collect event parameters, only one scanning is needed, and all data acquired by the terminal do not need to be stored. Parameters are acquired simultaneously through the terminal and the service platform and reported to the analysis platform, and user behaviors are analyzed through a data mining method to optimize the service.
In order to more clearly illustrate the technical solution provided by the embodiment of the present invention, the following further describes the processing procedure of the service analysis device by taking a certain service on the service platform as an example.
In the application scenario, the factor which influences the user satisfaction in a certain period of time is mined and analyzed by adopting the FP-Growth method as an example for explanation.
First, a database is entered, which includes historical data of a service and all available relevant factors, and a minimum support is determined.
Secondly, constructing an FP-Tree (sequencing all transaction data items in the transaction data table according to the support degree, and sequentially inserting the data items in each transaction into a Tree taking the NuLL as a root node according to the descending order).
1) And scanning the database once to obtain a set F of all the frequent items with the support degree greater than the minimum support degree, arranging the frequent items in a descending order according to the support degree of each frequent item, and recording the result as L.
2) And scanning the database again, forming a frequent pattern for the frequent items in each record according to the order of the frequent items in the L, and inserting the FP-Tree.
And finally, performing data mining on the FP-Tree to obtain an association rule, wherein the specific mining process is as follows:
the input is a Tree built by the FP-Tree algorithm, and the output is all the frequent sets.
1) If the tree only contains a single path, a frequent set output is generated for each combination of nodes in the path.
2) If the Tree contains a plurality of paths, starting from a frequent pattern (initial postfix pattern) with the length of l, constructing a condition pattern base (comprising a set of prefix paths appearing in the FP-Tree together with the postfix pattern), then constructing a condition Tree (forming a new FP-Tree from the condition pattern base according to the construction principle of the FP-Tree), and recursively mining on the Tree. And outputting a frequent set mode, wherein the frequent set mode is realized by connecting a postfix mode and a frequent mode generated by a condition tree, and the obtained frequent mode is an association rule between the service and relevant factors thereof.
In order to more clearly illustrate the technical solution provided by the embodiment of the present invention, the following further describes the processing procedure of the service analysis device by taking as an example that a user on the service platform uses a data service with client software, that is, details of the data collection situation of the terminal side and the service device and the method of joint analysis are described.
In this application scenario, as shown in the schematic diagram of system acquisition shown in fig. 6, the used data service has a client, and the user can configure the service at the client, and in the using process, receives an irrelevant multimedia message, and after finishing checking the information, the user returns to the client to continue the service use until the user exits the client after finishing the use.
In this application scenario, T0-T9 are parameters and occurrence times of user behaviors or events of the client that can be collected by the terminal side, and T0-T3 are parameters and occurrence times of user behaviors or events of the server side that can be collected by the service device.
After reporting all the data to the service analysis device, the service analysis device will perform sequencing analysis on the records according to time and event types, and obtain more complete user behavior of using the service and various parameters of the service at the client.
(1) According to the reported data at the terminal side, partial results can be obtained as follows:
client start time: T1-T0;
total time of user logging in once at client: T2-T1;
the total time for completing configuration information submission of the user at the client side is as follows: T9-T8;
the total time of the user staying at the client for one configuration operation is as follows: T8-T3- (T7-T6).
(2) According to the reported data of the service platform, partial results are obtained as follows:
the service platform needs time for processing one-time client login: t1-t 0;
the service platform needs time for processing one-time client configuration modification: t3-t 2;
the user logs in once at time t0 of the day and modifies the configuration information once at time t 2.
(3) The results of the association analysis of the terminal and the service platform are as follows:
network transmission delay of service client login: T0-T1 and T2-T1;
and the service completion user configures network transmission time delay T2-T8 and T9-T3 once.
Based on the same inventive concept as the method, an embodiment of the present invention further provides a data processing device, where the data processing device is the service analysis apparatus, and as shown in fig. 7, the data processing device includes:
the terminal parameter collection module 11 is configured to receive first service data reported by a terminal;
the platform parameter collecting module 12 is configured to receive second service data reported by the service device;
and an association analysis module 13, configured to perform association analysis on the service on the terminal according to the first service data received by the terminal parameter collection module 11 and the second service data received by the platform parameter collection module 12.
The association analysis module 13 is specifically configured to perform data sorting on the first service data and the second service data; and performing correlation analysis on the first service data and the second service data after data arrangement by adopting a correlation analysis method.
The association analysis module 13 is further configured to perform data integration on the first service data and the second service data; and subject data selection is carried out on the data after data integration according to the business analysis purpose and the task.
The data integration mode comprises one or more of the following modes: selecting a data source; data cleaning; denoising; and filling in the vacant data.
In the embodiment of the present invention, the apparatus further includes:
and an optimizing module 14, configured to optimize the service according to a result of the association analysis performed by the association analysis module 13.
And the output module 15 is configured to output the result of the association analysis to an output device, and the output device displays the result of the association analysis.
The modules of the device can be integrated into a whole or can be separately deployed. The modules can be combined into one module, and can also be further split into a plurality of sub-modules.
Based on the same inventive concept as the above method, an embodiment of the present invention further provides a terminal, as shown in fig. 8, including:
a clock synchronization module 21, configured to synchronize a time of the terminal with a time of the service device;
the monitoring execution module 22 is configured to collect first service data corresponding to a service, and timestamp the first service data;
and a parameter reporting module 23, configured to report the first service data collected by the monitoring execution module to a service analysis device.
The modules of the device can be integrated into a whole or can be separately deployed. The modules can be combined into one module, and can also be further split into a plurality of sub-modules.
Based on the same inventive concept as the above method, an embodiment of the present invention further provides a service apparatus, as shown in fig. 9, including:
a clock synchronization module 31, configured to synchronize the time of the service device with the time of the terminal;
the parameter acquisition module 32 is configured to acquire second service data corresponding to the terminal service; and time stamping the second service data;
and a parameter reporting module 33, configured to report the second service data acquired by the parameter acquiring module to a service analyzing apparatus.
The modules of the device can be integrated into a whole or can be separately deployed. The modules can be combined into one module, and can also be further split into a plurality of sub-modules.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
Those skilled in the art will appreciate that the drawings are merely schematic representations of one preferred embodiment and that the blocks or flow diagrams in the drawings are not necessarily required to practice the present invention.
Those skilled in the art will appreciate that the modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, and may be correspondingly changed in one or more devices different from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The above disclosure is only for a few specific embodiments of the present invention, but the present invention is not limited thereto, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present invention.
Claims (12)
1. A data processing method, comprising:
receiving first service data reported by a terminal and second service data reported by a service device; and analyzing the service according to the association of the first service data and the second service data.
2. The method of claim 1, wherein analyzing traffic based on the first traffic data and the second traffic data association comprises:
performing data sorting on the first service data and the second service data; and performing correlation analysis on the first service data and the second service data after data arrangement by adopting a correlation analysis method.
3. The method of claim 2, wherein performing data grooming on the first service data and the second service data comprises:
performing data integration on the first service data and the second service data; and subject data selection is carried out on the data after data integration according to the business analysis purpose and the task.
4. The method of claim 3, wherein the data integration comprises one or more of the following: selecting a data source; data cleaning; denoising; and filling in the vacant data.
5. The method of claim 1, wherein analyzing traffic based on the first traffic data and the second traffic data association, thereafter further comprises:
optimizing the service according to the result of the correlation analysis; or,
and outputting the result of the correlation analysis to an output device, and displaying the result of the correlation analysis by the output device.
6. The method of claim 1, wherein the receiving the first service data reported from the terminal and the second service data reported from the service device further comprises:
the terminal synchronizes the time of the terminal with the time of the service device; time stamping the first service data;
the service device synchronizes the time of the service device with the time of the terminal; and time stamping the second service data.
7. A data processing apparatus, characterized by comprising:
the terminal parameter collection module is used for receiving first service data reported by a terminal;
the platform parameter collection module is used for receiving second service data reported by the service device;
and the association analysis module is used for associating and analyzing services according to the first service data received by the terminal parameter collection module and the second service data received by the platform parameter collection module.
8. The apparatus of claim 7,
the association analysis module is specifically configured to perform data sorting on the first service data and the second service data; and performing correlation analysis on the first service data and the second service data after data arrangement by adopting a correlation analysis method.
9. The apparatus of claim 8,
the association analysis module is further configured to perform data integration on the first service data and the second service data; and subject data selection is carried out on the data after data integration according to the business analysis purpose and the task.
10. The apparatus of claim 7, further comprising:
the optimization module is used for optimizing the service according to the correlation analysis result of the correlation analysis module;
and the output module is used for outputting the result of the correlation analysis to output equipment, and the output equipment displays the result of the correlation analysis.
11. A terminal, comprising:
the clock synchronization module is used for synchronizing the time of the terminal with the time of the service device;
the monitoring execution module is used for acquiring first service data corresponding to a service and stamping a time stamp on the first service data;
and the parameter reporting module is used for reporting the first service data acquired by the monitoring execution module.
12. A business apparatus, comprising:
the clock synchronization module is used for synchronizing the time of the service device with the time of the terminal;
the parameter acquisition module is used for acquiring second service data corresponding to the terminal service; and time stamping the second service data;
and the parameter reporting module is used for reporting the second service data acquired by the parameter acquisition module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105273172A CN102457893A (en) | 2010-10-26 | 2010-10-26 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105273172A CN102457893A (en) | 2010-10-26 | 2010-10-26 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102457893A true CN102457893A (en) | 2012-05-16 |
Family
ID=46040422
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105273172A Pending CN102457893A (en) | 2010-10-26 | 2010-10-26 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102457893A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915423A (en) * | 2012-09-11 | 2013-02-06 | 中国电力科学研究院 | System and method for filtering electric power business data on basis of rough sets and gene expressions |
| CN104660438A (en) * | 2013-11-25 | 2015-05-27 | 中兴通讯股份有限公司 | Problem positioning processing method and device |
| CN105847083A (en) * | 2016-03-22 | 2016-08-10 | 乐视云计算有限公司 | Business centralized monitoring method and system |
| CN106056221A (en) * | 2016-05-27 | 2016-10-26 | 大连楼兰科技股份有限公司 | FP-Tree sequence pattern mining and fault code classification-based vehicle remote diagnosis and spare part retrieval method |
| CN106056223A (en) * | 2016-05-27 | 2016-10-26 | 大连楼兰科技股份有限公司 | Platform for vehicle remote diagnosis and spare part retrieval |
| CN110084695A (en) * | 2019-04-03 | 2019-08-02 | 平安普惠企业管理有限公司 | A kind of data transfering method and relevant device |
| CN111797848A (en) * | 2019-04-09 | 2020-10-20 | 成都鼎桥通信技术有限公司 | User classification method, device, equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1494268A (en) * | 2002-10-31 | 2004-05-05 | ��Ϊ��������˾ | Collection method of subscriber network use data |
| US20080109380A1 (en) * | 2006-11-02 | 2008-05-08 | Vhs, Llc | System, Report, and Computer-Readable Medium for Analyzing a Stock Portfolio |
| CN101184306A (en) * | 2007-12-14 | 2008-05-21 | 中国移动通信集团广东有限公司 | Mobile phone survey report and system for signaling data collection, process and correlation analysis |
| CN101184106A (en) * | 2007-11-30 | 2008-05-21 | 北京航空航天大学 | Associated transaction processing method of mobile database |
| CN101616428A (en) * | 2009-04-15 | 2009-12-30 | 北京协成致远网络科技有限公司 | A kind of mobile data service monitoring analysis system and its implementation |
| CN101751422A (en) * | 2008-12-08 | 2010-06-23 | 北京摩软科技有限公司 | Method, mobile terminal and server for carrying out intelligent search at mobile terminal |
| CN101789929A (en) * | 2009-04-30 | 2010-07-28 | 广东国笔科技股份有限公司 | Online business management network and operation method thereof |
| CN101790147A (en) * | 2009-01-23 | 2010-07-28 | 华为技术有限公司 | Method and system for correlating charging information |
-
2010
- 2010-10-26 CN CN2010105273172A patent/CN102457893A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1494268A (en) * | 2002-10-31 | 2004-05-05 | ��Ϊ��������˾ | Collection method of subscriber network use data |
| US20080109380A1 (en) * | 2006-11-02 | 2008-05-08 | Vhs, Llc | System, Report, and Computer-Readable Medium for Analyzing a Stock Portfolio |
| CN101184106A (en) * | 2007-11-30 | 2008-05-21 | 北京航空航天大学 | Associated transaction processing method of mobile database |
| CN101184306A (en) * | 2007-12-14 | 2008-05-21 | 中国移动通信集团广东有限公司 | Mobile phone survey report and system for signaling data collection, process and correlation analysis |
| CN101751422A (en) * | 2008-12-08 | 2010-06-23 | 北京摩软科技有限公司 | Method, mobile terminal and server for carrying out intelligent search at mobile terminal |
| CN101790147A (en) * | 2009-01-23 | 2010-07-28 | 华为技术有限公司 | Method and system for correlating charging information |
| CN101616428A (en) * | 2009-04-15 | 2009-12-30 | 北京协成致远网络科技有限公司 | A kind of mobile data service monitoring analysis system and its implementation |
| CN101789929A (en) * | 2009-04-30 | 2010-07-28 | 广东国笔科技股份有限公司 | Online business management network and operation method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 陈旭翔 等: "集团客户关联的监测分析研究", 《电信工程技术与标准化》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915423A (en) * | 2012-09-11 | 2013-02-06 | 中国电力科学研究院 | System and method for filtering electric power business data on basis of rough sets and gene expressions |
| CN104660438A (en) * | 2013-11-25 | 2015-05-27 | 中兴通讯股份有限公司 | Problem positioning processing method and device |
| CN105847083A (en) * | 2016-03-22 | 2016-08-10 | 乐视云计算有限公司 | Business centralized monitoring method and system |
| CN106056221A (en) * | 2016-05-27 | 2016-10-26 | 大连楼兰科技股份有限公司 | FP-Tree sequence pattern mining and fault code classification-based vehicle remote diagnosis and spare part retrieval method |
| CN106056223A (en) * | 2016-05-27 | 2016-10-26 | 大连楼兰科技股份有限公司 | Platform for vehicle remote diagnosis and spare part retrieval |
| CN110084695A (en) * | 2019-04-03 | 2019-08-02 | 平安普惠企业管理有限公司 | A kind of data transfering method and relevant device |
| CN110084695B (en) * | 2019-04-03 | 2023-11-28 | 上海七猫文化传媒有限公司 | Data transfer method and related equipment |
| CN111797848A (en) * | 2019-04-09 | 2020-10-20 | 成都鼎桥通信技术有限公司 | User classification method, device, equipment and storage medium |
| CN111797848B (en) * | 2019-04-09 | 2023-10-24 | 成都鼎桥通信技术有限公司 | User classification method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102457893A (en) | Data processing method and device | |
| CN102420712B (en) | Testing method and equipment | |
| US7526322B2 (en) | Real-time analyst program for processing log files from network elements | |
| US8352589B2 (en) | System for monitoring computer systems and alerting users of faults | |
| WO2017041406A1 (en) | Failure positioning method and device | |
| CN110730246A (en) | Distributed link tracking method under micro-service architecture | |
| CN101197621B (en) | Method and system for remote diagnosing and locating failure of network management system | |
| US9497451B2 (en) | Cable network data analytics system | |
| CN105023117B (en) | Based on the mobile live detection standardized work management system with cloud computing technology | |
| CN112000502B (en) | Processing method and device for mass error logs, electronic device and storage medium | |
| JPWO2014049804A1 (en) | System operation tracing method in distributed system | |
| CN105354042A (en) | Application installation processing method and apparatus | |
| CN111400127A (en) | Service log monitoring method and device, storage medium and computer equipment | |
| US20130073718A1 (en) | Test analysis system of network and analysis method thereof | |
| CN102571424A (en) | Processing method, device and system for engineering event | |
| CN113472577B (en) | Cluster inspection method, device and system | |
| CN101784060B (en) | Parameter processing method, network diagnosis method, terminal, server and system | |
| Nguyen et al. | Passive conformance testing of service choreographies | |
| CN110245120B (en) | Stream type computing system and log data processing method thereof | |
| CN111405603B (en) | Time delay acquisition method, testing device and computer storage medium | |
| CN110969082A (en) | Clock synchronization test inspection method and system | |
| CN112312376B (en) | Method and system for remotely and interactively managing multifunctional electric meter | |
| CN112612673B (en) | Analysis method and device for dial testing log, storage medium and electronic device | |
| CN114490560A (en) | Full-boundary power network data analysis method based on artificial intelligence technology | |
| CN109921937B (en) | Testing system and method for automatically verifying long-time task of flow protocol |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120516 |