CN105376335B

CN105376335B - Collected data uploading method and device

Info

Publication number: CN105376335B
Application number: CN201510938737.2A
Authority: CN
Inventors: 李宠波; 王景正
Original assignee: Beijing Qihoo Technology Co Ltd; Qizhi Software Beijing Co Ltd
Current assignee: Beijing Qihoo Technology Co Ltd
Priority date: 2015-12-15
Filing date: 2015-12-15
Publication date: 2019-12-24
Anticipated expiration: 2035-12-15
Also published as: CN105376335A

Abstract

The embodiment of the invention provides a method and a device for uploading collected data. The method comprises the following steps: calling an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; caching the parameter information of each acquisition parameter locally; and when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information. Therefore, the workload of developers of each service platform is reduced, the labor cost is further reduced, the system can be used in various applications, and the expandability is good.

Description

Collected data uploading method and device

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a collected data uploading method and a collected data uploading apparatus.

Background

Along with the rapid development of communication technology, especially along with the popularization of mobile internet, intelligent terminal's popularization is faster and faster, and intelligent terminal's function is also more and more abundant. The intelligent terminal realizes the functions thereof by installing various applications and provides diversified services.

In order to improve the service quality, developers of each service platform need to set a data acquisition scheme to acquire application data so as to know business requirements. Wherein, the application data collected by different services are different. Therefore, developers of the service platform need to set different application data acquisition schemes according to different services.

Specifically, a developer of the service platform needs to set acquisition parameters of each application according to a service provided by the service platform, perform data acquisition on each application according to the set acquisition parameters, and generate a service report of the service platform to know service requirements. However, the number of applications installed on the intelligent terminal is large, that is, the number of application data to be collected is large, so that the data collection formats of the service platforms are difficult to unify. Therefore, developers of each service platform need to define a data acquisition format of each application, so that each service platform can acquire data of each application according to the set data acquisition format, and the workload of the developers is further increased.

Disclosure of Invention

In view of the above problems, the present invention has been made to provide a collected data uploading method and a corresponding collected data uploading apparatus that overcome or at least partially solve the above problems.

According to an aspect of the present invention, there is provided a method for uploading collected data, the method including: calling an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; caching the parameter information of each acquisition parameter locally; and when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information.

Optionally, an upload interval is determined according to the current network type, and whether an upload condition is met is determined according to the upload interval.

Optionally, when the current time reaches the uploading interval, determining that the uploading condition is met; compressing and coding the parameter information according to a preset algorithm to generate a parameter data packet; and uploading the parameter data packet to a server.

Optionally, obtaining memory information of the mobile terminal where the application is located, and judging whether the memory information reaches an acquisition threshold; when the memory information reaches an acquisition threshold value, acquiring parameter information of each acquisition parameter in the application according to acquisition configuration information; and when the memory information does not reach the acquisition threshold value, acquiring the acquisition parameters of the static parameters and uploading the acquisition parameters to the server so as to finish the acquisition step.

Optionally, detecting whether the parameter information exceeds an upload threshold; and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

Optionally, detecting whether the parameter information of the local cache exceeds a cache threshold; and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

Optionally, the acquisition times and the uploading interval of the parameter information of the acquisition parameters are determined according to the priority of each acquisition parameter in the configuration information.

According to another aspect of the present invention, there is provided an apparatus for uploading collected data, the apparatus comprising: the acquisition module is used for calling an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; the cache module is used for caching the parameter information of each acquisition parameter locally; and the uploading module is used for compressing the cached parameter information and uploading the compressed parameter information to the server when the uploading condition is met, so that the server performs data analysis according to the parameter information.

Optionally, the uploading module is further configured to determine an uploading interval according to the current network type, and determine whether an uploading condition is met according to the uploading interval.

Optionally, the determining sub-module is configured to determine that the uploading condition is met when the current time reaches the uploading interval; the compression submodule is used for carrying out compression coding on the parameter information according to a preset algorithm to generate a parameter data packet; and the data packet uploading submodule is used for uploading the parameter data packet to a server.

Optionally, the uploading module is further configured to obtain memory information of the mobile terminal where the application is located, and determine whether the memory information reaches an acquisition threshold; when the memory information reaches an acquisition threshold value, acquiring parameter information of each acquisition parameter in the application according to acquisition configuration information; and when the memory information does not reach the acquisition threshold value, acquiring the acquisition parameters of the static parameters and uploading the acquisition parameters to the server to finish acquisition.

Optionally, the data packet partitioning sub-module is configured to detect whether the parameter information exceeds an upload threshold; and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

Optionally, the cache module is further configured to detect whether parameter information of the local cache exceeds a cache threshold; and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

Optionally, the acquisition module is further configured to determine acquisition times and an upload interval of the parameter information of the acquisition parameters according to priorities of the acquisition parameters in the configuration information.

Calling an acquisition tool to acquire parameter information of each acquisition parameter in the application during the running of the application, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; caching the parameter information of each acquisition parameter locally; and when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information. The workload of developers of each service platform can be reduced, so that the problem of heavy workload of the developers of each service platform is solved, the labor cost is reduced, the system can be used in various applications, and the beneficial effect of better expandability is achieved.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 shows a block diagram of a data collection analysis system according to one embodiment of the invention;

FIG. 2 is a flow chart illustrating steps of an embodiment of a collected data uploading method according to an embodiment of the present invention;

FIG. 3 is a flow chart illustrating steps of an embodiment of a method for uploading collected data according to another embodiment of the present invention;

FIG. 4 is a block diagram illustrating an embodiment of an apparatus for uploading collected data according to an embodiment of the present invention;

fig. 5 is a block diagram illustrating an embodiment of an apparatus for uploading collected data according to another embodiment of the present invention;

fig. 6 shows a block diagram of a data acquisition and analysis system according to another embodiment of the invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Referring to fig. 1, a block diagram of a data collection analysis system is shown, according to one embodiment of the present invention.

The data acquisition and analysis system comprises: the system comprises an acquisition control server 102, a data analysis server 104, a report generation server 106 and at least one data terminal 108, wherein an acquisition tool is installed in the data terminal.

The acquisition control server 102 is configured to configure the acquisition configuration information to a data terminal and send the acquisition configuration information to a corresponding data terminal, where the configuration information is used to control acquisition of acquisition parameters;

the data terminal 108 is configured to invoke the acquisition tool during application running, and acquire parameter information of acquisition parameters according to acquisition configuration information; uploading the acquired parameter information to the data analysis server;

the data analysis server 104 is configured to analyze the uploaded parameter information corresponding to each acquisition parameter to obtain a data analysis result; sending the data analysis result to the report generation server;

the report generation server 106 is configured to generate a data report according to the data analysis result.

Therefore, through the control of the acquisition control server (cloud server for short), the acquisition tool can acquire parameter information of each acquisition parameter in the data terminal in application operation, and after the parameter information is uploaded to the data analysis server, data analysis is carried out according to statistical requirements, so that a data report of corresponding services is generated, user requirements, habits and the like are acquired through the data report, the services are adjusted, and the user requirements are better met.

The embodiment of the invention discusses the method for uploading the collected data in detail.

Example one

Referring to fig. 2, a flowchart illustrating steps of an embodiment of a collected data uploading method according to an embodiment of the present invention is shown, which may specifically include the following steps:

step 202, calling an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, wherein the acquisition parameters comprise static parameters and/or dynamic parameters.

The application of the embodiment of the invention refers to an application program installed on a terminal, such as a mobile phone assistant, a browser, a security guard, a player and the like. To ensure the integrity and accuracy of the collected data, the collection tool may be invoked at application runtime. The collection tools include Software Development Kit (SDK).

The cloud server can configure acquisition parameters and acquisition configuration information of each application and generate control information of an acquisition tool for different applications and terminals where the applications are located. When the terminal where the application is located calls the acquisition tool to acquire data of the application, the acquisition tool can acquire parameter information of each acquisition parameter in the application according to the acquisition configuration information. And the acquisition configuration information is used for controlling acquisition and uploading of the acquisition parameters. The acquisition parameters include static parameters and/or dynamic parameters. The static parameter can be used for acquiring static parameter information corresponding to the application; the dynamic parameters can be used to collect dynamic parameter information during the running of the application.

And step 204, caching the parameter information of each acquisition parameter locally.

After the parameter information is collected, the collected parameter information can be cached locally.

And step 206, when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information.

And after acquiring the parameter information of each acquisition parameter in the application, the acquisition tool judges whether the uploading condition is met, and if the uploading condition is met, the acquisition tool compresses the cached parameter information according to a preset algorithm and uploads the parameter information to the server. In the embodiment of the invention, the cloud server can be connected with the terminal where each application is located to issue the configuration information in a wireless or wired connection mode, so that after the acquisition tool acquires the parameter information of each acquisition parameter, the acquisition tool can upload the parameter information corresponding to the acquisition parameter to the data analysis server. The data analysis server can analyze the parameter information according to a preset algorithm, and the report generation server generates business information required by each service platform based on the statistical result, such as generating a business report, determining business indexes and the like.

In summary, the acquisition tool is called when the application runs; the acquisition tool acquires parameter information of each acquisition parameter in the application; caching the parameter information of each acquisition parameter locally; and when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information. Therefore, the workload of developers of each service platform is reduced, the labor cost is further reduced, the system can be used in various applications, and the expandability is good.

Example two

Referring to fig. 3, a flowchart illustrating steps of an embodiment of a collected data uploading method according to another embodiment of the present invention is shown, which may specifically include the following steps:

step 302, determining the collection times and uploading intervals of the parameter information of the collection parameters according to the priority of each collection parameter in the configuration information.

The cloud server can also control the acquisition times and the uploading interval of the parameter information of the acquisition parameters by setting the priority of each acquisition parameter. For example, referring to table 1, when an application or a terminal where the application is located determines a priority of an acquisition parameter, the information of the acquisition parameter may be acquired according to the acquisition times corresponding to the determined priority, and the parameter information of the acquisition parameter is uploaded according to an uploading interval corresponding to the priority, where if the priority of the acquisition parameter I is level a, the acquisition times of the parameter information of the acquisition parameter I is 1000 times, and the uploading interval is 20 minutes, that is, the parameter information of the acquisition parameter I is uploaded once every 20 minutes.

Priority of acquisition parameters	Number of times of acquiring parameter information	Upload interval of parameter information
			Class A	1000 times (one time)	20 minutes
Class B	700 times	1 hour
			Class C	300 times (twice)	3 hours
Class D	10 times of	12 hours
			Class F	1 time of	24 hours

TABLE 1

In the embodiment of the invention, the priority of the acquisition parameters, the acquisition times of the parameter information corresponding to each priority and the uploading interval can be preset according to the service requirements or the characteristics of the acquired data, so that the cloud server can control the application or the terminal where the application is located to acquire and upload the parameter information according to the priority of the sampling parameters by adding the priority of the acquisition parameters into the acquisition configuration information. By acquiring the priority of the parameters, the parameter information of each application acquisition parameter can be acquired in time, and the acquired parameter information can be uploaded in time.

Step 304, obtaining the memory information of the mobile terminal where the application is located, and judging whether the memory information reaches an acquisition threshold value.

If the mobile terminal where the application is located is a sampling terminal, in the embodiment of the present invention, in order to reduce the influence of data acquisition on the terminal and enable the terminal to normally operate the application, the cloud server may set an acquisition threshold according to the memory of the terminal where the application is located and add the acquisition threshold to the acquisition configuration information, so as to control the acquired acquisition parameters according to the memory information of the terminal, in consideration of different terminal configurations, where some terminal memories are large, such as a memory reaching 2G, and some terminal memories are small.

If the memory information does not reach the collection threshold, executing step 306; when the memory information reaches the collection threshold, step 308 is executed.

And step 306, acquiring the parameter information of the static parameters and uploading the parameter information to the server so as to finish the acquisition step.

If the mobile terminal where the application is located is a non-sampling terminal, in order to avoid the problem of large data processing pressure of the cloud server caused by sampling of a large number of terminals, dynamic parameters can be collected only by uploading static parameters such as terminal identification once without collecting the dynamic parameters.

And 308, calling an acquisition tool to acquire parameter information of each acquisition parameter in the application during the running of the application, wherein the acquisition parameters comprise static parameters and/or dynamic parameters.

The cloud server can determine the applications to be acquired based on the data required to be acquired by each service platform. After determining the application, the cloud server may determine a static parameter of the application to be acquired according to the application type. And calling an acquisition tool during application running to acquire parameter information of the static parameters in the application.

The static parameters may be used to collect static parameter information corresponding to the application, such as identification information, version information, channel information, network type and resolution information of the terminal where the application is located. The identification information of the terminal can be used for identifying the terminal where the application is located, for example, an M2 value can be calculated based on information such as hardware of the terminal, the M2 value can be calculated through a serial number, a machine model number, version information and the like, the M2 value can identify different terminals and cannot reveal hardware information of the terminals, and privacy and safety of users are guaranteed; the version information may be used to record a version corresponding to the application, such as an operating system version, a software version, or a software development kit version of the terminal; the channel information may be used to identify an acquisition path for the application; the network type is used for determining the network type connected with the terminal where the application is located; the resolution information is used to determine the display resolution of the terminal where the application is located, which may also be referred to as the screen resolution.

Meanwhile, the cloud server can determine the applications to be acquired based on the data required to be acquired by each service platform. After determining the application, the capture tool may determine dynamic parameters of the application to be captured according to the application type. And calling an acquisition tool during application running to acquire parameter information of the dynamic parameters in the application.

The dynamic parameters may be used to collect parameter information generated by the application during the operation process, such as application interface start information, button click information, duration information, exception information, session information, and the like.

The application interface starting information is used for recording a page before switching and an interface after switching when the interface of the application is switched; the button clicking information is used for recording the button numbers corresponding to the buttons clicked by the user in the application and the clicking times corresponding to the button numbers; the duration information is determined according to the starting time point and the closing time point, such as application running time and interface running time; the abnormal information is used for recording information of abnormal application operation, such as time points or times of application starting failure; the session information is used to record messages of the conversational class generated by the application at runtime.

In a preferred embodiment of the present invention, the acquiring, in real time, the parameter information of the dynamic parameter according to the running of the application includes:

when detecting the interface switching of the application, recording that the interface before switching is the previous page, recording that the interface after switching is the current page, and adding the previous page and the current page into the application interface starting information. That is, when a user accesses a record in an application interface, in order to ensure continuity of access, a previous access interface may be recorded while recording a current interface, so that each interface record is current page + previous page, and of course, if the current interface record is the first page, the previous page is empty.

And when the application button is detected to be triggered, acquiring a button number corresponding to the button, updating the click times corresponding to the button number, and adding the button number and the click times into button click information. Many buttons may be included in an application, and each button may be numbered to identify a different button, so that when the button is clicked, the button number and the number of clicks are recorded, i.e. the number of clicks is initially 0 and is incremented by one each click.

The application can be operated in a multi-process mode, generally, when the multi-process is operated, the application is started by adopting a main process to record the starting time point, and the last process is adopted to record the ending time point, however, when the application starting time is too short, the problem that the main process is not completely started and the application is closed can occur, the recorded running time can be a negative value, or the time length is calculated by adopting the starting time point of the last start and the ending time point of the current closing, the condition of over long time length can occur, and the time length recording is inaccurate.

In order to solve the inaccurate problem, collecting the running time related to the application as time length information, wherein the time length information includes: interface runtime and application runtime. Wherein the collecting the running time related to the application as the duration information comprises: and recording a starting time point and an ending time point by the main process of the application, recording running time according to the starting time point and the ending time point, and taking the running time as duration information. That is, for the application running in multiple processes, the start time and the end time are recorded by using the main process in the embodiment, that is, the start time point is recorded by using the main process when the application is started, and the end time point is recorded by using the main process when the application is closed, so that the difference between the end time point and the start time point is calculated to be the running time, and the running time length is ensured to be accurately recorded.

And step 310, caching the parameter information of each acquisition parameter locally.

Step 312, it is detected whether the parameter information of the local cache exceeds the cache threshold.

If the parameter information of the local cache does not exceed the cache threshold, go to step 316; if the parameter information of the local cache exceeds the cache threshold, go to step 314.

Step 314, if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

The caching threshold in the embodiment of the invention is used for determining the data volume of the parameter information cached at the terminal. Specifically, different terminals have different capabilities, and thus the amount of parameter information that can be cached by the terminals is different. According to the data amount which can be cached by the terminal where the application is located, a cache threshold corresponding to the terminal can be set, and the cache threshold is added into the acquisition configuration information. When the locally cached parameter information of the terminal exceeds the caching threshold, the cloud server can control the terminal to delete the parameter information before the caching time until the cached parameter information does not exceed the caching threshold, so that the data acquisition efficiency is ensured. For example, a cache threshold is set to 2M, and when the acquisition parameter information is to be stored, it is determined that the local cache has exceeded 2M, the parameter information may be deleted from the front in the sequence before and after the time until the acquisition parameter is stored and does not exceed 2M.

Step 316, detecting whether the parameter information exceeds an upload threshold.

If the parameter information does not exceed the upload threshold, go to step 320; when the parameter information exceeds the upload threshold, step 318 is performed.

Step 318, when the parameter information exceeds the uploading threshold, dividing the parameter information into at least two data packets.

The embodiment of the invention uploads the parameter information to the server through the data packet. If the amount of data contained in a data packet is too large, uploading failure is easily caused. The embodiment of the invention limits the size of each uploading data packet through the uploading threshold. When the parameter information needing to be uploaded exceeds an uploading threshold value, a sub-package measure is taken, namely the parameter information is divided into at least two data packages. And the sub-package uploading can avoid uploading failure caused by overlarge data volume. For example, when the upload threshold is 100k, and when the parameter information to be uploaded is greater than 100k, the parameter information may be divided into at least two data packets, and for example, the parameter information to be uploaded is 250k, the parameter information may be divided into three data packets 100k, and 50 k.

And 320, determining an uploading interval according to the current network type, and judging whether an uploading condition is met according to the uploading interval.

The data is transmitted through different types of networks at different rates. Therefore, the uploading interval of the network type corresponding to the terminal where the application is located can be set according to the data transmission rate corresponding to the network type, and the uploading intervals of different network types are added into the acquisition configuration information. And determining an uploading interval according to the current network type, and judging whether an uploading condition is met according to the uploading interval. Detecting whether the current time reaches an uploading interval, if so, judging that the uploading condition is met according to the uploading interval, and executing a step 322; if not, that is, the current time does not reach the uploading interval, at this time, it is determined that the uploading condition is not met according to the uploading interval, and the step 320 is returned to, and the uploading condition is continuously determined.

And 322, performing compression coding on the parameter information according to a preset algorithm to generate a parameter data packet.

Step 324, uploading the parameter data packet to a server, so that the server performs data analysis according to the parameter information.

And uploading the parameter information when the parameter information meets the uploading condition in the sampling configuration information. For example, the cloud server may set an upload condition of each parameter information by setting an upload priority or an upload interval of each parameter information, and store the upload condition in the acquisition configuration information. When the current time reaches the uploading interval time of the parameter information, the terminal where the application is located can compress and encode the parameter information to generate a parameter data packet. And uploading the parameter data packet to a cloud server, namely uploading the parameter information meeting the uploading condition. So that the server performs data analysis according to the parameter information. Obviously, the cloud server may set an upload condition for different parameter information, add the upload condition to the acquisition configuration information, and further may control the upload of each parameter information through the upload condition.

In summary, whether the memory information reaches an acquisition threshold value is judged by acquiring the memory information of the mobile terminal where the application is located; when the memory information reaches an acquisition threshold value, executing a step of acquiring the parameter information of the application according to acquisition configuration information; and when the memory information does not reach the acquisition threshold value, acquiring the static parameters and uploading the static parameters to the server so as to finish the acquisition step. Meanwhile, the terminals with different configurations are considered, so that data acquisition is carried out on the application of the terminal while the normal use of the terminal is not influenced.

For simplicity of explanation, the method embodiments are described as a series of acts or combinations, but those skilled in the art will appreciate that the embodiments are not limited by the order of acts described, as some steps may occur in other orders or concurrently with other steps in accordance with the embodiments of the invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

EXAMPLE III

Referring to fig. 4, a block diagram of an embodiment of an apparatus for uploading collected data according to an embodiment of the present invention is shown.

An acquisition module 402, configured to invoke an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, where the acquisition parameter includes a static parameter and/or a dynamic parameter.

The cache module 404 is configured to cache the parameter information of each acquisition parameter locally.

And the uploading module 406 is configured to, when the uploading condition is met, compress the cached parameter information and upload the compressed parameter information to the server, so that the server performs data analysis according to the parameter information.

Example four

Referring to fig. 5, a block diagram of an embodiment of an apparatus for uploading collected data according to another embodiment of the present invention is shown.

The uploading module 406 is further configured to determine an uploading interval according to the current network type, and determine whether an uploading condition is met according to the uploading interval.

The upload module 406 includes: the confirming submodule 4062 is configured to confirm that the uploading condition is met when the uploading interval corresponding to the network type is reached at the current time; the compression submodule 4064 is used for carrying out compression coding on the parameter information according to a preset algorithm to generate a parameter data packet; and the uploading sub-module 4066 is configured to upload the parameter data packet to a server.

The uploading module is further used for acquiring the memory information of the mobile terminal where the application is located and judging whether the memory information reaches an acquisition threshold value; when the memory information reaches an acquisition threshold value, acquiring parameter information of each acquisition parameter in the application according to acquisition configuration information; and when the memory information does not reach the acquisition threshold value, acquiring the acquisition parameters of the static parameters and uploading the acquisition parameters to the server to finish acquisition.

The upload module 406 further includes:

the data packet dividing submodule 4068 is used for detecting whether the parameter information exceeds an uploading threshold value; and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

The cache module 404 is further configured to detect whether the parameter information of the local cache exceeds a cache threshold; and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

The acquisition module 402 is further configured to determine acquisition times and an upload interval of the parameter information of the acquisition parameters according to priorities of the acquisition parameters in the configuration information.

In the data collecting and analyzing system, the collected data uploading apparatus of this embodiment may be configured in a data terminal, as shown in fig. 6, and the data collecting and analyzing system includes: the system comprises an acquisition control server 102, a data analysis server 104, a report generation server 106 and at least one data terminal 108, wherein an acquisition tool is installed in the data terminal. The acquisition control server 102 includes: cloud control server 1022 and development configuration server 1024.

The acquisition control server 102 configures the acquisition configuration information to the data terminals and issues the acquisition configuration information to the corresponding data terminals 108; the data terminal 108 calls the acquisition tool during application running, and acquires parameter information of acquisition parameters according to acquisition configuration information; the data terminal 108 uploads the acquired parameter information to the acquisition control server 104; the data analysis server 104 analyzes the uploaded parameter information corresponding to each acquisition parameter to obtain a data analysis result; the data analysis server 104 sends the data analysis result to the report generation server 106; and the report generation server 106 generates a data report according to the data analysis result.

In one example, the acquisition control server 102 is configured to configure the acquisition configuration information to a data terminal and send the configuration information to a corresponding data terminal, where the configuration information is used to control acquisition of acquisition parameters; the data terminal 108 is configured to invoke the acquisition tool during application running, and acquire parameter information of acquisition parameters according to acquisition configuration information; uploading the acquired parameter information to the data analysis server; the data analysis server 104 is configured to analyze the uploaded parameter information corresponding to each acquisition parameter to obtain a data analysis result; sending the data analysis result to the report generation server; the report generation server 106 is configured to generate a data report according to the data analysis result.

In an optional embodiment, the acquisition control server 102 includes a cloud control server 1022; the cloud control server 1022 is configured to determine acquisition parameters of the application according to an application type, where the acquisition parameters include: dynamic parameters and/or static parameters; configuring acquisition configuration information according to the application and the terminal where the application is located, wherein the acquisition configuration information is used for controlling acquisition and uploading of the acquisition parameters; and transmitting the acquisition parameters and the acquisition configuration information to the data terminal where the application is located.

The acquiring control server configures the acquiring configuration information to the data terminal and issues the acquiring configuration information to the corresponding data terminal, and the acquiring control server includes: the acquisition control server determines acquisition parameters of the application according to the application type, wherein the acquisition parameters comprise: dynamic parameters and/or static parameters; acquiring configuration information configured according to the application and the terminal where the application is located, wherein the acquiring configuration information is used for controlling acquisition and uploading of the acquisition parameters; and transmitting the acquisition parameters and the acquisition configuration information to the data terminal where the application is located.

The cloud control server 1022 is configured to configure an uploading condition and/or a sampling condition of the parameter information, and add the uploading condition and/or the sampling condition to the acquisition configuration information; configuring an acquisition threshold value of a memory corresponding to the terminal where the application is located, and adding the acquisition threshold value into acquisition configuration information; configuring an uploading interval of the corresponding network type of the terminal where the application is located, and adding the uploading interval into the acquisition configuration information; configuring a cache threshold corresponding to a terminal where the application is located, and adding the cache threshold into acquisition configuration information; and configuring the protocol uploading sequence, the sampling rate and the priority of the sampling parameters in the acquisition configuration information.

The step of collecting configuration information configured according to the application and the terminal where the application is located comprises at least one of the following steps: configuring uploading conditions and/or sampling conditions of the parameter information, and adding the uploading conditions and/or the sampling conditions into the acquisition configuration information; configuring an acquisition threshold value of a memory corresponding to the terminal where the application is located, and adding the acquisition threshold value into acquisition configuration information; configuring an uploading interval of the corresponding network type of the terminal where the application is located, and adding the uploading interval into the acquisition configuration information; configuring a cache threshold corresponding to a terminal where the application is located, and adding the cache threshold into acquisition configuration information; and configuring the protocol uploading sequence, the sampling rate and the priority of the sampling parameters in the acquisition configuration information.

The data terminal 108 is configured to acquire parameter information of the static parameter of the application according to acquisition configuration information by using an acquisition tool; and determining dynamic parameters to be acquired by adopting the acquisition tool according to acquisition configuration information, and acquiring parameter information of the dynamic parameters in real time according to the operation of the application.

The acquiring of the parameter information of the acquisition parameters according to the acquisition configuration information comprises: acquiring parameter information of the static parameters of the application by adopting the acquisition tool according to acquisition configuration information; and determining dynamic parameters to be acquired by adopting the acquisition tool according to acquisition configuration information, and acquiring parameter information of the dynamic parameters in real time according to the operation of the application.

The data terminal 108 is configured to, when detecting that the interface of the application is switched, record that the interface before switching is a previous page, record that the interface after switching is a current page, and add the previous page and the current page to application interface start information; when the application button is detected to be triggered, acquiring a button number corresponding to the button, updating the click times corresponding to the button number, and adding the button number and the click times into button click information; collecting the running time related to the application as duration information, wherein the duration information comprises: interface runtime and application runtime.

The step of collecting parameter information of the dynamic parameters in real time according to the operation of the application comprises at least one of the following: when detecting the interface switching of the application, recording the interface before switching as a previous page, recording the interface after switching as a current page, and adding the previous page and the current page into application interface starting information; when the application button is detected to be triggered, acquiring a button number corresponding to the button, updating the click times corresponding to the button number, and adding the button number and the click times into button click information; collecting the running time related to the application as duration information, wherein the duration information comprises: interface runtime and application runtime.

Wherein the parameter information of the static parameter comprises at least one of: the method comprises the steps of identifying a terminal, version information, channel information, network type and resolution information; the parameter information of the dynamic parameter includes at least one of: application interface start information, button click information, duration information, exception information, and session information.

In this embodiment, the report generation server 106 is configured to obtain service statistical information according to the self-defined management information of the self-defined event corresponding to the configured application; and generating a self-defined event report according to the self-defined management information and the service statistical information.

The report generation server generates a data report according to the data analysis result, and the report generation method comprises the following steps: the report generation server acquires business statistical information according to the self-defined management information of the self-defined event corresponding to the configured application; and generating a self-defined event report according to the self-defined management information and the service statistical information.

The report generating server 106 is further configured to obtain an application path of the application from the service management page and custom event information of a custom event corresponding to the application, where the custom event information includes: event name and event attribute; and generating custom management information by adopting the application path and the custom event information.

The method also comprises the steps of configuring custom management information: acquiring an application path of the application and custom event information of a custom event corresponding to the application from the service management page, wherein the custom event information comprises: event name and event attribute; and generating custom management information by adopting the application path and the custom event information.

The report generation server 106 is configured to determine an application according to the application path; acquiring a logic name corresponding to the event name and a logic attribute corresponding to the event attribute for each user-defined event; searching acquisition parameters corresponding to the logic names, and determining service statistical information corresponding to the acquisition parameters according to the logic attributes; and determining the event name corresponding to each acquisition parameter, and generating a self-defined event report of the application by adopting each event name and the service statistical information corresponding to the acquisition parameter.

Acquiring service statistical information according to the user-defined management information of the user-defined event corresponding to the configured application; generating a custom event report according to the custom management information and the service statistical information, wherein the custom event report comprises the following steps: determining an application according to the application path; acquiring a logic name corresponding to the event name and a logic attribute corresponding to the event attribute for each user-defined event; searching acquisition parameters corresponding to the logic names, and determining service statistical information corresponding to the acquisition parameters according to the logic attributes; and determining the event name corresponding to each acquisition parameter, and generating a self-defined event report of the application by adopting each event name and the service statistical information corresponding to the acquisition parameter.

The acquisition control server is further used for configuring test information of the test equipment corresponding to the target application; the data analysis server is also used for analyzing the parameter information uploaded correspondingly by the target application; when the parameter information is judged to belong to the test equipment, analyzing the parameter information belonging to the test equipment in real time; and the report generation server is also used for generating a data report in real time according to the data analysis result when the parameter information is judged to belong to the test equipment.

The method further comprises the following steps: the acquisition control server configures test information of the test equipment corresponding to the target application; the data analysis server analyzes the parameter information uploaded correspondingly by the target application; when the parameter information is judged to belong to the test equipment, analyzing the parameter information belonging to the test equipment in real time; and when the report generation server judges that the parameter information belongs to the test equipment, generating a data report in real time according to a data analysis result.

Wherein, the collection control server 102 includes: a development configuration server 1024; the development configuration server is used for configuring a test identifier of at least one test device for the target application after the target application finishes registration; configuring acquisition parameters to be tested for the target application, and taking the test identification and the acquisition parameters as test information; the data analysis server is also used for acquiring the parameter information of the acquisition parameters corresponding to the target application from the uploaded parameter information; corresponding the terminal parameter in the parameter information to the terminal identifier; and judging whether the terminal identification is the same as the test identification in the test equipment table, and judging that the parameter information belongs to the test equipment when the terminal identification is the same as any test identification in the test equipment table.

Configuring test information of the test equipment corresponding to the target application, wherein the configuration comprises the following steps: after the target application finishes registering, configuring a test identifier of at least one test device for the target application; configuring acquisition parameters to be tested for the target application, and taking the test identification and the acquisition parameters as test information; the method further comprises the following steps: the data analysis server acquires parameter information of acquisition parameters corresponding to the target application from the uploaded parameter information; corresponding the terminal parameter in the parameter information to the terminal identifier; and judging whether the terminal identification is the same as the test identification in the test equipment table, and judging that the parameter information belongs to the test equipment when the terminal identification is the same as any test identification in the test equipment table.

The data analysis server 104 is further configured to search, from the parameter information, verification information corresponding to a verification parameter; and analyzing the verification information to determine a hijacking verification result.

The method further comprises the following steps: the data analysis server searches the check information corresponding to the check parameter from the parameter information; and analyzing the verification information to determine a hijacking verification result.

The data analysis server 104 is configured to determine a parameter identifier of a verification parameter, and search, from the parameter information, verification information corresponding to the parameter identifier; detecting whether the check information is target information; if the verification information is not the target information, determining that hijacking abnormity exists in the flow, and recording the hijacking abnormity in the hijacking verification result; and if the verification information is the target information, determining that the data transmission is normal, and recording the normal transmission in the hijack verification result.

Searching check information corresponding to the check parameters from the parameter information; analyzing the verification information to determine a hijacking verification result, comprising: determining a parameter identifier of a verification parameter, and searching verification information corresponding to the parameter identifier from the parameter information; detecting whether the check information is target information; if the verification information is not the target information, determining that hijacking abnormity exists in the flow, and recording the hijacking abnormity in the hijacking verification result; and if the verification information is the target information, determining that the data transmission is normal, and recording the normal transmission in the hijack verification result.

The data analysis server 104 is further configured to search storage information corresponding to the storage parameter from the parameter information; and detecting the data quality according to the storage information, and determining a quality check result.

The method further comprises the following steps: the data analysis server searches storage information corresponding to the storage parameters from the parameter information; and detecting the data quality according to the storage information, and determining a quality check result.

The data analysis server 104 is configured to determine a storage identifier of a storage parameter, and search, from the parameter information, storage information corresponding to the storage identifier; searching at least one terminal identification according to the storage information; when the stored information corresponds to a terminal identification, generating a quality verification result with qualified quality; when the stored information corresponds to more than one terminal identification, searching for successfully uploaded information corresponding to the terminal identification; and sequencing the uploading successful information corresponding to each terminal identification, judging data abnormity when determining that the uploading successful information has no redundancy according to the sequencing result, and generating a quality check result of the data abnormity.

Searching storage information corresponding to the storage parameters from the parameter information; detecting data quality according to the stored information, and determining a quality check result, wherein the quality check result comprises the following steps: determining a storage identifier of a storage parameter, and searching storage information corresponding to the storage identifier from the parameter information; searching at least one terminal identification according to the storage information; when the stored information corresponds to a terminal identification, generating a quality verification result with qualified quality; when the stored information corresponds to more than one terminal identification, searching for successfully uploaded information corresponding to the terminal identification; and sequencing the uploading successful information corresponding to each terminal identification, judging data abnormity when determining that the uploading successful information has no redundancy according to the sequencing result, and generating a quality check result of the data abnormity.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the collected data upload method and data collection upload apparatus device according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The invention discloses A1 and a collected data uploading method, which comprises the following steps: calling an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; caching the parameter information of each acquisition parameter locally; and when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information.

A2, the method of a1, further comprising: and determining an uploading interval according to the current network type, and judging whether an uploading condition is met according to the uploading interval.

A3, the method according to A2, wherein when the uploading condition is met, the compressing and uploading the parameter information to the server includes: when the current moment reaches the uploading interval, confirming that the uploading condition is met; compressing and coding the parameter information according to a preset algorithm to generate a parameter data packet; and uploading the parameter data packet to a server.

A4, the method as in a1, further including, before the acquiring tool is called to acquire the parameter information of each acquisition parameter in the application: acquiring memory information of the mobile terminal where the application is located, and judging whether the memory information reaches an acquisition threshold value; when the memory information reaches an acquisition threshold value, acquiring parameter information of each acquisition parameter in the application according to acquisition configuration information; and when the memory information does not reach the acquisition threshold value, acquiring the acquisition parameters of the static parameters and uploading the acquisition parameters to the server so as to finish the acquisition step.

A5, wherein before the compressing and encoding the parameter information according to the preset algorithm, the method according to A3 further comprises: detecting whether the parameter information exceeds an uploading threshold; and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

A6, the method as in a2, further comprising, after caching the parameter information of each acquisition parameter locally: detecting whether the parameter information of the local cache exceeds a cache threshold value; and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

A7, the method of a1, further comprising: and determining the acquisition times and the uploading interval of the parameter information of the acquisition parameters according to the priority of each acquisition parameter in the configuration information.

The invention also discloses B8, a device for uploading collected data, comprising: the acquisition module is used for calling an acquisition tool to acquire parameter information of each acquisition parameter in the application when the application runs, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; the cache module is used for caching the parameter information of each acquisition parameter locally; and the uploading module is used for compressing the cached parameter information and uploading the compressed parameter information to the server when the uploading condition is met, so that the server performs data analysis according to the parameter information.

B9, the apparatus according to B8, and the upload module, further configured to determine an upload interval according to the current network type, and determine whether the upload condition is satisfied according to the upload interval.

B10, the apparatus as in B9, the upload module comprising: the confirmation submodule is used for confirming that the uploading condition is met when the uploading interval is reached at the current moment; the compression submodule is used for carrying out compression coding on the parameter information according to a preset algorithm to generate a parameter data packet; and the data packet uploading submodule is used for uploading the parameter data packet to a server.

The device according to B11, as described in B8, the upload module is further configured to obtain memory information of the mobile terminal where the application is located, and determine whether the memory information reaches an acquisition threshold; when the memory information reaches an acquisition threshold value, acquiring parameter information of each acquisition parameter in the application according to acquisition configuration information; and when the memory information does not reach the acquisition threshold value, acquiring the acquisition parameters of the static parameters and uploading the acquisition parameters to the server to finish acquisition.

B12, the apparatus as described in B10, the upload module further comprising: the data packet dividing submodule is used for detecting whether the parameter information exceeds an uploading threshold value; and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

B13, the apparatus according to B9, the cache module, further configured to detect whether the parameter information of the local cache exceeds a cache threshold; and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

B14, the apparatus according to B8, and the acquisition module, further configured to determine acquisition times and an upload interval of the parameter information of the acquisition parameters according to priorities of the acquisition parameters in the configuration information.

Claims

1. A method for uploading collected data is applied to a data collection and analysis system, the data collection and analysis system comprises a collection control server and a data terminal, the collection control server is used for configuring collection configuration information to the data terminal and issuing the configuration information to the corresponding data terminal, wherein the configuration information is used for controlling collection of collection parameters, and the method comprises the following steps:

calling an acquisition tool during application running to acquire parameter information of each acquisition parameter in the application according to acquisition configuration information, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; the acquisition tool is installed in the data terminal;

caching the parameter information of each acquisition parameter locally;

when the uploading condition is met, compressing the cached parameter information and uploading the compressed parameter information to a server so that the server performs data analysis according to the parameter information;

before the acquisition tool is called to acquire the parameter information of each acquisition parameter in the application, the method further comprises the following steps:

acquiring memory information of the mobile terminal where the application is located, and judging whether the memory information reaches an acquisition threshold value;

when the memory information reaches an acquisition threshold value, acquiring parameter information of each acquisition parameter in the application according to acquisition configuration information;

and when the memory information does not reach the acquisition threshold value, acquiring the acquisition parameters of the static parameters and uploading the acquisition parameters to the server so as to finish the acquisition step.

2. The method of claim 1, further comprising:

and determining an uploading interval according to the current network type, and judging whether an uploading condition is met according to the uploading interval.

3. The method according to claim 2, wherein when the uploading condition is satisfied, the compressing and uploading the parameter information to the server comprises:

when the current moment reaches the uploading interval, confirming that the uploading condition is met;

compressing and coding the parameter information according to a preset algorithm to generate a parameter data packet;

and uploading the parameter data packet to a server.

4. The method of claim 3, wherein before the compression-encoding the parameter information according to the preset algorithm, further comprising:

detecting whether the parameter information exceeds an uploading threshold;

and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

5. The method of claim 2, wherein after caching the parameter information of each acquisition parameter locally, further comprising:

detecting whether the parameter information of the local cache exceeds a cache threshold value;

and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

6. The method of claim 1, further comprising:

and determining the acquisition times and the uploading interval of the parameter information of the acquisition parameters according to the priority of each acquisition parameter in the configuration information.

7. The utility model provides a collection data upload device, the device is applied to in the data acquisition analytic system, the data acquisition analytic system includes acquisition control server and data terminal, acquisition control server is used for, and configuration acquisition configuration information gives data terminal and issues to corresponding data terminal, and wherein, configuration information is used for controlling the collection of gathering the parameter, the device includes:

the acquisition module is used for calling an acquisition tool to acquire parameter information of each acquisition parameter in the application according to acquisition configuration information during the running of the application, wherein the acquisition parameters comprise static parameters and/or dynamic parameters; the acquisition tool is installed in the data terminal;

the cache module is used for caching the parameter information of each acquisition parameter locally;

the uploading module is used for compressing the cached parameter information and uploading the compressed parameter information to a server when the uploading condition is met so that the server can analyze data according to the parameter information;

8. The apparatus of claim 7,

and the uploading module is also used for determining an uploading interval according to the current network type and judging whether the uploading condition is met according to the uploading interval.

9. The apparatus of claim 8, wherein the upload module comprises:

the confirmation submodule is used for confirming that the uploading condition is met when the uploading interval is reached at the current moment;

the compression submodule is used for carrying out compression coding on the parameter information according to a preset algorithm to generate a parameter data packet;

and the data packet uploading submodule is used for uploading the parameter data packet to a server.

10. The apparatus of claim 9, the upload module further comprising:

the data packet dividing submodule is used for detecting whether the parameter information exceeds an uploading threshold value; and when the parameter information exceeds an uploading threshold, dividing the parameter information into at least two data packets.

11. The apparatus of claim 8,

the cache module is also used for detecting whether the parameter information of the local cache exceeds a cache threshold value; and if the parameter information of the local cache exceeds the cache threshold, deleting the parameter information with the previous cache time until the cached parameter information does not exceed the cache threshold.

12. The apparatus of claim 7, wherein the first and second electrodes are disposed on opposite sides of the substrate,

and the acquisition module is also used for determining the acquisition times and the uploading interval of the parameter information of the acquisition parameters according to the priority of each acquisition parameter in the configuration information.