WO2017101606A1 - Système et procédé de collecte et d'analyse de données - Google Patents

Système et procédé de collecte et d'analyse de données Download PDF

Info

Publication number
WO2017101606A1
WO2017101606A1 PCT/CN2016/104923 CN2016104923W WO2017101606A1 WO 2017101606 A1 WO2017101606 A1 WO 2017101606A1 CN 2016104923 W CN2016104923 W CN 2016104923W WO 2017101606 A1 WO2017101606 A1 WO 2017101606A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
parameter
collection
application
data
Prior art date
Application number
PCT/CN2016/104923
Other languages
English (en)
Chinese (zh)
Inventor
王景正
Original Assignee
北京奇虎科技有限公司
奇智软件（北京）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京奇虎科技有限公司, 奇智软件（北京）有限公司 filed Critical 北京奇虎科技有限公司
Publication of WO2017101606A1 publication Critical patent/WO2017101606A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/02—Capturing of monitoring data
    • H04L43/022—Capturing of monitoring data by sampling
    • H04L43/024—Capturing of monitoring data by sampling by adaptive sampling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation

Definitions

  • the present invention relates to the field of data processing technologies, and in particular, to a data acquisition and analysis method, a data acquisition and analysis system, a program, and a readable medium.
  • the smart terminal provides a variety of services by installing various applications to realize its functions.
  • developers of each service platform need to set up a data collection scheme to collect application data to understand business needs.
  • the application data collected by different services is different. Therefore, the developer of the service platform needs to set different application data collection schemes according to different services.
  • the developer of the service platform needs to set the collection parameters of each application according to the services provided by the service platform, collect data for each application according to the set collection parameters, and generate a service report of the service platform to understand the business requirements.
  • the developers of each service platform also need to define the data collection format of each application, so that each service platform can collect the data of each application according to the set data collection format, further increasing the workload of the developer.
  • the present invention has been made in order to provide a data acquisition and analysis method and a corresponding data acquisition and analysis system that overcome the above problems or at least partially solve the above problems.
  • a data collection and analysis system includes: an acquisition control server, a data analysis server, a report generation server, and at least one data terminal, wherein the data terminal is provided with an acquisition tool; the collection control server And configured to collect the configuration information to the data terminal and send the data to the corresponding data terminal, where the collection configuration information is used to control the collection of the collection parameters, and the data terminal is configured to invoke the collection tool when the application is running.
  • the data analysis server is configured to analyze the corresponding parameter information of each uploaded parameter and obtain the data And analyzing the result; sending the data analysis result to the report generation server; and the report generation server is configured to generate a data report according to the data analysis result.
  • the data collection and analysis method is provided to be applied to a data collection and analysis system
  • the data collection and analysis system includes: the collection control server, a data analysis server, a report generation server, and at least one data terminal.
  • An acquisition tool is installed in the data terminal; the method includes: the collection control server is configured to collect configuration information and send the configuration information to the data terminal, and the data terminal invokes the collection tool when the application is running. Collecting parameter information of the collected parameter according to the collected configuration information; the data terminal uploads the collected parameter information to the collection control server; the data analysis server analyzes the corresponding parameter information of each uploaded parameter, and obtains data analysis.
  • the data analysis server sends the data analysis result to the report generation server; the report generation server generates a data report according to the data analysis result.
  • a program comprising readable code, when the readable code is run on a computing device, causing the computing device to perform the method according to any of the embodiments of the present invention Data collection and analysis methods.
  • the embodiment of the invention further provides a readable medium, wherein the program according to the embodiment of the invention is stored.
  • the collection tool When the application starts and initializes, the collection tool is invoked.
  • the collection tool may collect parameter information of each collection parameter in the application according to the collection configuration information.
  • the collection tool determines whether the upload condition is met. If yes, the collection tool compresses the parameter information according to the preset algorithm, and then uploads the data to the data analysis server.
  • the data analysis server analyzes the corresponding parameter information of each uploaded parameter, such as the pair of each collection parameter.
  • the parameter information is collected to obtain the data analysis result, and the data analysis result is sent to the report generation server, and the report generation server configures the data analysis result of each logic according to the data analysis result.
  • the attribute information generates a data report so that the usage of the application can be determined based on the data report. Therefore, the collection tool can collect the parameter information of each collection parameter in the data terminal of the application operation, and after uploading to the data analysis server, perform data analysis according to the statistical requirement, thereby generating a data report of the corresponding service, and obtaining the user requirement through the data report. , habits, etc., to adjust the business, more in line with user needs.
  • FIG. 1 is a block diagram showing the structure of a data acquisition and analysis system according to an embodiment of the present invention
  • FIG. 2 is a flow chart showing the steps of a data acquisition and analysis method according to an embodiment of the present invention
  • FIG. 3 is a block diagram showing the structure of a data acquisition and analysis system according to another embodiment of the present invention.
  • FIG. 4 is a flow chart showing the steps of a data acquisition and analysis method according to another embodiment of the present invention.
  • FIG. 5 is a flow chart showing the steps of a custom event processing method according to another embodiment of the present invention.
  • FIG. 6 is a flow chart showing the steps of a test method of a target application according to another embodiment of the present invention.
  • FIG. 7 is a flow chart showing the steps of a transmission verification method according to another embodiment of the present invention.
  • FIG. 8 is a flow chart showing the steps of a data quality verification method according to another embodiment of the present invention.
  • Figure 9 shows a block diagram of a computing device for performing a data acquisition analysis method in accordance with the present invention.
  • Figure 10 illustrates a memory unit for holding or carrying program code implementing a data acquisition analysis method in accordance with the present invention.
  • Embodiments of the present invention provide a data collection and analysis system, which is versatile and can collect, analyze, and generate different service reports for various applications.
  • FIG. 1 a block diagram of a data acquisition and analysis system in accordance with one embodiment of the present invention is shown.
  • the data collection and analysis system includes: an acquisition control server 102, a data analysis server 104, a report generation server 106, and at least one data terminal 108, in which an acquisition tool is installed.
  • the collection control server 102 is configured to configure the collection configuration information to the data terminal and send the data to the corresponding data terminal, where the collection configuration information is used to control the collection of the collection parameters, and the data terminal 108 is used in the application.
  • the collection tool is invoked during operation, and the parameter information of the collection parameter is collected according to the collection configuration information; and the collected parameter information is uploaded to the data analysis server; the data analysis server 104 is configured to upload the collected parameters.
  • the data analysis result is obtained, and the data analysis result is sent to the report generation server.
  • the report generation server 106 is configured to generate a data report according to the data analysis result.
  • FIG. 2 there is shown a flow chart of the steps of a data acquisition analysis method in accordance with one embodiment of the present invention.
  • Step 202 The collection control server configures the collection configuration information to the data terminal and sends the configuration information to the corresponding data terminal.
  • Step 204 The data terminal invokes the collection tool when the application is running, and collects parameter information of the collected parameter according to the collected configuration information.
  • Step 206 The data terminal uploads the collected parameter information to the collection control server.
  • Step 208 The data analysis server analyzes the uploaded parameter information of each collected parameter, and obtains a data analysis result.
  • Step 210 The data analysis server sends the data analysis result to the report generation server.
  • Step 212 The report generation server generates a data report according to the data analysis result.
  • the application of the embodiment of the present invention refers to an application installed on the terminal, such as a mobile phone assistant, a browser, a security guard, a player, and the like.
  • Collection tools include the Software Development Kit (SDK).
  • SDK Software Development Kit
  • the collection control server can configure the collection parameters of each application and collect configuration information to generate control information of the collection tool.
  • the collection tool may collect parameter information of each collection parameter in the application according to the collection configuration information.
  • the collection configuration information is used to control collection and upload of the collection parameters.
  • the acquisition parameters include static parameters and/or dynamic parameters.
  • the static parameter may be used to collect static parameter information corresponding to the application; the dynamic parameter may be used to collect dynamic parameter information when the application is running.
  • the collection tool is invoked.
  • the collection tool may collect parameter information of each collection parameter in the application according to the collection configuration information.
  • the collection tool determines whether the upload condition is met. If yes, the collection tool compresses the parameter information according to the preset algorithm, and then uploads the data to the data analysis server.
  • the data analysis server analyzes the parameter information corresponding to each of the uploaded collection parameters, such as performing statistics on the parameter information corresponding to each collection parameter, thereby obtaining the data analysis result, and then sending the data analysis result to the report generation server,
  • the report generation server generates a data report by configuring readable attribute information for each logical data analysis result according to the data analysis result, so that the application usage can be determined based on the data report. Therefore, the collection tool can collect the parameter information of each collection parameter in the data terminal of the application operation, and after uploading to the data analysis server, perform data analysis according to the statistical requirement, thereby generating a data report of the corresponding service, and obtaining the user requirement through the data report. , habits, etc., to adjust the business, more in line with user needs.
  • the collection control server 102 includes a cloud control server 1022.
  • the cloud control server 1022 is configured to determine an acquisition parameter of the application according to an application type, where the collection parameter includes: a dynamic parameter and/or a static parameter;
  • the terminal is configured to collect configuration information, where the collection configuration information is used to control the collection and uploading of the collection parameters, and the collection parameters and the collection configuration information are sent to the data terminal where the application is located.
  • the cloud control server 1022 is configured to configure an upload condition and/or a sampling condition of the parameter information, and add an upload condition and/or a sampling condition to the collection configuration information; and configure a collection threshold corresponding to the memory of the terminal where the application is located, The collection threshold is added to the collection configuration information; the uploading interval corresponding to the network type of the terminal where the application is located is configured, and the uploading interval is added to the collection configuration information; and the cache threshold of the terminal where the application is located is configured, The cache threshold is added to the acquisition configuration information; and the priority of the protocol upload sequence, the sampling rate, and the sampling parameter are configured in the collection configuration information.
  • the data terminal 108 is configured to collect the parameter information of the static parameter of the application according to the collection configuration information by using the collection tool, and determine the dynamic parameter to be collected according to the collection configuration information by using the collection tool, according to the operation of the application.
  • the parameter information of the dynamic parameter is collected in real time.
  • the data terminal 108 is configured to: when the interface switching of the application is detected, record the interface before the switching as the previous page, record the switched interface as the current page, and add the previous page and the current page to the application.
  • the interface startup information when the button of the application is detected, the button number corresponding to the button is acquired, the number of clicks corresponding to the button number is updated, and the button number and the click number are added to the button click information. Collecting the running time related to the application as duration information, wherein the duration information Including: interface runtime and application runtime.
  • the application can be used to collect and upload the parameter information in the data terminal to generate a corresponding report.
  • the specific steps are as follows:
  • FIG. 4 there is shown a flow chart of the steps of a data acquisition and analysis method in accordance with another embodiment of the present invention.
  • Step 402 The collection control server determines an acquisition parameter of the application according to an application type.
  • the cloud server can configure acquisition parameters of each application and collect configuration information to generate control information of the collection tool.
  • the cloud server can control the data collection of each application by the terminal where the application is located by using the control information of the collection tool.
  • the collection parameters include: dynamic parameters and/or static parameters.
  • the parameter information of the static parameter includes at least one of the following: a terminal identifier, a version information, a channel information, a network type, and resolution information.
  • the parameter information of the dynamic parameter includes at least one of the following: an application interface startup information, and a button click information.
  • the static parameter can be used to collect the static parameter information corresponding to the application, such as the identification information, version information, channel information, network type, and resolution information of the terminal where the application is located.
  • the identifier information of the terminal may be used to identify the terminal where the application is located. For example, the M2 value may be calculated based on information such as the hardware of the terminal.
  • the M2 value may be calculated by using a serial number, a model, a version information, etc., and the M2 value can identify different
  • the terminal does not disclose the hardware information of the terminal to ensure the privacy of the user; the version information can be used to record the version corresponding to the application, such as the operating system version of the terminal, the software version or the software development kit version, etc.; the channel information can be used.
  • the network device is used to determine the network type of the terminal where the application is connected; the resolution information is used to determine the display resolution of the terminal where the application is located, and the display resolution may be referred to as a screen resolution.
  • the dynamic parameter can be used to collect parameter information generated by the application during the running process, such as application interface startup information, button click information, duration information, abnormal information, and session information.
  • the application interface startup information is used to record the page before the switching of the interface of the application and the interface after the switching;
  • the button click information is used to record the button number corresponding to each button in the application and the number of clicks corresponding to the button number;
  • the duration information is determined according to the startup time point and the shutdown time point, such as the application running time and the interface running time;
  • the abnormal information is used to record the information of the application running abnormality, such as the time point or the number of times the application fails to start;
  • the session information is used to record the application.
  • the message of the session class generated at runtime.
  • Step 404 The collection control server collects configuration information according to the configuration of the application and the terminal where the application is located.
  • the step of collecting configuration information according to the application and the terminal where the application is configured includes at least one of the following: an upload condition and/or a sampling condition of the configuration parameter information, and an upload condition and/or a sampling condition are added to the collection configuration information;
  • the upload interval of the network type corresponding to the terminal where the application is located is added, and the upload interval is added to the collection configuration information;
  • the terminal where the application is located corresponds to a cache threshold, and the cache threshold is added to the collection configuration information.
  • the priority of the protocol upload sequence, the sampling rate, and the sampling parameter are configured in the collection configuration information.
  • Step 406 The collection control server sends the collection parameters and the collection configuration information to the data terminal where the application is located.
  • the collection configuration information is used to control collection and upload of the collection parameters.
  • the application and the terminal where the application is located are collected according to the collection configuration information, and the collected data needs to be uploaded according to the collection configuration information. Therefore, the cloud server can set the collection configuration information according to the application to be collected and the terminal where the application is located, so that the terminal of each application or application can collect and upload the collection parameters according to the collection configuration information.
  • the uploading condition is used to control uploading of the parameter information according to the uploading success flag bit
  • the sampling condition is used to control the collection of the parameter information according to the test identifier.
  • the cloud server can set the sampling condition and add the sampling condition to the collection configuration information, so that the terminal where the application is located collects data according to the sampling condition in the collection configuration information. Specifically, by detecting, it can be determined whether the data of the application meets the sampling condition. For example, when the application has the test identifier, all the parameter information of the application meets the sampling condition, and the collection tool acquires the parameter information of each static parameter and the parameter information of the dynamic parameter in the application. For example, in the process of maintaining the application, the developer constantly updates the version of the application according to the requirements, and some of the versions are beta versions of the beta version. After the test is passed, the official version will be launched, and the test logo can be configured for the beta version of the application.
  • the collected parameter information can also be cached locally.
  • the parameter information satisfies the upload condition in the sampling configuration information
  • the parameter information is uploaded.
  • the cloud server may set an uploading condition of each parameter information by setting a priority or an uploading interval of each parameter information, and save the uploading condition to the collecting configuration information.
  • the terminal where the application is located may compress the parameter information and upload it to the cloud server, that is, upload the parameter information that satisfies the uploading condition.
  • the cloud server can set upload conditions for different parameter information, and add to the collection configuration information, and then the uploading of each parameter information can be controlled by the upload condition.
  • the collection threshold of the memory corresponding to the terminal where the application is located is added, and the collection threshold is added to the collection configuration information.
  • the collection threshold is used to determine the collected acquisition parameters according to the memory information of the terminal.
  • some terminals have a large memory such as 2G in memory, and some terminals have small memory. In order to reduce the impact of data collection on the terminal, the terminal can run the application normally.
  • the server can collect the threshold according to the memory setting of the terminal where the application is located, and add the collection threshold to the collection configuration information, so that the collected collection parameters can be controlled according to the memory information of the terminal. For example, if the collection threshold is 50M, the memory of the terminal can be limited to 50M. You can limit the collection of the collection parameters without affecting the usage of the terminal. If no dynamic parameters are collected, only static parameters such as the terminal identifier can be uploaded once. As a specific example of the present invention, when the memory information of the terminal where the application is located does not reach the collection threshold, the parameter information corresponding to the dynamic parameter of the application on the terminal may not be collected, and the parameter information corresponding to the static parameter may be collected, such as the collection terminal.
  • the parameter information of each collection parameter in the application is collected according to the collection configuration information, such as the parameter information of the static parameter and the parameter information of the dynamic parameter in the application.
  • the uploading interval of the network type corresponding to the terminal where the application is located is configured, and the uploading interval is added to the collection configuration information.
  • the uploading interval is an interval of timing uploading, such as 5 minutes, 10 minutes, and the like. In essence, data is transmitted at different rates through different types of networks. Therefore, the upload interval of the network type corresponding to the terminal where the application is located can be set according to the data transmission rate corresponding to the network type, and the upload interval of different network types is added to the collection configuration information.
  • the uploading interval corresponding to the network type may be selected according to the type of the network to which the terminal is located, so that each application can upload the parameter information of the collected parameter according to the data transmission rate of the currently connected network, so that the application can be fully utilized. Network resources, and guarantee the upload rate of parameter information.
  • the upload interval may be determined according to the network type and the priority of the collection parameter itself.
  • the cache threshold of the terminal where the application is located is configured, and the cache threshold is added to the collection configuration information. The cache threshold is used to determine the amount of data of the cached parameter information at the terminal. Specifically, the performance of different terminals is different, so the amount of data of parameter information that can be cached is different.
  • the cache threshold corresponding to the terminal can be set according to the amount of data that can be cached by the terminal, and the cache threshold is added to the collection configuration information.
  • the cloud server may control the terminal to delete the parameter information of the cache time before the cached parameter information does not exceed the cache threshold, thereby ensuring the efficiency of data collection. For example, if the cache threshold is set to 2M, and the local cache is determined to exceed 2M when the collection parameter information is to be stored, the parameter information may be deleted from the front in the order of time before the storage parameter is stored, and the storage parameter does not exceed 2M.
  • the priority of the protocol upload order, the sampling rate, and the sampling parameters are configured in the collection configuration information.
  • the protocol uploading sequence is used to determine an uploading protocol of the collecting parameter, where the sampling rate is used to determine the sampled terminal, and the priority is used to determine the number of times the parameter information is collected and the uploading interval.
  • the cloud server may perform statistics on the network transmission protocol used by the network, determine the priority of each network transmission protocol, and generate a protocol uploading sequence.
  • the protocol uploading sequence may be as follows: 1. Transmission Control Protocol (TCP); 2. Wireless Application Protocol (WAP); 3. User Datagram Protocol (UDP); Real-time Transport Protocol (RTP); 5. Hypertext Transfer Protocol (HTTP).
  • TCP Transmission Control Protocol
  • WAP Wireless Application Protocol
  • UDP User Datagram Protocol
  • RTP Real-time Transport Protocol
  • HTTP Hypertext Transfer Protocol
  • the sequence number corresponding to the network transmission protocol may represent the priority of the network transmission protocol.
  • the protocol uploading sequence of the above example the parameter information is preferably uploaded using the highest priority TCP protocol.
  • the WAP protocol is used for uploading, that is, the uploading protocol is selected according to the priority order, so as to avoid the problem that the parameter information upload fails due to the terminal and the application not supporting the use of a certain network transmission protocol, and the uploading efficiency and uploading of the parameter information are guaranteed. quality.
  • the cloud server can perform statistics on the terminal where the application is located to determine the sampling rate.
  • the sampling rate the number of terminals to be sampled can be determined, and the terminal where the application is located can be randomly selected according to the determined number of terminals, and the selected terminal is used as the target terminal, and the terminal determined as the sampling.
  • the terminal identifier of each terminal may be calculated by a preset algorithm, and the target terminal is selected, for example, the target terminal is determined by the last digit or two digits of the terminal identifier.
  • the cloud server may determine the sampled terminal by using other methods, for example, by performing modulo or suffixing on the terminal identifier to determine the sampled terminal, which is not limited in this embodiment of the present invention.
  • the cloud server can control the number of terminals to be sampled, avoiding the problem of large data processing pressure of the cloud server by sampling a large number of terminals, ensuring the sampling efficiency of the data, and thereby accelerating the progress of data collection.
  • the cloud server can also control the collection times and upload intervals of the parameter information of the collection parameters by setting the priority of each collection parameter.
  • the information of the collection parameter may be collected according to the collection times corresponding to the determined priority, and uploaded according to the upload interval corresponding to the priority.
  • the parameter information of the collection parameter such as the priority of the acquisition parameter I is A level
  • the collection number of the parameter information of the collection parameter I is 1000 times
  • the upload interval is 20 minutes, that is, every 20 minutes, the uploading is performed once every 20 minutes.
  • the parameter information of parameter I is collected.
  • Priority of acquisition parameters Number of times the parameter information is collected Parameter information upload interval Class A 1000 times 20 minutes Class B 700 times 1 hour Class C 300 times 3 hours Class D 10 times 12 hours Class F 1 time 24 hours
  • the priority of the collection parameter and the number of times of the parameter information collection and the upload interval corresponding to each priority may be set in advance according to the service requirement or the characteristics of the collected data, so that the cloud server can pass the priority of the collection parameter.
  • Step 408 Collect the parameter information of the static parameter of the application according to the collection configuration information by using the collection tool.
  • Step 410 The collection tool determines the dynamic parameter to be collected according to the collection configuration information, and collects the parameter information of the dynamic parameter in real time according to the running of the application.
  • the collection parameters may be collected during the application running, including: determining the number of times the parameter information of the collection parameter is collected according to the priority of each collection parameter in the configuration information. And upload interval. Acquiring the memory information of the mobile terminal where the application is located, and determining whether the memory information reaches the collection threshold; when the memory information does not reach the collection threshold, collecting the collection parameters of the static parameter and uploading the parameters to the server to end the collection step
  • the application runtime invokes the collection tool to collect parameter information of each acquisition parameter in the application, where the collection parameter includes a static parameter and/or a dynamic parameter.
  • some terminals have a large memory such as 2G in memory, and some terminals have small memory.
  • the terminal can run the application normally.
  • the server can collect the threshold according to the memory setting of the terminal where the application is located, and add the collection threshold to the collection configuration information, so that the collected collection parameters can be controlled according to the memory information of the terminal.
  • the memory information of the terminal where the application is located does not reach the collection threshold. If the value of the terminal is less than 50M, the value of the parameter corresponding to the static parameter may be collected, such as the identifier of the collection terminal.
  • the collection of the collection parameters can be limited. If the dynamic parameters are not collected, only the static parameters such as the terminal identifier can be uploaded once.
  • the step of collecting parameter information of the dynamic parameter in real time according to the running of the application includes at least one of the following:
  • the interface before the record switching is the previous page
  • the interface after the record switching is the current page
  • the previous page and the current page are added to the application interface startup information. That is, when the user accesses the record in the application interface, in order to ensure the continuity of the access, the previous access interface may be recorded while recording the current interface, so that each interface record is the current page + the previous page, of course, if it is the home page, The previous page is empty.
  • the application can include a number of buttons, each button can be numbered to identify a different button, so that when the button is clicked, the button number and the number of clicks are recorded, that is, the number of clicks is initially 0, and each click is incremented by one.
  • the application can be run in a multi-process mode.
  • the application starts with the main process record start time point, and the last process records the end time point.
  • the main process may not be fully started.
  • the running time of the record may be negative, or the time between the start time of the last start and the end time of the current close will be calculated, and the length of time will be too long, resulting in a record of the duration. accurate.
  • the running time related to the application is collected as the duration information, where the duration information includes: an interface running time and an application running time.
  • the collecting the running time related to the application as the duration information includes: the main process of the application records a start time point and an end time point, and the running time is recorded according to the start time point and the end time point, and the running is performed. Time is used as duration information. That is, in the embodiment, for the application running in multiple processes, the records of the start and end times are recorded from the main process, that is, the start time point of the main process record is used when the application starts, and the application time is closed by the end time of the main process record. The difference between the end time point and the start time point is the running time, ensuring accurate recording of the running time.
  • the collected parameter information may be cached locally.
  • a cache threshold may be configured, where the cache threshold is used to determine parameter information cached in the terminal.
  • the amount of data Specifically, the performance of different terminals is different, so the amount of data of parameter information that can be cached is different.
  • the cache threshold corresponding to the terminal can be set according to the amount of data that can be cached by the terminal, and the cache threshold is added to the collection configuration information.
  • the cloud server may control the terminal to delete the parameter information of the cache time before the cached parameter information does not exceed the cache threshold, thereby ensuring the efficiency of data collection.
  • the cache threshold is set to 2M
  • the local cache is determined to exceed 2M when the collection parameter information is to be stored
  • the parameter information may be deleted from the front in the order of time before the storage parameter is stored, and the storage parameter does not exceed 2M.
  • Step 412 The data terminal uploads the collected parameter information to the collection control server.
  • the upload interval processing is determined according to the parameter priority.
  • the upload interval can be determined according to the current network type to determine whether the current time meets the upload interval.
  • you can also configure the upload threshold that is, the threshold for each data upload, and check whether the current cached parameter information is greater than the upload threshold.
  • the cloud server may set an uploading condition of each parameter information by setting a priority or an uploading interval of each parameter information, and save the uploading condition to the collecting configuration information.
  • the embodiment of the present invention uploads parameter information to the server through data packet uploading. If a packet contains too much data, it is easy to cause the upload to fail.
  • the size of each upload data packet is limited by uploading a threshold.
  • the packetization measure is adopted, that is, the parameter information is divided into at least two data packets. Subcontract uploading can avoid upload failures caused by excessive data volume. For example, when the upload threshold is 100k, when the parameter information to be uploaded is greater than 100k, the parameter information can be divided into at least two.
  • the data packet for example, the parameter information to be uploaded is 250k, and can be divided into three data packets 100k, 100k and 50k. Then, the parameter information is compression-encoded according to a preset algorithm to generate a parameter data packet.
  • the uploading protocol is selected according to the uploading sequence of the preset protocol, and the parameter information is uploaded to the server according to the uploading protocol.
  • uploading failures may occur when parameter information is uploaded to the server. In this case, you may need to upload the parameter information that has not been uploaded. In order to distinguish the uploading of the parameter information, you can configure the upload success flag according to the upload result.
  • the upload result is identified by uploading a success flag bit.
  • the upload success flag is successfully set.
  • the mobile terminal where the application is located is unlocked, the upload success flag is detected.
  • the upload success flag is detected.
  • the upload success is marked as failed, the current network type is obtained, and it is determined whether the network type allows uploading data.
  • the current network type can be obtained, and whether the network type is allowed to upload data, such as whether the network is connected or not, is determined to be 2G, 3G, 4G or WIFI and so on. If yes, that is, when the network type allows uploading data, the parameter information that is not uploaded is uploaded to the server.
  • the upload success flag bit is detected, and the uploading of the parameter information is determined. If the upload success is marked as failed, the current network type is obtained, and it is determined whether the network type is allowed to upload data.
  • the step of uploading the unuploaded parameter information to the server is performed.
  • Step 414 The data analysis server analyzes the uploaded parameter information of each collected parameter, and obtains a data analysis result.
  • Step 416 The data analysis server sends the data analysis result to the report generation server.
  • Step 418 The report generation server generates a data report according to the data analysis result.
  • the data analysis server analyzes the parameter information corresponding to each of the uploaded collection parameters, such as performing statistics on the parameter information corresponding to each collection parameter, thereby obtaining the data analysis result, and then sending the data analysis result to the report generation server,
  • the report generation server generates a data report by configuring readable attribute information for each logical data analysis result according to the data analysis result, so that the application usage can be determined based on the data report.
  • the collection tool is called when the application is running; the collection tool collects the parameter information of each collection parameter in the application; determines the upload interval according to the current network type, determines whether the current time meets the upload interval; and when the current time reaches the upload interval, And compressing the parameter information according to a preset algorithm to generate a parameter data packet; uploading the parameter data packet to a server, so that the server performs data analysis according to the parameter information.
  • the report generation server 106 is configured to obtain service statistics information according to the customized management information of the configured application corresponding to the customized event, and generate a customized event report according to the customized management information and the service statistics.
  • statistics are collected on events, it is often only possible to perform statistics on common events.
  • General events can only be used for statistics on common features, while for some unique features, general events are used for statistics, and the statistical effect is not obvious.
  • each application on the terminal is customized according to the service that needs statistics, and a custom event is generated, and the customized management information is configured according to the custom event corresponding to each application. .
  • the report generation server 106 is further configured to: obtain an application path of the application from the service management page, and the customized event information of the application corresponding to the customized event, where the customized event information includes: an event Name and event attributes; use the application path and custom event information to generate custom management information.
  • the report generation server 106 is configured to determine an application according to the application path, and obtain, for each custom event, a logical name corresponding to the event name, and a logical attribute corresponding to the event attribute; Collecting parameters, determining service statistics information corresponding to the collection parameters according to the logical attributes; determining an event name corresponding to each collection parameter, and generating a customized event report of the application by using the event name and the service statistics information of the corresponding collection parameter.
  • the report generation server generates a data report according to the data analysis result, and the report generation server obtains service statistics information according to the customized management information of the configured application corresponding to the customized application; and according to the customized management information and service Statistics generate custom event reports.
  • the step of configuring the custom management information the application path of the application is obtained from the service management page, and the custom event information corresponding to the custom event is used by the application, where
  • the customized event information includes: an event name and an event attribute; and the customized management information is generated by using the application path and the customized event information.
  • the service statistics information is obtained, and the customized event report is generated according to the customized management information and the service statistics, including: determining an application according to the application path; Defining an event, obtaining a logical name corresponding to the event name, and a logical attribute corresponding to the event attribute; searching for the logical name corresponding to the collection parameter, determining, according to the logical attribute, service statistics information corresponding to the collection parameter; determining each Collect the event name corresponding to the parameter, and generate the application's custom event report by using the event name and the service statistics of the corresponding collection parameter.
  • FIG. 5 a flow chart of the steps of a custom event processing method in accordance with another embodiment of the present invention is shown.
  • Step 502 Feedback the service management page according to the configuration request.
  • Step 504 Obtain an application path of the application from the service management page, and apply custom event information corresponding to the customized event.
  • Step 506 Generate custom management information by using an application path and custom event information. Customize some unique events applied in the mobile terminal to enable statistics on some unique data in the application through custom events.
  • the corresponding custom event of the application is processed and fed back to the service management page, and the application path corresponding to the application is obtained from the service management page, for example, according to each custom event, which service belongs to it, and then according to Determining the service information, determining the application corresponding to the service, determining the application path corresponding to the application, and obtaining the customized event information corresponding to the custom event, wherein the customized event information includes: an event name and an event attribute,
  • the defined event includes at least one of the following: a traffic statistics event, a click event, and a selected event.
  • Event attributes include: how statistics are customized for events. Generate custom management information for application paths, event names for custom events, and event properties. For example, the statistics of the traffic consumption in a certain period of time, the traffic statistics event is set as a custom event; and the traffic of all the applications in the period is consumed; the corresponding application path of the application is obtained, and the time is found. Only the mobile assistant and the browser consume traffic, and the mobile assistant and the browser consume traffic during this period of time as custom management information.
  • Step 508 determining an application according to the application path.
  • Step 510 Acquire, for the application, the service statistics information corresponding to the event name and the event attribute of the respective defined event.
  • the application is determined according to the application path in the service management page, and the service statistics corresponding to the event name and the event attribute of each custom event are obtained from the application after the application path is determined.
  • the cloud server can send the collection parameters and collection configuration information to the terminal where the application resides.
  • the collection tool can call the SDK (Software Development Kit) to collect data for each application, that is, collect the collection parameters in the application according to the collection configuration information.
  • Parameter information wherein the acquisition parameters include static parameters and dynamic parameters.
  • the service statistics can be performed.
  • the obtaining the service statistics information according to the customized management information includes the following sub-steps: obtaining, for each custom event, a logical name corresponding to the event name, and a logical attribute corresponding to the event attribute. Find the logical parameter corresponding to the collection parameter, and determine the service statistics corresponding to the collection parameter according to the logical attribute.
  • the server performs data processing and data statistics according to the customized management information.
  • each custom event there is a corresponding event name and a corresponding logical name.
  • a button in the page triggers an event, and the middle button of the page is in turn
  • the number is specified, and the button trigger event is defined as a button event.
  • the event name is button 1
  • the corresponding logical name is button.
  • Each custom event has a logical attribute, and statistics are used to count each custom event.
  • a page button trigger event is defined as button event 1, which is triggered once, accumulated, and its final value is obtained. . According to each custom event, obtain the logical name corresponding to each custom event name, and the logical attribute corresponding to the event attribute.
  • the so-called logical name and logical attribute are actually a representation of the custom event in the underlying code, the background server
  • the data is counted by logical name and logical attribute.
  • Each application corresponds to a service, and each service has multiple custom events, and the collection parameters corresponding to the custom event are searched according to the logical name of each event, according to the logic.
  • the attribute is collected and the statistics of the collected parameters are statistically obtained, and the service statistics corresponding to the customized event are obtained.
  • the service statistics corresponding to each custom event are determined. For example, if a page includes three paragraphs of text, it is necessary to count the number of times each of them is selected and copied; define it as a service, and then define three custom events, such as the first paragraph is selected and copied.
  • the event is defined as selected, the second segment is selected and the copied event is defined as selected 2, the third segment is selected and the copied event is defined as selected 3; the logical name selected by the custom event is check; the collection parameters include Defines the number of executions selected by the event, that is, the number of times each piece of text is selected and copied. For example, in a certain period of time, the first paragraph of text is selected and the number of times of copying is 0, the second paragraph of text is selected and the number of times of copying is 1, the third paragraph of text is selected and the number of times of copying is 6.
  • the event name is selected, the logical name is check1, and the service statistics corresponding to the collection parameter are 0; the event name is selected 2, the logical name is check2, the service statistics corresponding to the collection parameter is 1; and the event name is selected.
  • the logical name is check3, and the service statistics corresponding to the collection parameters are 6. Determine the logical name corresponding to the event name according to the event name, determine the logical attribute corresponding to the event attribute according to the event attribute, and then determine the corresponding collection parameter and service statistics information according to the logical name and the logical attribute; the service corresponding to each custom event can be clearly determined. Statistics.
  • Step 512 Determine an event name corresponding to each collection parameter, and generate a customized event report of the application by using the event name and the service statistics information of the corresponding collection parameter.
  • the data analysis server receives the parameter information, and determines the event name, the logical name, and the service statistics corresponding to the collection parameter corresponding to the custom event according to the parameter information; and the service corresponding to the event name, the logical name, and the collection parameter of the custom event.
  • the statistics information is sent to the report generation server; the report generation server determines a plurality of custom events corresponding to the service according to each service, and then determines, according to the defined event, an event name, a logical name, and an acquisition parameter corresponding to the custom event.
  • the service statistics information is generated according to the event name corresponding to each custom event and the service statistics corresponding to the collection parameter, where the service statistics information is the collection data corresponding to the collection parameters, and the collected data is accumulated in an accumulated manner. statistics.
  • Step 514 Obtain service statistics information corresponding to the general event information, and add the general event corresponding service statistics information to the custom event report.
  • An optional embodiment of the present invention only needs to obtain the corresponding service statistics information according to the event name corresponding to the common event for the general event, and add the event name of the common event and the service statistics corresponding to the common event to the custom event.
  • the embodiment of the present invention obtains the application path of the application from the service management interface, and applies the customized event information corresponding to the customized event; determines the application according to the application path, and obtains the service corresponding to the event name and the event attribute of the respective defined event for the application.
  • the statistics information is generated according to the service statistics information.
  • Each application corresponds to a service.
  • a service includes multiple custom events.
  • the custom event report is generated according to the service statistics corresponding to each custom event.
  • the service statistics corresponding to each custom event can be displayed in an intuitive manner.
  • the statistical method is more representative, and some unique events can be counted, and the statistical effect is enhanced.
  • the event is added to the custom event report, and the combined statistics of the common event and the custom event are implemented, so that the user can obtain the statistics of various events required from a business report to meet various user requirements.
  • the collection control server is further configured to configure test information corresponding to the test device of the target application; the data analysis server is further configured to analyze parameter information corresponding to the uploaded target application; and determine the parameter information.
  • the collection control server 102 includes: a development configuration server 1024.
  • the development configuration server is configured to: after the target application completes registration, configure a test identifier of the at least one test device for the target application; Configuring the collection parameter to be tested, and using the test identifier and the collection parameter as test information; the data analysis server is further configured to obtain parameter information of the target application corresponding collection parameter from the uploaded parameter information; The terminal parameter corresponds to the terminal identifier; determining whether the terminal identifier is the same as the test identifier in the test device table. When the terminal identifier is the same as any one of the test identifiers in the test device table, determining that the parameter information belongs to the test device.
  • the method further includes: the collection control server configuring test information corresponding to the test device of the target application; the data analysis server analyzing parameter information corresponding to the uploaded target application; determining that the parameter information belongs to the test device Real-time participation in the test equipment The number information is analyzed; when the report generation server determines that the parameter information belongs to the test device, the data report is generated in real time according to the data analysis result.
  • Configuring the test information of the test device corresponding to the target application including: after the target application is registered, configuring a test identifier of the at least one test device for the target application; configuring the collection parameter to be tested for the target application, The test identifier and the collection parameter are used as the test information.
  • the method further includes: the data analysis server obtains parameter information of the target application corresponding to the collection parameter from the uploaded parameter information; and the terminal parameter corresponding to the terminal identifier in the parameter information; Whether the terminal identifier is the same as the test identifier in the test device table. When the terminal identifier is the same as any test identifier in the test device table, the parameter information is determined to belong to the test device.
  • the steps for configuring the target application and testing the configuration server 1024 are as follows:
  • FIG. 6 a flow chart of the steps of a test method for a target application in accordance with another embodiment of the present invention is shown.
  • Step 602 Receive a registration request of the target application.
  • Step 604 Assign an application identifier to the target application according to the registration request.
  • the corresponding collection parameters can be collected during the application running process to determine the running status of the application.
  • each application uses a common collection tool, so it is possible to pre-test the application calling collection tool for parameter collection.
  • the application that needs to be tested can be targeted.
  • the test of the target application usually needs to know the test result in real time, so the test device can be configured for the target application, and the collection result of the test device can be analyzed in real time.
  • the target application is registered, and a registration request may be sent.
  • the registration request carries relevant information of the target application, and the server obtains an application identifier according to the registration request for the target application, so that an application is uniquely identified by the application identifier.
  • the collection tool that is, the SDK required to collect the data, can be sent to the target application, so that the target application can call the collection tool to collect at runtime.
  • Step 606 Configure a test identifier of the at least one test device for the target application.
  • Step 608 Configure the collection parameters to be tested for the target application, and use the test identifier and the collection parameters as test information.
  • the terminal identifier of the test device such as the M2 value
  • the development configuration server configures the terminal identifier as the test identifier of the test device, so that the test identifier of at least one test device can be configured for the target application.
  • the acquisition parameters to be tested by the target application that is, to determine the acquisition parameters that need to be collected from the target application during the test, thereby obtaining the parameter information of the corresponding acquisition parameters. Test identification and acquisition parameters are used as test information.
  • the target application can call the collection tool to collect, that is, the target application is a test application that adds the collection tool.
  • the cloud server generates the collection configuration information according to the collection parameters and the corresponding collection policy, and sends the collection configuration information to each data terminal, so that each corresponding application can report the parameter information corresponding to the collection parameter according to the collection configuration information, that is, the cloud server can
  • the terminal where the application is located collects the parameter information corresponding to each collection parameter by collecting parameters and collecting configuration information, and automatically completes the application data collection.
  • the terminal device invokes the collection tool according to the collection configuration information, such as calling the SDK to collect parameters of the target application, and uploading the collected parameter information to the data analysis server.
  • Step 610 Obtain parameter information of the target application corresponding collection parameter from the uploaded parameter information.
  • Step 612 Analyze the source of the parameter information to determine whether the parameter information belongs to the test device.
  • the data analysis server obtains the parameter information of the corresponding acquisition parameter of the target application from the uploaded parameter information, and then searches for the terminal identifier corresponding to the terminal parameter from the parameter information, and compares the terminal identifier with the test identifier in the pre-configured test device table to determine Whether the terminal identifier is the same as the test identifier in the test device table.
  • the parameter information is determined to belong to the test device, that is, the parameter information is collected from the test device. .
  • determining whether the parameter information belongs to the test device may include: determining, by using the terminal identifier corresponding to the terminal parameter in the parameter information, the source of the parameter information; determining whether the terminal identifier is the same as the test identifier in the test device table.
  • the judgment parameter information belongs to the test device.
  • the test identifier of each test device may be saved in the server, and a corresponding test device table is generated to facilitate determining the test device during subsequent analysis. The terminal identifier corresponding to the terminal parameter is searched from the parameter information, and the terminal identifier can identify the source of the parameter information, that is, which terminal is collected.
  • the terminal identifier Compare the terminal identifier with the test identifier in the test equipment table. It is determined whether the terminal identifier is the same as the test identifier in the test device table, that is, whether the current parameter information belongs to the test device. If the terminal identifier is the same as any one of the test identifiers in the test device table, the parameter information is determined to belong to the test device, that is, the parameter information is collected from the test device. On the other hand, if the terminal identifier is different from the test identifier in the test device table, the parameter information is not collected from the test device, that is, the parameter information does not belong to the test device, and the report can be generated according to the general processing flow, without errors. Real-time statistics.
  • the terminal identifier is used to determine whether the uploaded parameter information is from the test device. If yes, the parameter information belongs to the test device, and step 614 is performed; if not, the parameter information does not belong to the test device, and the process is ended without real-time statistics.
  • Step 614 Perform statistics on the corresponding parameter information according to a statistical manner for each parameter information belonging to the test device.
  • Step 616 Generate a test report according to the statistical result corresponding to each parameter information. After determining that the parameter information belongs to the test device, the parameter information of the test device is processed, and the corresponding parameter information is statistically calculated according to a statistical manner. If the number of times a button is triggered on a page, all the buttons on the page are numbered, that is, each button has its own corresponding number. The button is triggered once, and the number of times the button is triggered is superimposed. For example, the button numbered 1 triggers. 3 times, that is, the corresponding statistics data includes button 1 trigger 3 times. Another example is the switching of the page, first determining the current page information, and then looking up to determine the page information of the previous page.
  • the previous page is recorded as empty, and then the page information of the next page is searched, and then sequentially statistics.
  • the statistical information corresponding to each parameter one or more collection parameters corresponding to each target application are determined; and the statistical data corresponding to each parameter is determined, and the test data corresponding to each parameter and the parameter is generated according to the applied classification.
  • Step 618 Analyze the running result of the collection tool in the target application according to the test report.
  • An acquisition tool is added to the target application to assist the target application in collecting the test information through the acquisition tool.
  • the test report can also be used to analyze the running result of the collection tool in the target application.
  • the running result includes at least one of the following: data collection results, data upload results, report generation results, and application running results.
  • the data collection result is the analysis of the collection parameters of the collection tool in the test equipment through the test report, such as whether it can be collected normally.
  • the result of data uploading is to analyze the parameter information of the collection tool in the test device through the test report, such as whether the upload is normal.
  • the report generation result is that after the test report analyzes the uploading parameter information of the collecting tool in the test device, the report is generated, that is, the parameter information can be generated by the normal statistical parameter information.
  • the application running result is to analyze the running status of the target application when the test device collects and uploads through the collection tool through the test report, such as whether the target application is running normally, whether there is a problem such as a stuck, a flashback, or a crash.
  • the target application is installed in the test device, and the application identifier is allocated to the target application, the test identifier is configured for the test device, and the collection parameter and the test identifier of the target application are used as the test information upload server, and the server uploads by analyzing multiple devices.
  • the data information is determined according to the test identifier, and the test information belongs to the test device, and the parameter information is statistically calculated, and the test report is generated according to the statistical result, and the test report can clearly reflect the current target application in the test device operation;
  • the test application auxiliary parameter information of the acquisition tool is installed in the target application, and the test report is used to analyze the operation of the collection tool in the target application.
  • the data analysis server 104 is further configured to search for verification parameter corresponding verification information from the parameter information, and analyze the verification information to determine a hijacking verification result.
  • the data analysis server 104 is configured to determine a parameter identifier of the verification parameter, search for the parameter identifier corresponding to the verification information from the parameter information, and detect whether the verification information is target information; If the target information is not used, it is determined that the traffic has a hijacking abnormality, and the hijacking abnormality is recorded in the hijacking verification result; if the verification information is the target information, it is determined that the data transmission is normal, and the recording transmission is normal in the hijacking verification result.
  • the method further includes: the data analysis server searching for the verification parameter corresponding to the verification information from the parameter information; and analyzing the verification information to determine the hijacking verification result.
  • the information is the target information, and it is determined that the data transmission is normal, and the recording transmission is normal in the hijacking verification result.
  • Step 702 Receive uploaded parameter information.
  • the parameter information is collected for each acquisition parameter when the application is running.
  • the acquisition control server determines the application to be collected based on the data that needs to be collected by each service platform.
  • the cloud server may determine the static parameters and the dynamic parameters of the application to be collected according to the application type, and set the collection configuration information, so that the terminals of the applications or applications can collect and upload the collected collections according to the collection configuration information.
  • the parameters correspond to the parameter information.
  • the uploaded parameter information may include: parameter information of the destination address returned by the host after the hijacking, and parameter information of the direct upload destination address.
  • the destination address belongs to the address list. In essence, the address list can be pre-set in the data analysis server.
  • the address list contains the IP addresses uploaded to the data analysis server, which is equivalent to the target address; and the address information corresponding to each IP address, which is equivalent to the target address information.
  • the target address can be used as the destination address.
  • the parameter information may be uploaded from the terminal where the application is located to the data analysis server. Therefore, the uploaded IP address can be recorded in the parameter information, and the parameter information of the upload destination address can be generated. For example, the last upload IP address is generated to generate LastIP information.
  • the parameter information of the upload destination address may include parameter information of directly uploading the destination address and parameter information of the destination address after the hijacking is returned.
  • Step 704 Determine a parameter identifier of the verification parameter, and search for the verification information corresponding to the parameter identifier from the parameter information.
  • the data analysis and analysis server may determine the verification parameter that needs to be verified according to the verification requirement. By determining the parameter identification of the verification parameter, the search can be performed according to the determined parameter identification. If the verification information corresponding to the determined parameter identifier is found from the parameter information, the found verification information may be extracted to detect whether the parameter information is intercepted during the transmission.
  • the analysis server B can extract the parameter information according to the IP address from the uploaded parameter information.
  • the LastIP information corresponding to the identifier of the IP address, and the LastIP information includes each IP address uploaded by the parameter information from the terminal A to the analysis server B, such as LastIP1, LastIP2, LastIP3, and the like.
  • Step 706 Detect whether the verification information is target information. If the verification information is not the target information, step 708 is performed; if the verification information is the target information, step 710 is performed.
  • the data analysis server may determine whether the parameter information is intercepted by detecting that the IP address information matches the target address information in the address list.
  • the foregoing step 706 may be to detect that the IP address information matches the target address information in the address list. Specifically, if the IP address corresponding to the IP address information extracted from the parameter information is not in the preset address list, that is, when the address list cannot find the target address corresponding to the extracted IP address information, the extracted IP is determined.
  • step 708 If the address information does not match the target address information in the address list, it may be determined that the parameter information is intercepted, and step 708 is performed. If the IP address corresponding to the IP address information extracted from the parameter information is in a preset list, that is, the IP address corresponding to the extracted IP address information can be found in the address list, the extracted IP address information and the address list are determined. If the target address information is matched, it can be determined that the parameter information is not intercepted, and step 710 is performed.
  • LastIP1, LastIP2, and LastIP3 can be found in the preset address list, it can be determined that the parameter information is not intercepted, and step 710 is performed; if the LastIP1 cannot be found in the address list, it can be determined that the parameter information is If the illegal terminal or the malicious program intercepts, and the IP address corresponding to the illegal terminal or the malicious program that intercepts the parameter information is LastIP1, step 708 is performed.
  • Step 708 determining that the traffic has a hijacking abnormality, and recording the hijacking abnormality in the hijacking verification result.
  • the data analysis server may determine that the traffic hijacking phenomenon exists in the parameter information uploading process, and the terminal parameter or the malicious program that may be intercepted by the intercepted parameter information may be tampered to determine the parameter.
  • the traffic corresponding to the information upload has a hijacking abnormality.
  • the hijacking abnormality of the parameter information is recorded in the hijacking verification result.
  • the parameter information is intercepted by the terminal of LastIP1 or the LastIP1 is recorded as an illegal IP address and a malicious IP address.
  • the data analysis results are generated, thereby avoiding the use of intercepted parameter information for statistics, and ensuring the accuracy of the data analysis results.
  • step 710 it is determined that the data transmission is normal, and the recording transmission is normal in the hijacking verification result.
  • the parameter information is not intercepted, it can be determined that the data transmission is normal, and the transmission of the parameter information in the hijacking verification result is normal, thereby ensuring the accuracy of the data analysis result.
  • the data analysis server 104 is further configured to search for storage parameter corresponding storage information from the parameter information, and determine a quality check result according to the stored information.
  • the data analysis server 104 is configured to determine a storage identifier of the storage parameter, search for the storage identifier corresponding storage information from the parameter information, and search for at least one terminal identifier according to the storage information; and when the storage information corresponds to one terminal Identifying, generating a quality-qualified quality check result; when the stored information corresponds to more than one terminal identifier, searching for the terminal identifier corresponding to the successful uploading information; sorting the terminal identifiers corresponding to the uploading success information, and determining the When there is no redundancy in the upload success information, the data is judged to be abnormal, and a quality check result of the data abnormality is generated.
  • the data terminal such as a mobile phone or a tablet computer collects application data through the collection tool, after collecting the parameter information corresponding to each acquisition parameter, the collected parameter information is uploaded, so that the data analysis is performed.
  • the server can receive the uploaded parameter information.
  • the parameter information may include storage information, verification information, information corresponding to the acquisition parameters, and the like.
  • the storage letter can be used to detect data quality and determine quality verification results, such as Secure Digital Memory Card information (SD card information), Multimedia Media Card (MMC) information, and SM card ( Smart Media Card information, CF card (Compact Flash Card) information, etc.; verification information can be used to analyze data transmission conditions and determine hijacking verification results, such as IP (Internet Protocol) address information.
  • SD card information Secure Digital Memory Card information
  • MMC Multimedia Media Card
  • SM card Smart Media Card information, CF card (Compact Flash Card) information, etc.
  • verification information can be used to analyze data transmission conditions and determine hijacking verification results, such as IP (Internet Protocol) address information.
  • the data analysis server may analyze the parameter information, and search for the storage parameter corresponding storage information from the parameter information.
  • preset storage parameters such as preset SD card parameters, may be set according to detection requirements.
  • the data analysis server uses the stored information to perform data quality detection on the parameter information, to determine whether the collected parameter information has data fraud, that is, to determine the accuracy of the collected parameter information, and determine the quality check result.
  • the method further includes: the data analysis server searching for the storage parameter corresponding storage information from the parameter information; detecting the data quality according to the stored information, and determining a quality check result.
  • the quality of the collected data can also be checked.
  • the quality verification steps for the transmitted data are as follows:
  • Step 802 Receive uploaded parameter information.
  • the parameter information is collected for each acquisition parameter when the application is running.
  • the acquisition control server determines the application to be collected based on the data that needs to be collected by each service platform.
  • the cloud server may determine the static parameters and the dynamic parameters of the application to be collected according to the application type, and set the collection configuration information, so that the terminals of the applications or applications can collect and upload the collected collections according to the collection configuration information.
  • the parameters correspond to the parameter information.
  • Step 804 determining a storage identifier of the storage parameter, and searching for the storage identifier corresponding storage information from the parameter information.
  • the data analysis and analysis server may determine the storage information that needs to be extracted according to the requirements of the data quality detection, and perform data quality detection on the parameter information by using the extracted storage information to determine whether the uploaded parameter information is abnormal. Specifically, by determining the storage identifier of the storage parameter, the search may be performed according to the determined parameter identifier; if the storage information corresponding to the determined storage identifier is found from the parameter information, the found storage information may be extracted, so as to The uploaded parameter information is used to detect the data quality, such as determining whether the uploaded parameter information is fraudulent data.
  • the storage parameters set by the data analysis server can include The SD card parameter
  • the storage parameter corresponding storage information may include SD card information.
  • the data analysis server may search for the received parameter information according to the SD card identifier by determining the SD card identifier, and extract the SD card identifier corresponding to the SD card information from the parameter information. It should be noted that each SD card has a unique corresponding SD card identifier, and the SD card information can be used to find the SD card identifier corresponding to the SD card.
  • Step 806 Find at least one terminal identifier according to the stored information.
  • the data quality detection may be performed according to the stored information, and it is determined whether the collected parameter information has data fraud. Specifically, extracting identification information of each terminal from all the received parameter information, and determining whether the terminal identifier of the extracted identification information corresponds to the extracted storage information; if the terminal identifier corresponds to the extracted storage information, The extracted storage information is the storage information of the terminal, and the corresponding relationship between the extracted storage information and the terminal identifier of the terminal is established.
  • terminal A, terminal B, and terminal C are connected to the same SD card, labeled as SD card 1, and the SD card information of SD card 1 is labeled as SD card information 1; terminal D is connected to another The SD card is labeled as SD card 2, and the SD card information of the SD card 2 is marked as SD card information 2.
  • the data analysis server extracts the SD card information from the parameter information uploaded by the terminal A, the terminal B, or the terminal C, the extracted SD card information is the SD card information 1, and the SD card identifier is found according to the SD card information 1, and the terminal can be found.
  • the SD card identifier corresponding to A, the SD card identifier corresponding to the terminal B, and the SD card identifier corresponding to the terminal C is used to search for the uploaded parameter information, and the three SD card identifiers corresponding to the SD card information 1 can be obtained.
  • the data analysis server extracts the SD card information from the parameter information uploaded by the terminal D, the extracted SD card information is the SD card information 2, and the SD card identifier is found according to the SD card information 2, and the SD card identifier corresponding to the terminal D can be found. That is, according to the SD card information 2, the uploaded parameter information is searched, and one SD card identifier corresponding to the SD card information 2 can be obtained.
  • Step 808 The terminal identifier is used to detect data quality, and the quality check result is determined.
  • the data analysis server may perform statistics on the terminal identifier corresponding to the stored information, determine the number of the terminal identifier corresponding to the storage information, and determine whether the stored information belongs to a terminal based on the number of the terminal identifier, which is equivalent to determining the storage information. Whether the corresponding parameter information has data fraud. If the number of the terminal identifiers corresponding to the stored information is 1, the stored information belongs to one terminal. In the above example, the SD card information 2 belongs to the terminal D, and it can be determined that the collected data is normal, that is, the SD card information 2 is determined.
  • the parameter information is not falsified data, and the quality verification result that the quality of the data collection is qualified is generated. If the number of terminal identifiers corresponding to the stored information is not 1, the stored information does not belong to one terminal.
  • the SD card information 1 belongs to three terminals, and the collected data may be abnormal, that is, the SD card information 1
  • the corresponding parameter information may be fraudulent data, and a quality check result of the data abnormality is generated.
  • a quality qualified quality verification result is generated.
  • a quality check result of the data abnormality is generated.
  • the data analysis server may determine that the parameter information corresponding to the storage information is normal data, and may use the parameter information to perform data analysis, and generate a data report required by each service platform to understand Business needs, adjustments to the business.
  • searching for the terminal identifier corresponding to the successful uploading information searching for the terminal identifier corresponding to the successful uploading information; sorting each terminal identifier corresponding to the uploading success information; determining, according to the sorting result, that the uploading success information does not exist redundantly In the remaining time, the data is judged to be abnormal.
  • some terminal devices such as monitors, have no storage space locally, and connect to the processor or hard disk to process data using the storage space of the processor or hard disk.
  • the multiple terminals can share the same processor or the storage space of the hard disk to process data.
  • multiple displays are connected to the same SD card, that is, the storage space of the same SD card is shared.
  • data collection is performed on multiple terminals connected to the same hard disk or processor, and the parameter information collected in essence is data of the same hard disk or processor.
  • the data analysis server may search for the successful upload information corresponding to each terminal identifier, such as successfully transmitting log information (transformation).
  • the successfully uploaded information corresponding to the terminal identifier includes the number of successful transmissions, such as the number of times the transmission log is successfully transmitted (referred to as the tt value). Each time the parameter information collected is successfully uploaded, the tt value is automatically increased according to the unit value, such as automatically incremented by 1.
  • the data analysis server may sort the upload success information corresponding to each terminal identifier, such as sorting according to the size of the tt value, and determine whether the upload success information is redundant based on the sorting result.
  • the data analysis server sorts the number of successfully transmitted logs according to the size of the tt value, and determines whether the number of successfully transmitted logs is repeated; when the number of successfully transmitted logs is not repeated If there is no redundancy in the number of times the log is successfully transmitted, the data may be abnormal.
  • the tt value of the SD card corresponding to the terminal A is 1, the tt value of the SD card corresponding to the terminal B is 3, and the terminal C
  • the tt value of the corresponding SD card identifier is 2, that is, there is no redundancy in the number of times of successfully transmitting the log, and it can be determined that the parameter information including the SD card information 1 is abnormal, which is equivalent to determining that the parameter information including the SD card information 1 has data fraud, and generates
  • the quality check result of the data abnormality avoids the parameter information for data analysis, that is, avoids using abnormal parameter information for analysis, and improves the accuracy of data collection.
  • the data analysis server may determine that the data is abnormal, and generate a quality check result of the data abnormality, so as to avoid using abnormal data for data analysis, and ensuring data analysis results. accuracy.
  • modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components.
  • any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed, or All processes or units of the device are combined.
  • Each feature disclosed in this specification may be replaced by alternative features that provide the same, equivalent or similar purpose.
  • the various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof.
  • a microprocessor or digital signal processor DSP
  • the invention may also be embodied as a device or device program for performing some or all of the methods described herein (eg, Computer programs and computer program products).
  • Such a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
  • Figure 9 illustrates a computing device that can implement a data acquisition analysis method in accordance with the present invention.
  • the computing device conventionally includes a processor 910 and a program product or readable medium in the form of a memory 920.
  • Memory 920 can be an electronic memory such as a flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, or ROM.
  • Memory 920 has a memory space 930 for program code 931 for performing any of the method steps described above.
  • storage space 930 for program code may include various program code 931 for implementing various steps in the above methods, respectively.
  • These program codes can be read from or written to one or more program products.
  • These program products include program code carriers such as memory cards.
  • Such a program product is typically a portable or fixed storage unit as described with reference to FIG.
  • the storage unit may have storage segments, storage spaces, and the like that are similarly arranged to memory 920 in the computing device of FIG.
  • the program code can be compressed, for example, in an appropriate form.
  • the storage unit includes readable code 931', ie, code that can be read by a processor, such as 910, which, when executed by a computing device, causes the computing device to perform various steps in the methods described above. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Debugging And Monitoring (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

La présente invention concerne un système et un procédé de collecte et d'analyse de données. Le système comprend un serveur de commande de collecte, un serveur d'analyse de données, un serveur de génération de rapport, et au moins un terminal de données. Un outil de collecte est monté dans le terminal de données. Le serveur de commande de collecte est utilisé pour attribuer des informations de configuration de collecte au terminal de données et fournir les informations de configuration de collecte au terminal de données correspondant. Le terminal de données est utilisé pour appeler l'outil de collecte lorsqu'une application fonctionne, collecter des informations paramétriques de paramètres de collecte selon les informations de configuration de collecte, et télécharger en amont les informations paramétriques collectées vers le serveur d'analyse de données. Le serveur d'analyse de données est utilisé pour analyser les informations paramétriques téléchargées en amont correspondant à chacun des paramètres de collecte, de manière à obtenir le résultat d'analyse de données, et envoyer le résultat d'analyse de données au serveur de génération de rapport. Le serveur de génération de rapport est utilisé pour générer un rapport de données selon le résultat d'analyse de données. Des demandes, des habitudes, et analogues, d'utilisateurs sont obtenues au moyen du rapport de données, de façon à régler un service afin que le service satisfasse mieux les utilisateurs.
PCT/CN2016/104923 2015-12-15 2016-11-07 Système et procédé de collecte et d'analyse de données WO2017101606A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510938627.6A CN105553769B (zh) 2015-12-15 2015-12-15 一种数据采集分析系统和方法
CN201510938627.6 2015-12-15

Publications (1)

Publication Number Publication Date
WO2017101606A1 true WO2017101606A1 (fr) 2017-06-22

Family

ID=55832708

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/104923 WO2017101606A1 (fr) 2015-12-15 2016-11-07 Système et procédé de collecte et d'analyse de données

Country Status (2)

Country Link
CN (1) CN105553769B (fr)
WO (1) WO2017101606A1 (fr)

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109905253A (zh) * 2017-12-07 2019-06-18 航天信息股份有限公司 一种日志信息采集方法及装置
CN110159252A (zh) * 2019-06-04 2019-08-23 国网北京市电力公司 数据采集系统及方法
CN110633205A (zh) * 2019-06-20 2019-12-31 北京无限光场科技有限公司 一种埋点事件的检测方法、装置、终端设备及介质
CN110995524A (zh) * 2019-10-28 2020-04-10 北京三快在线科技有限公司 流量数据监控方法、装置、电子设备和计算机可读介质
CN111198942A (zh) * 2018-10-31 2020-05-26 合肥神策数据网络科技有限公司 数据分析报告生成方法、装置、移动终端及存储介质
CN111611256A (zh) * 2020-04-30 2020-09-01 广东良实机电工程有限公司 设备能耗监测方法、装置、终端设备及存储介质
CN112015870A (zh) * 2020-09-14 2020-12-01 支付宝(杭州)信息技术有限公司 数据上传方法及装置
CN112130854A (zh) * 2020-09-09 2020-12-25 海看网络科技(山东)股份有限公司 一种iptv检测探针系统关键字的方法
CN112181804A (zh) * 2020-08-31 2021-01-05 五八到家有限公司 一种参数校验方法、设备以及存储介质
CN112328619A (zh) * 2020-09-24 2021-02-05 杭州小电科技股份有限公司 数据质量监控方法、装置、系统、电子装置和存储介质
CN112395173A (zh) * 2019-08-14 2021-02-23 腾讯科技（深圳）有限公司 信息流数据上报的方法和装置
CN112434023A (zh) * 2019-08-26 2021-03-02 长鑫存储技术有限公司 工艺数据分析方法及装置、存储介质及计算机设备
CN112817782A (zh) * 2021-02-20 2021-05-18 腾讯科技（深圳）有限公司 一种数据采集上报方法、装置、电子设备和存储介质
CN113014647A (zh) * 2021-02-25 2021-06-22 浙江正元智慧科技股份有限公司 一种基于大数据的运维平台远程数据采集系统
CN113114736A (zh) * 2021-03-25 2021-07-13 北京数字磐石科技有限公司 矿机管理方法和系统
CN113126562A (zh) * 2020-01-16 2021-07-16 智能云科信息科技有限公司 数据采集方法、装置、系统及计算机可读存储介质
CN113515507A (zh) * 2021-03-12 2021-10-19 中电建电力检修工程有限公司 应用于大坝渗水检测的方法及系统
CN113626477A (zh) * 2021-07-19 2021-11-09 浙江浙大中控信息技术有限公司 基于水厂物联网系统的可配置报表系统及方法
CN113760564A (zh) * 2020-10-20 2021-12-07 北京沃东天骏信息技术有限公司 数据处理方法、装置和系统
CN114296009A (zh) * 2022-03-10 2022-04-08 山东汇能电气有限公司 一种变压器运行智能分析系统
CN114385478A (zh) * 2020-10-22 2022-04-22 腾讯科技（深圳）有限公司 一种数据采集上报方法、装置、电子设备和存储介质
CN114401289A (zh) * 2021-12-31 2022-04-26 深圳市麦谷科技有限公司 任务分批上传方法及系统
CN114460400A (zh) * 2022-01-19 2022-05-10 西北核技术研究所 一种脉冲电流注入自动化实验系统及方法
CN114615291A (zh) * 2021-12-08 2022-06-10 江苏众亿国链大数据科技有限公司 一种基于物联网的数据采集方法
CN114785784A (zh) * 2022-04-14 2022-07-22 中国第一汽车股份有限公司 数据处理方法、装置、电子设备
CN114827774A (zh) * 2022-04-26 2022-07-29 浙江正泰物联技术有限公司 一种电气数据采集方法、装置及系统
CN115004667A (zh) * 2020-04-24 2022-09-02 深圳市欢太科技有限公司 信息推送方法、装置、电子设备及计算机可读介质
CN115102936A (zh) * 2022-06-22 2022-09-23 苏州达家迎信息技术有限公司 批量信息上传方法、装置、存储介质及电子设备
CN116303689A (zh) * 2023-05-25 2023-06-23 武汉科迪智能环境股份有限公司 数据报表的生成方法、装置、计算机设备及可读存储介质
CN117215542A (zh) * 2023-11-07 2023-12-12 上海华创自动化工程股份有限公司 一种自定义数据处理系统及方法
CN118474575A (zh) * 2024-07-12 2024-08-09 北京博维亚讯技术有限公司 一种集中器的数据采集分析方法及系统

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105553769B (zh) * 2015-12-15 2019-03-05 北京奇虎科技有限公司 一种数据采集分析系统和方法
CN106101175B (zh) * 2016-05-25 2019-08-23 腾讯科技（深圳）有限公司 设备管控方法、装置和系统
CN107809344B (zh) * 2016-09-09 2021-01-22 中华电信股份有限公司 实时讯务量搜集与分析系统及方法
CN106549830A (zh) * 2016-11-09 2017-03-29 上海最会保网络科技有限公司 一种移动应用数据采集装置和方法
CN106506851B (zh) * 2016-11-28 2021-01-01 深圳前海谷谷力生物科技有限公司 移动终端的数据统计适配方法与装置
CN106649240A (zh) * 2016-12-27 2017-05-10 北京奇虎科技有限公司 基于自定义事件展示报表的方法及装置
CN106649241A (zh) * 2016-12-27 2017-05-10 北京奇虎科技有限公司 基于自定义事件生成报表的方法及装置
CN107102860A (zh) * 2017-04-21 2017-08-29 上海青橙实业有限公司 数据采集方法、移动终端以及服务器
CN107018046A (zh) * 2017-06-06 2017-08-04 上海鋆创信息技术有限公司 一种数据采集方法、装置、终端及存储介质
CN109769305A (zh) * 2017-11-09 2019-05-17 普天信息技术有限公司 调度处理的方法、基站、电子设备和存储介质
CN108418787B (zh) * 2018-01-02 2021-02-26 深圳壹账通智能科技有限公司 企业资源计划数据的采集方法、终端设备及介质
CN110221922A (zh) * 2018-03-02 2019-09-10 网易（杭州）网络有限公司 程序产品的卡顿定位方法及装置、介质及计算设备
CN108833459B (zh) * 2018-04-09 2022-05-13 平安普惠企业管理有限公司 数据上报方法及终端设备
CN110391925A (zh) * 2018-04-20 2019-10-29 阿里巴巴集团控股有限公司 一种数据采集方法、数据采集装置及服务器
CN108762960B (zh) * 2018-05-14 2021-07-06 国网福建省电力有限公司 一种移动应用监测系统
CN109189664B (zh) * 2018-07-24 2022-08-02 网宿科技股份有限公司 应用程序的信息采集方法及终端
CN109376140A (zh) * 2018-08-24 2019-02-22 国网吉林省电力有限公司信息通信公司 一种静态资源自动化采集方法、系统、设备及存储介质
CN109711156B (zh) * 2018-12-28 2023-01-31 上海北信源信息技术有限公司 一种内网计算机硬盘信息监控管理的方法及系统
CN111857789A (zh) * 2019-04-29 2020-10-30 天津五八到家科技有限公司 数据处理系统
CN110287471A (zh) * 2019-06-28 2019-09-27 重庆回形针信息技术有限公司 一种图表分析系统
CN110275902A (zh) * 2019-06-28 2019-09-24 重庆回形针信息技术有限公司 一种数据采集分析系统及方法
CN110569168B (zh) * 2019-08-26 2022-07-08 腾讯科技（深圳）有限公司 一种数据交互方法、相关设备及存储介质
CN110515963A (zh) * 2019-08-30 2019-11-29 北京东软望海科技有限公司 收集统计信息的方法、装置、电子设备及可读存储介质
CN111209057B (zh) * 2020-01-10 2022-09-09 厦门瑞为信息技术有限公司 一种快速调整感知终端的方法以及装置
CN111581931B (zh) * 2020-04-26 2023-08-11 泰康保险集团股份有限公司 基于数据防泄漏系统的报表生成方法及装置
CN111694720B (zh) * 2020-06-15 2023-11-24 山东浪潮云服务信息科技有限公司 一种自动监控数据采集进度的方法
CN111897714B (zh) * 2020-06-24 2024-03-26 方欣科技有限公司 一种基于辅助测试工具的测试数据沉淀方法、装置及系统
CN112235060A (zh) * 2020-09-29 2021-01-15 石家庄市善理通益科技有限公司 一种应用于对讲终端的异常行为监控采集方法
CN118394667B (zh) * 2024-07-01 2024-09-06 浙江微特电子信息有限公司 基于人工智能的软件系统分析方法及系统

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103188088A (zh) * 2011-12-27 2013-07-03 北京新媒传信科技有限公司 设备信息采集系统及方法
CN103248651A (zh) * 2012-02-09 2013-08-14 腾讯科技（深圳）有限公司 一种性能监控的方法和系统以及客户端和服务器
WO2014067439A1 (fr) * 2012-10-29 2014-05-08 Tencent Technology (Shenzhen) Company Limited Procédé, système et dispositif de surveillance de données
CN104657278A (zh) * 2015-03-13 2015-05-27 百度在线网络技术（北京）有限公司 客户端性能评估方法及系统
CN105119899A (zh) * 2015-07-17 2015-12-02 世纪龙信息网络有限责任公司 非入侵式客户端运行时数据采集方法与系统
CN105376335A (zh) * 2015-12-15 2016-03-02 北京奇虎科技有限公司 一种采集数据上传方法和装置
CN105373293A (zh) * 2015-12-15 2016-03-02 北京奇虎科技有限公司 一种数据采集方法和装置
CN105391602A (zh) * 2015-12-15 2016-03-09 北京奇虎科技有限公司 一种数据采集测试方法和装置
CN105515909A (zh) * 2015-12-15 2016-04-20 北京奇虎科技有限公司 一种数据采集测试方法和装置
CN105516321A (zh) * 2015-12-15 2016-04-20 北京奇虎科技有限公司 一种数据采集方法和装置
CN105553769A (zh) * 2015-12-15 2016-05-04 北京奇虎科技有限公司 一种数据采集分析系统和方法
CN105553770A (zh) * 2015-12-15 2016-05-04 北京奇虎科技有限公司 一种数据采集控制方法和装置
CN105577472A (zh) * 2015-12-15 2016-05-11 北京奇虎科技有限公司 一种数据采集测试方法和装置
CN105574150A (zh) * 2015-12-15 2016-05-11 北京奇虎科技有限公司 一种数据处理方法和装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101557582B (zh) * 2008-04-08 2012-08-29 中国移动通信集团公司 一种移动通信用户信息统计方法及装置
CN102711141B (zh) * 2012-05-15 2014-12-17 上海交通大学无锡研究院 一种基于移动终端的移动网络服务质量分布式采集方法
US20150120914A1 (en) * 2012-06-13 2015-04-30 Hitachi, Ltd. Service monitoring system and service monitoring method
CN103118095A (zh) * 2013-01-25 2013-05-22 浪潮电子信息产业股份有限公司 一种云数据中心信息差量采集方法
CN104239577A (zh) * 2014-10-09 2014-12-24 北京奇虎科技有限公司 检测网页数据真伪的方法和装置

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103188088A (zh) * 2011-12-27 2013-07-03 北京新媒传信科技有限公司 设备信息采集系统及方法
CN103248651A (zh) * 2012-02-09 2013-08-14 腾讯科技（深圳）有限公司 一种性能监控的方法和系统以及客户端和服务器
WO2014067439A1 (fr) * 2012-10-29 2014-05-08 Tencent Technology (Shenzhen) Company Limited Procédé, système et dispositif de surveillance de données
CN104657278A (zh) * 2015-03-13 2015-05-27 百度在线网络技术（北京）有限公司 客户端性能评估方法及系统
CN105119899A (zh) * 2015-07-17 2015-12-02 世纪龙信息网络有限责任公司 非入侵式客户端运行时数据采集方法与系统
CN105376335A (zh) * 2015-12-15 2016-03-02 北京奇虎科技有限公司 一种采集数据上传方法和装置
CN105373293A (zh) * 2015-12-15 2016-03-02 北京奇虎科技有限公司 一种数据采集方法和装置
CN105391602A (zh) * 2015-12-15 2016-03-09 北京奇虎科技有限公司 一种数据采集测试方法和装置
CN105515909A (zh) * 2015-12-15 2016-04-20 北京奇虎科技有限公司 一种数据采集测试方法和装置
CN105516321A (zh) * 2015-12-15 2016-04-20 北京奇虎科技有限公司 一种数据采集方法和装置
CN105553769A (zh) * 2015-12-15 2016-05-04 北京奇虎科技有限公司 一种数据采集分析系统和方法
CN105553770A (zh) * 2015-12-15 2016-05-04 北京奇虎科技有限公司 一种数据采集控制方法和装置
CN105577472A (zh) * 2015-12-15 2016-05-11 北京奇虎科技有限公司 一种数据采集测试方法和装置
CN105574150A (zh) * 2015-12-15 2016-05-11 北京奇虎科技有限公司 一种数据处理方法和装置

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109905253A (zh) * 2017-12-07 2019-06-18 航天信息股份有限公司 一种日志信息采集方法及装置
CN111198942A (zh) * 2018-10-31 2020-05-26 合肥神策数据网络科技有限公司 数据分析报告生成方法、装置、移动终端及存储介质
CN111198942B (zh) * 2018-10-31 2023-07-04 合肥神策数据网络科技有限公司 数据分析报告生成方法、装置、移动终端及存储介质
CN110159252A (zh) * 2019-06-04 2019-08-23 国网北京市电力公司 数据采集系统及方法
CN110633205A (zh) * 2019-06-20 2019-12-31 北京无限光场科技有限公司 一种埋点事件的检测方法、装置、终端设备及介质
CN112395173A (zh) * 2019-08-14 2021-02-23 腾讯科技（深圳）有限公司 信息流数据上报的方法和装置
CN112395173B (zh) * 2019-08-14 2023-10-13 腾讯科技（深圳）有限公司 信息流数据上报的方法和装置
CN112434023A (zh) * 2019-08-26 2021-03-02 长鑫存储技术有限公司 工艺数据分析方法及装置、存储介质及计算机设备
CN110995524A (zh) * 2019-10-28 2020-04-10 北京三快在线科技有限公司 流量数据监控方法、装置、电子设备和计算机可读介质
CN113126562A (zh) * 2020-01-16 2021-07-16 智能云科信息科技有限公司 数据采集方法、装置、系统及计算机可读存储介质
CN115004667B (zh) * 2020-04-24 2024-04-02 深圳市欢太科技有限公司 信息推送方法、装置、电子设备及计算机可读介质
CN115004667A (zh) * 2020-04-24 2022-09-02 深圳市欢太科技有限公司 信息推送方法、装置、电子设备及计算机可读介质
CN111611256A (zh) * 2020-04-30 2020-09-01 广东良实机电工程有限公司 设备能耗监测方法、装置、终端设备及存储介质
CN111611256B (zh) * 2020-04-30 2023-05-09 广东良实机电工程有限公司 设备能耗监测方法、装置、终端设备及存储介质
CN112181804B (zh) * 2020-08-31 2023-09-08 五八到家有限公司 一种参数校验方法、设备以及存储介质
CN112181804A (zh) * 2020-08-31 2021-01-05 五八到家有限公司 一种参数校验方法、设备以及存储介质
CN112130854B (zh) * 2020-09-09 2023-12-22 海看网络科技(山东)股份有限公司 一种iptv检测探针系统关键字的方法
CN112130854A (zh) * 2020-09-09 2020-12-25 海看网络科技(山东)股份有限公司 一种iptv检测探针系统关键字的方法
CN112015870A (zh) * 2020-09-14 2020-12-01 支付宝(杭州)信息技术有限公司 数据上传方法及装置
CN112015870B (zh) * 2020-09-14 2024-06-07 支付宝(杭州)信息技术有限公司 数据上传方法及装置
CN112328619A (zh) * 2020-09-24 2021-02-05 杭州小电科技股份有限公司 数据质量监控方法、装置、系统、电子装置和存储介质
CN113760564A (zh) * 2020-10-20 2021-12-07 北京沃东天骏信息技术有限公司 数据处理方法、装置和系统
CN114385478A (zh) * 2020-10-22 2022-04-22 腾讯科技（深圳）有限公司 一种数据采集上报方法、装置、电子设备和存储介质
CN112817782A (zh) * 2021-02-20 2021-05-18 腾讯科技（深圳）有限公司 一种数据采集上报方法、装置、电子设备和存储介质
CN112817782B (zh) * 2021-02-20 2023-10-13 腾讯科技（深圳）有限公司 一种数据采集上报方法、装置、电子设备和存储介质
CN113014647B (zh) * 2021-02-25 2022-05-03 浙江正元智慧科技股份有限公司 一种基于大数据的运维平台远程数据采集系统
CN113014647A (zh) * 2021-02-25 2021-06-22 浙江正元智慧科技股份有限公司 一种基于大数据的运维平台远程数据采集系统
CN113515507A (zh) * 2021-03-12 2021-10-19 中电建电力检修工程有限公司 应用于大坝渗水检测的方法及系统
CN113515507B (zh) * 2021-03-12 2024-02-02 中电建电力检修工程有限公司 应用于大坝渗水检测的方法及系统
CN113114736A (zh) * 2021-03-25 2021-07-13 北京数字磐石科技有限公司 矿机管理方法和系统
CN113626477B (zh) * 2021-07-19 2023-10-10 浙江中控信息产业股份有限公司 基于水厂物联网系统的可配置报表系统及方法
CN113626477A (zh) * 2021-07-19 2021-11-09 浙江浙大中控信息技术有限公司 基于水厂物联网系统的可配置报表系统及方法
CN114615291A (zh) * 2021-12-08 2022-06-10 江苏众亿国链大数据科技有限公司 一种基于物联网的数据采集方法
CN114401289A (zh) * 2021-12-31 2022-04-26 深圳市麦谷科技有限公司 任务分批上传方法及系统
CN114401289B (zh) * 2021-12-31 2024-08-16 深圳市麦谷科技有限公司 任务分批上传方法及系统
CN114460400A (zh) * 2022-01-19 2022-05-10 西北核技术研究所 一种脉冲电流注入自动化实验系统及方法
CN114296009B (zh) * 2022-03-10 2022-05-24 山东汇能电气有限公司 一种变压器运行智能分析系统
CN114296009A (zh) * 2022-03-10 2022-04-08 山东汇能电气有限公司 一种变压器运行智能分析系统
CN114785784B (zh) * 2022-04-14 2023-10-24 中国第一汽车股份有限公司 数据处理方法、装置、电子设备
CN114785784A (zh) * 2022-04-14 2022-07-22 中国第一汽车股份有限公司 数据处理方法、装置、电子设备
CN114827774A (zh) * 2022-04-26 2022-07-29 浙江正泰物联技术有限公司 一种电气数据采集方法、装置及系统
CN115102936B (zh) * 2022-06-22 2023-11-21 苏州达家迎信息技术有限公司 批量信息上传方法、装置、存储介质及电子设备
CN115102936A (zh) * 2022-06-22 2022-09-23 苏州达家迎信息技术有限公司 批量信息上传方法、装置、存储介质及电子设备
CN116303689A (zh) * 2023-05-25 2023-06-23 武汉科迪智能环境股份有限公司 数据报表的生成方法、装置、计算机设备及可读存储介质
CN117215542B (zh) * 2023-11-07 2024-05-14 上海华创自动化工程股份有限公司 一种自定义数据处理系统及方法
CN117215542A (zh) * 2023-11-07 2023-12-12 上海华创自动化工程股份有限公司 一种自定义数据处理系统及方法
CN118474575A (zh) * 2024-07-12 2024-08-09 北京博维亚讯技术有限公司 一种集中器的数据采集分析方法及系统

Also Published As

Publication number Publication date
CN105553769A (zh) 2016-05-04
CN105553769B (zh) 2019-03-05

Similar Documents

Publication Publication Date Title
WO2017101606A1 (fr) Système et procédé de collecte et d'analyse de données
US10733079B2 (en) Systems and methods for end-to-end testing of applications using dynamically simulated data
WO2018113385A1 (fr) Procédé et système de suivi dynamique d'application de client, client et serveur
CN105376335B (zh) 一种采集数据上传方法和装置
WO2017113677A1 (fr) Procédé et système de traitement de données de comportement d'utilisateur
CN109684052B (zh) 事务分析方法、装置、设备及存储介质
CN113489713B (zh) 网络攻击的检测方法、装置、设备及存储介质
CN106878108B (zh) 网络流量回放测试方法及装置
CN106649063A (zh) 用于监测程序运行时耗时数据的方法及系统
CN108512720B (zh) 一种网站流量的统计方法及装置
CN108156141B (zh) 一种实时数据识别方法、装置及电子设备
CN109495291B (zh) 调用异常的定位方法、装置和服务器
CN105553770B (zh) 一种数据采集控制方法和装置
GB2569678A (en) Automation of SQL tuning method and system using statistic SQL pattern analysis
CN112232881A (zh) 一种数据检测方法、装置、电子设备及存储介质
CN112511459B (zh) 一种流量识别方法、装置、电子设备及存储介质
CN111740868A (zh) 告警数据的处理方法和装置及存储介质
CN107770805B (zh) 终端的标识信息的判定方法及装置
CN109491733B (zh) 基于可视化的界面显示方法及相关设备
CN110851339A (zh) 一种埋点数据的上报方法、装置、存储介质和终端设备
CN106294406B (zh) 一种用于处理应用访问数据的方法与设备
CN110851334B (zh) 流量统计方法、电子设备、系统及介质
CN110807050B (zh) 性能分析方法、装置、计算机设备及存储介质
CN110177024B (zh) 热点设备的监控方法及客户端、服务端、系统
CN114189455B (zh) 基于ebpf技术的容器网络流量监控统计方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16874663

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16874663

Country of ref document: EP

Kind code of ref document: A1