CN108156006B - Buried point data reporting method and device and electronic equipment - Google Patents

Buried point data reporting method and device and electronic equipment Download PDF

Info

Publication number
CN108156006B
CN108156006B CN201611102576.4A CN201611102576A CN108156006B CN 108156006 B CN108156006 B CN 108156006B CN 201611102576 A CN201611102576 A CN 201611102576A CN 108156006 B CN108156006 B CN 108156006B
Authority
CN
China
Prior art keywords
buried point
buried
point data
preset
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611102576.4A
Other languages
Chinese (zh)
Other versions
CN108156006A (en
Inventor
廖庆锟
罗志强
王令宇
贺星星
殷海翔
夏洋
李森
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201611102576.4A priority Critical patent/CN108156006B/en
Publication of CN108156006A publication Critical patent/CN108156006A/en
Application granted granted Critical
Publication of CN108156006B publication Critical patent/CN108156006B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3692—Test management for test results analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/06—Generation of reports

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Alarm Systems (AREA)

Abstract

The application discloses a buried point data reporting method, a buried point data reporting device and electronic equipment, two buried point data processing methods, two buried point data processing devices and electronic equipment, and two buried point data processing systems. The buried point data reporting method comprises the following steps: storing the triggered embedded points in the page to an embedded point queue to be sent; when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to form a buried point data packet of the page; and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet. By adopting the method provided by the application, a plurality of buried points in the buried point queue are dynamically sampled, buried point data is reported in batches, and excessive requests are prevented from being sent to a server; therefore, two aspects of reporting sufficient data of the buried point and ensuring the performance of the client can be considered.

Description

Buried point data reporting method and device and electronic equipment
Technical Field
The application relates to the technical field of page monitoring, in particular to a buried point data reporting method. The application also relates to a buried point data reporting device and an electronic device, two buried point data processing methods, two buried point data processing devices and an electronic device, and two buried point data processing systems.
Background
Many Web applications in the internet do not exist independently or work independently, but are interwoven together to form a whole, which means that if any one of the Web applications has a problem, other Web applications may not work. Therefore, monitoring the running state of the Web application is a crucial link. According to the running environment, a Web application can be divided into two parts: a client and a server; the server runs on the server, the running environment is single and easy to monitor, the client sides are completely different, due to the arrival of the mobile era, users gradually migrate from personal computers to mobile devices such as mobile phones, and other factors such as the terminal types and the network states of the users can influence the running of Web applications, so that the running conditions of the client sides need to be reported, the running conditions of the whole applications can be obtained after the summary and analysis processing, and each ring is full of difficulties and challenges.
In order to acquire the operation condition data of the client, a point burying process is required to be performed at a key position of the page, and the operation condition of the page can be acquired through the point burying process, so that the client is monitored. The current common buried point data reporting and alarming scheme is that the data is directly reported by fixed ratio sampling at a client, and then whether to trigger alarming is judged according to whether the total reported data amount reaches a fixed value. This processing method inevitably causes the following problems:
1) the client capacity cannot be reasonably utilized, and when the data volume needing to be sent is large, too many requests can be sent, so that page loading is slowed down, user experience is reduced, and if the data volume needing to be sent is small, the sample volume is insufficient, so that the accuracy of alarm information is influenced.
2) Due to the fixed sampling ratio, the transmission opportunity of the buried point with small data quantity is preempted by the buried point with large data quantity, and the server cannot receive enough buried point data.
3) The stability algorithm (buried point alarm algorithm) has a simple calculation model, is mainly calculated according to a fixed threshold value, cannot be evaluated by combining historical flow and change trend, and is easy to cause false alarm due to different service forms.
Therefore, how to research and develop a new buried point data reporting mode, which can combine two aspects of reporting sufficient buried point data and ensuring the performance of the client becomes a problem that needs to be solved urgently by the technical personnel in the field.
Disclosure of Invention
The application provides a buried point data reporting method, which aims to solve the problem that the prior art cannot give consideration to both the aspects of reporting sufficient buried point data and ensuring the performance of a client. The application also provides a buried point data reporting device and an electronic device, two buried point data processing methods, two buried point data processing devices and an electronic device, and two buried point data processing systems.
The application provides a buried point data reporting method, which comprises the following steps:
storing the triggered embedded points in the page to an embedded point queue to be sent;
when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to form a buried point data packet of the page;
and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet.
Optionally, the type of the buried point includes a common type or an error type;
the preset dynamic sampling algorithm comprises the following steps:
randomly extracting a common type of buried points from the buried point queue; randomly extracting the buried points with the error types from the buried point queue;
taking the maximum length of the request data packet allowed by the transmission protocol as a limit, splicing the extracted serialized character strings corresponding to the embedded points of the common type to the maximum extent, and obtaining a sampling embedded point data packet of the common type; splicing the extracted serialized character strings corresponding to the buried points of the error types to the maximum extent to obtain sampling buried point data packets of the error types;
generating a first data packet according to the sampling buried point data packet of the common type and a buried point sampling ratio, wherein the buried point sampling ratio is a ratio of the total number of the buried points of the common type to the number of the buried points which are successfully spliced; generating a second data packet according to the sampling buried point data packet of the error type;
the sending of the buried point data processing request for the data packet to the server includes:
sending a first buried point data processing request aiming at the page to the server, wherein the first buried point data processing request comprises the first data packet; and sending a second embedded data processing request aiming at the page to the server, wherein the second embedded data processing request comprises the second data packet.
Optionally, before randomly extracting a general type of buried point from the buried point queue and before randomly extracting an error type of buried point from the buried point queue, the method further includes:
merging the buried points with the same name in the buried point queue, and recording the merging number;
randomly extracting a common type of buried point from the buried point queue by adopting the following method:
randomly extracting common type buried points from a buried point queue after the buried points with the same name are merged;
randomly extracting the buried points with error types from the buried point queue by adopting the following method:
randomly extracting buried points of error types from the buried point queue after the buried points with the same name are merged;
the method further comprises the following steps:
writing the combined number corresponding to the successfully spliced buried points of the common type into a first data packet; and writing the merging number corresponding to the buried point of the successfully spliced error type into a second data packet.
Optionally, the method further includes:
removing the buried point of the error type which is successfully sent from the buried point queue; and removing the common type of buried point from the buried point queue.
Optionally, the preset condition is as follows:
in the page loading process, after the resource file included in the page is downloaded, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a first preset time interval;
and in the page display process, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a second preset time interval.
Optionally, the buried point data packet includes: the information of the buried point and the information of the page to which the buried point belongs.
Optionally, the information of the buried point at least includes a name of the buried point, and includes at least one of the following information: the type of the buried point, the father buried point, the root buried point, the merging quantity and the acquisition time.
Optionally, the information of the page at least includes a page identifier, and includes at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
The present application further provides a device for reporting buried point data, including:
the embedded point acquisition unit is used for storing the triggered embedded points in the page to an embedded point queue to be sent;
the embedded point sampling unit is used for dynamically sampling the embedded points in the embedded point queue through a preset dynamic sampling algorithm when a preset condition is met, and acquiring an embedded point data packet corresponding to the sampling embedded point;
and the request sending unit is used for sending a buried point data processing request aiming at the page to a server, and the buried point data processing request comprises the buried point data packet.
The present application further provides an electronic device, comprising:
a processor; and
the device is powered on and executes the program of the buried point data reporting method through the processor, and then the following steps are executed: storing the triggered embedded points in the page to an embedded point queue to be sent; when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to obtain buried point data packets corresponding to the sampling buried points; and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet.
The application also provides a buried point data processing method, which comprises the following steps:
receiving a plurality of buried point data processing requests aiming at a specific page, which are sent by at least one client, wherein the buried point data processing requests comprise buried point data packets;
according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point;
determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index;
and determining a preset monitoring buried point needing alarming according to the alarming probability and a preset alarming probability threshold value.
Optionally, the preset index related to the alarm probability algorithm includes any one of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
the alarm probability algorithm comprises:
acquiring the average value and the variance of the preset index according to the current data and the historical data of the preset index;
and acquiring the alarm probability of the preset index through a Gaussian model according to the data of the preset index, the average value and the variance, and taking the alarm probability as the alarm probability of the preset monitoring buried point.
Optionally, the preset indexes related to the alarm probability algorithm include at least two of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
the alarm probability algorithm comprises:
acquiring the average value and the variance of each preset index according to the current data and the historical data of the preset index;
obtaining the alarm probability of each preset index through a Gaussian model according to the data, the average value and the variance of the preset index;
and acquiring the alarm probability of the preset monitoring buried point according to the preset alarm probability weight corresponding to each preset index and the alarm probability of each preset index.
Optionally, the preset monitoring buried points include a plurality of preset monitoring buried points, different preset monitoring buried points correspond to different alarm probability algorithms, and different preset monitoring buried points correspond to different alarm probability thresholds.
Optionally, the method further includes:
generating a corresponding alarm message for the preset monitoring buried point needing to be alarmed;
acquiring an alarm person and an alarm mode corresponding to the preset monitoring buried point needing to be alarmed;
and providing the alarm message to the alarm person according to the alarm mode.
Optionally, the alarm mode includes: short message of mobile phone, mail, instant communication.
Optionally, after receiving the buried point data processing request, the method further includes:
and analyzing the buried point data packet to obtain buried point data corresponding to the buried point data packet.
Optionally, the buried point data packet includes a plurality of buried point data of a common type and a buried point sampling ratio;
the analyzing the buried point data packet and obtaining the buried point data corresponding to the buried point data packet includes:
determining the occurrence frequency of each buried point in the buried point data packet according to the buried point sampling ratio;
and forming a plurality of buried point data corresponding to the buried point data packet according to each buried point and the occurrence frequency.
Optionally, the buried point data packet includes a merging number of each buried point;
and determining the occurrence frequency of each buried point in the buried point data packet according to the buried point sampling ratio by adopting the following mode:
and taking the product of the sampling ratio of the buried points and the combined number of the buried points as the occurrence number of the buried points.
Optionally, the buried point data packet includes a plurality of buried point data of an error type and a merging number of each buried point;
analyzing the buried point data packet, and obtaining buried point data corresponding to the buried point data packet by adopting the following mode:
and forming a plurality of buried point data corresponding to the buried point data packet according to the plurality of buried points and the merging number of each buried point.
Optionally, the information of the buried point at least includes a name of the buried point, and includes at least one of the following information: the type of the embedded points, the belonged father embedded points, the merging quantity and the acquisition time.
Optionally, the buried point data packet includes information of the specific page.
Optionally, the information of the specific page at least includes a page identifier, and includes at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
Optionally, the method further includes:
and storing the plurality of buried point data obtained by analysis.
Optionally, the method further includes:
and storing the information of the specific page corresponding to each buried point data.
Optionally, the method further includes:
and storing the information of the client corresponding to each buried point data.
Optionally, the information of the client includes at least one of the following information: device type, browser, operating system, user identification, IP address.
The present application further provides a buried point data processing apparatus, including:
the system comprises a request receiving unit, a processing unit and a processing unit, wherein the request receiving unit is used for receiving a plurality of buried point data processing requests aiming at a specific page and sent by at least one client, and the buried point data processing requests comprise buried point data packets;
the data statistics unit is used for carrying out statistics on the buried point data in the multiple buried point data packets of the specific page received in the time interval range according to a preset time interval to obtain the current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point;
the alarm probability calculation unit is used for determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index;
and the alarm buried point determining unit is used for determining the preset monitoring buried point needing alarming according to the alarm probability and a preset alarm probability threshold value.
The present application further provides an electronic device, comprising:
a processor; and
a memory for storing a program for implementing the buried point data processing method, wherein the following steps are executed after the device is powered on and the program for implementing the buried point data processing method is executed by the processor: receiving a plurality of buried point data processing requests aiming at a specific page, which are sent by at least one client, wherein the buried point data processing requests comprise buried point data packets; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point; determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index; and determining a preset monitoring buried point needing alarming according to the alarming probability and a preset alarming probability threshold value.
The present application further provides a buried point data processing system, including: according to the buried point data reporting device and the buried point data processing device.
The application also provides another buried point data processing method, which comprises the following steps:
receiving a buried point data processing request aiming at a specific page and sent by at least one client, wherein the buried point data processing request comprises a buried point data packet;
according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index corresponding to a preset monitoring buried point;
and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
Optionally, the preset index includes any one of the following indexes: the ratio of the number of times of integration of father-son buried points, the ratio of the number of times of integration of root-son buried points, the daily-ring ratio of the number of times of integration, and the weekly-ring ratio of the number of times of integration.
The present application also provides another buried point data processing apparatus, including:
the system comprises a request receiving unit, a processing unit and a processing unit, wherein the request receiving unit is used for receiving a buried point data processing request aiming at a specific page and sent by at least one client, and the buried point data processing request comprises a buried point data packet;
the data counting unit is used for counting the buried point data in the plurality of buried point data packets of the specific page received in the time interval range according to a preset time interval to obtain the current data of a preset index corresponding to a preset monitoring buried point;
and the alarm buried point determining unit is used for determining the preset monitoring buried point needing alarming according to the data of the preset index and the preset threshold corresponding to the preset index.
The present application also provides another electronic device, comprising:
a processor; and
a memory for storing a program for implementing the buried point data processing method, wherein the following steps are executed after the device is powered on and the program for implementing the buried point data processing method is executed by the processor: receiving a buried point data processing request aiming at a specific page and sent by at least one client, wherein the buried point data processing request comprises a buried point data packet; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index corresponding to a preset monitoring buried point; and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
The present application further provides another buried point data processing system, including: the buried point data reporting device is the buried point data processing device.
Compared with the prior art, the method has the following advantages:
the method for reporting the buried point data comprises the steps of storing a triggered buried point in a page to a buried point queue to be sent, dynamically sampling the buried point in the buried point queue through a preset dynamic sampling algorithm when a preset condition is met to form a buried point data packet of the page, and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet; in the processing mode, a plurality of buried points in the buried point queue are dynamically sampled, and buried point data is reported in batches, so that excessive requests are prevented from being sent to a server; therefore, two aspects of reporting sufficient data of the buried point and ensuring the performance of the client can be considered.
According to the embedded data processing method, an embedded data processing request which is sent by at least one client and aims at a specific page is received, the embedded data processing request comprises embedded data packets, embedded data in a plurality of embedded data packets of the specific page received in a time interval range is counted according to a preset time interval, current data of a preset index relevant to an alarm probability algorithm corresponding to a preset monitoring embedded point is obtained, then the alarm probability of the preset monitoring embedded point is determined through the alarm probability algorithm according to the current data and historical data of the preset index, and finally the preset monitoring embedded point needing to be alarmed is determined according to the alarm probability and a preset alarm probability threshold; the processing mode enables evaluation to be carried out by combining historical flow and change trend, and the situation of false alarm caused by different business forms is avoided; therefore, the alarm accuracy can be effectively improved.
Drawings
Fig. 1 is a flowchart of an embodiment of a buried point data reporting method provided in the present application;
fig. 2 is a detailed flowchart of an embodiment of a buried point data reporting method provided in the present application;
fig. 3 is a schematic diagram of an embodiment of a buried point data reporting apparatus provided in the present application;
FIG. 4 is a schematic diagram of an embodiment of an electronic device provided herein;
FIG. 5 is a flowchart of an embodiment of a buried point data processing method provided in the present application;
FIG. 6 is a flowchart illustrating an embodiment of a buried point data processing method according to the present invention;
FIG. 7 is a schematic diagram of an embodiment of a buried point data processing apparatus provided in the present application;
FIG. 8 is a schematic diagram of an embodiment of an electronic device provided herein;
FIG. 9 is a schematic diagram of an embodiment of a buried point data processing system provided herein;
FIG. 10 is a detailed schematic diagram of an embodiment of a buried point data processing system provided herein;
FIG. 11 is a flow chart of another embodiment of a buried point data processing method provided by the present application;
FIG. 12 is a schematic diagram of another embodiment of a buried point data processing apparatus provided in the present application;
FIG. 13 is a schematic diagram of an embodiment of an electronic device provided herein;
FIG. 14 is a schematic diagram of an embodiment of a buried point data processing system provided by the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.
In the application, a method, a device and an electronic device for reporting buried point data, two methods, devices and electronic devices for processing buried point data, and two systems for processing buried point data are provided. Details are described in the following examples one by one.
The basic core idea of the buried point data reporting method provided by the application is as follows: storing the triggered embedded points in the page to an embedded point queue to be sent, dynamically sampling the embedded points in the embedded point queue through a preset dynamic sampling algorithm when a preset condition is met to form an embedded point data packet, and sending an embedded point data processing request aiming at the page to a server, wherein the embedded point data processing request comprises the embedded point data packet. Because the multiple buried points in the buried point queue are dynamically sampled and the buried point data is reported in batch, excessive requests are prevented from being sent to the server, and therefore two aspects of reporting enough buried point data and ensuring the performance of the client can be considered.
Please refer to fig. 1, which is a flowchart illustrating an embodiment of a buried point data reporting method according to the present application. The method comprises the following steps:
step S101: and storing the triggered buried points in the page to a buried point queue to be sent.
The pages include, but are not limited to, HTML pages, HTML5 pages, or native pages of mobile APP, such as HTML page of the day cat top page, HTML5 page of the day cat top page, and the like. The HTML page can be displayed through a browser installed in a personal computer, the HTML5 page can be displayed through a browser installed in a smart phone, and the native page can be displayed through a mobile APP installed in the smart phone.
The page comprises a plurality of buried points, and the applicable technology for burying the page comprises but is not limited to code buried point or visual buried point technology, and dynamic buried point technology can also be applied. At page or mobile APP initialization, the buried point tool library (SDK) for sending buried point data will be initialized.
The page developer introduces an SDK (software development kit) on a page, the SDK provides a point embedding method for directly calling a page code, the page code calls a method of a point embedding tool library and transmits information such as a to-be-sent embedded point name and a father embedded point name, the point embedding tool library adds the to-be-sent embedded point data into a sending queue (to-be-sent embedded point queue), and the finally-sent embedded point data packet is generated by sampling from the embedded point queue through a dynamic sampling algorithm at intervals.
In the prior art, when an event occurs, if a buried point corresponding to the event is a point to be sampled which is determined according to a fixed sampling ratio, a corresponding data sending interface in the SDK is called to send data; according to the method, when a certain event occurs, the embedded point is firstly sent to the embedded point queue to be sent, and the embedded point data is sent to the server only when a preset condition is met.
Step S103: and when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to form a buried point data packet of the page.
According to the method for reporting the buried point data, when the preset condition is met, the buried points in the buried point queue to be sent can be dynamically sampled through a preset dynamic sampling algorithm, and a buried point data packet reported to the server at this time of the page is formed.
The preset conditions include, but are not limited to, at least one of the following ways:
1) in the page loading process, after downloading of resource files (such as picture resources and the like) included in a page is completed, dynamically sampling buried points in the buried point queue through a preset dynamic sampling algorithm according to a first preset time interval;
2) and in the page display process, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a second preset time interval.
The first predetermined time interval is usually larger than the second predetermined time interval, because more client resources are consumed in the page loading process, for example, downloading resource files, etc., compared to the page displaying process. In this case, in order to avoid that frequent dynamic sampling affects the client performance, the first preset time interval is usually set to be larger, for example, the first preset time interval is set to be 5 seconds, and the second preset time interval is set to be 2 seconds.
In specific implementation, the preset dynamic sampling algorithm may include the following steps:
1) randomly extracting a common type of buried points from the buried point queue; randomly extracting the buried points with the error types from the buried point queue; 2) taking the maximum length of the request data packet allowed by the transmission protocol as a limit, splicing the extracted serialized character strings corresponding to the embedded points of the common type to the maximum extent, and obtaining a sampling embedded point data packet of the common type; splicing the extracted serialized character strings corresponding to the buried points of the error types to the maximum extent to obtain sampling buried point data packets of the error types; 3) generating a first data packet according to the sampling buried point data packet of the common type and a buried point sampling ratio, wherein the buried point sampling ratio is a ratio of the total number of the buried points of the common type to the number of the buried points of the common type which are successfully spliced; and generating a second data packet according to the sampling buried point data packet of the error type. The steps are described in detail below.
1) Randomly extracting a common type of buried points from the buried point queue; and randomly extracting the buried points of the error types from the buried point queue.
The types of the buried points can be divided into a common type and an error type. The error type buried point data generally has a large influence on the stability analysis and page adjustment of the page, and therefore, the error type buried point data is preferably reported to the server side, so as to ensure the accuracy of the page stability analysis result and provide more useful information for page adjustment.
The buried point data reporting method provided by the application carries out dynamic buried point sampling processing on buried points of a common type and a wrong type respectively. Since the buried point data collected within a period of time is concentrated in the buried point queue to be sent, in order to ensure that the reported buried point data is uniformly distributed on the time axis, it is necessary to first randomly extract the buried points of the common type from the buried point queue, and randomly extract the buried points of the wrong type from the buried point queue.
2) Taking the maximum length of the request data packet allowed by the transmission protocol as a limit, splicing the extracted serialized character strings corresponding to the embedded points of the common type to the maximum extent, and obtaining a sampling embedded point data packet of the common type; and splicing the serialized character strings corresponding to the extracted buried points of the error types to the maximum extent to obtain sampling buried point data packets of the error types.
In this embodiment, the buried point data packets are sent to the server via a network transmission protocol (e.g., HTTP protocol or HTTPs protocol), so that the length of a buried point data processing request is limited by the maximum length of the request packet allowed by the transmission protocol, i.e., the number of sampling buried points included in one request is limited by the maximum length of the request packet allowed by the transmission protocol.
The buried point is an object, and therefore, the buried point needs to be serialized first, and then the serialized buried point character strings need to be spliced to the maximum extent. In the present embodiment, a normal type of sample buried point data packet and an error type of sample buried point data packet are formed separately.
3) Generating a first data packet according to the sampling buried point data packet of the common type and a buried point sampling ratio, wherein the buried point sampling ratio is a ratio of the total number of the buried points of the common type to the number of the buried points of the common type which are successfully spliced; and generating a second data packet according to the sampling buried point data packet of the error type.
In this embodiment, the general type of embedded point data is sampled from the embedded point queue through a dynamic sampling algorithm, and after the general type of embedded point data is reported to the server in a packet manner, the general type of embedded point data in the embedded point queue is removed, so that when the general type of embedded point data is reported, an actual embedded point sampling ratio needs to be reported, so that the server can accurately know an actual situation of the embedded point data. Because different buried points comprise different information quantities, different buried point data packets spliced to the maximum limit generally comprise different buried point quantities. It can be seen that the actual buried point sampling ratio is a dynamic sampling ratio. In this embodiment, a first data packet, that is, a data packet corresponding to the normal type sampling buried point, is generated according to the normal type sampling buried point data packet and the buried point sampling ratio.
In this embodiment, all the error-type buried point data is reported to the server, the buried points of the server which are not reported this time are continuously left in the buried point queue, and the buried point data is sampled again when reported next time until all the error-type buried point data are reported to the server.
In practical applications, the data of the same buried point generated at different times may be included in the buried point queue. As a preferable scheme, before randomly extracting a general type of buried point from the buried point queue and before randomly extracting a wrong type of buried point from the buried point queue, the method may further include the following steps: merging the buried points with the same name in the buried point queue, and recording the merging number; correspondingly, the step of randomly extracting the common type of buried points from the buried point queue can be implemented by adopting the following modes: randomly extracting common type buried points from a buried point queue after the buried points with the same name are merged; the step of randomly extracting the buried point with the error type from the buried point queue can be realized by adopting the following modes: and randomly extracting the buried points with the error types from the buried point queue after the buried points with the same name are merged. By adopting the processing mode, the sending opportunity of the buried points with small data volume can be prevented from being preempted by the buried points with large data volume, and the server can be ensured to receive enough data of the buried points, so that the alarm accuracy is improved.
In specific implementation, after the merging processing is performed on the same-name buried points in the buried point queue, the method further includes the following steps: writing the combined number corresponding to the successfully spliced buried points of the common type into a first data packet; and writing the merging number corresponding to the buried point of the successfully spliced error type into a second data packet. By adopting the processing mode, the server can be ensured to acquire the real data burying condition, thereby improving the alarm accuracy.
In summary, by merging the buried points with the same name before the sampling buried point, the number of the buried points in the buried point queue can be compressed, so that the buried points appearing many times and the buried points appearing only once have the same sampled probability, and the buried points with less appearing times are prevented from being missed. In order to enable the server side to accurately analyze the data of the embedded points, the combined number of the embedded points needs to be reported to the server side together.
For example, after the SDK is introduced into the page, if window. ctktack ('name', 'parent') is executed, a buried point with a buried point name of name and a parent buried point name of parent is added to a buried point queue to be sent; if five buried points including name1, name1 (repeatedly sent twice), name2, name3, name4 and name5 exist in the buried point queue to be sent, the dynamic sampling algorithm screens the finally sent buried point data, for example, the finally sent name1|10 is sent randomly; name3|5, where 10 indicates that name1 appears 10 times in the buried point queue and 5 indicates that name3 appears 5 times in the buried point queue.
In addition, in practical application, the server side performs buried point data processing on a plurality of pages belonging to different projects. Since different pages of different projects may have the same embedded point identifier, in order to enable the server to distinguish these embedded points of the same name, it is preferable to add not only the data of the sampled embedded point to the embedded point data packet, but also the information of the page to which the embedded point belongs to the embedded point data packet.
The information of the buried point at least comprises a buried point name and at least one of the following information: the type of the buried point (such as a common type or an error type), the belonged father buried point, the belonged root buried point, the combined number of the same-name buried points, the buried point acquisition time and the like. For example, the parent node is a node corresponding to the resource file to be loaded, the child nodes may be nodes corresponding to the resource file to be loaded successfully and nodes corresponding to the resource file to be loaded unsuccessfully, the root node may be a node set when the resource file enters the Page, and the number of occurrences of the root node is actually the browsing volume (PV, Page View) of the Page.
The information of the page at least comprises a page identification and at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs. In specific implementation, the page code can be constructed by an executable code before being issued, relevant information (warehouse name, warehouse version and the like) of the page can be injected into the page code in the construction process, and the data can be sent by calling a method of a buried point tool library when the page code runs.
Step S105: and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet.
The method for reporting buried point data provided in the embodiment of the present application processes a common type of buried point and an error type of buried point, respectively, and thus, the sending a request for processing the buried point data of the page to a server may include the following specific steps: sending a first buried point data processing request aiming at the page to the server, wherein the first buried point data processing request comprises the first data packet; and sending a second embedded data processing request aiming at the page to the server, wherein the second embedded data processing request comprises the second data packet.
It should be noted that after the common type of embedded points are reported successfully, all the common type of embedded points need to be removed from the embedded point queue; and after the error type buried point reports successfully, only removing the successfully sent error type buried point from the buried point queue.
Please refer to fig. 2, which is a flowchart illustrating an embodiment of a buried point data reporting method according to the present application. In this embodiment, in the process of loading a page or displaying a page by a client, a page code invokes a buried point sending method provided by a buried point tool library, adds buried point data to be sent to a buried point queue to be sent, and executes a dynamic sampling algorithm once at intervals (which may be determined according to the running state of the current page of a user), where the running process of dynamic sampling is approximately as follows:
1) since the error data is important in the page stability, it is necessary to determine the type of the to-be-sent embedded point, and different processing logics and different queues are used for the embedded points of the normal type and the embedded points of the error type.
2) Whether the embedded points are of a common type or of an error type, the embedded points with the same name in the embedded point queue to be sent are merged, and the merged number is recorded, if 5 embedded points with the name of init exist, only one embedded point with the name of init exists in the queue after merging is completed, but the merged number of the embedded points is recorded as 5 in an embedded point data packet.
3) For the buried points in the queue, the buried points are sorted out of order as the preparation work of subsequent random sampling
4) And taking out the disordered buried point queues one by one, serializing the disordered buried point queues into character strings, splicing according to the maximum length limit of the HTTP request, and stopping splicing after the maximum length is reached.
5) For the merged common type of buried point queues, the ratio of the total number of buried points in the queues to the number of the finally successfully spliced buried points is recorded, and the value obtained by multiplying the merged number of each buried point by the ratio is the final number of the buried points. For example, if there are 100 general types of buried points in total and 25 are finally successfully spliced, the ratio of the total number in the queue to the number of the finally successfully spliced buried points is 4, and if the number of merged points in the buried points is 5, the number of the finally merged points is 20, which means that one piece of data of the buried points represents 20 pieces of data. After the data transmission is completed, the data are transmitted to the server side, and the buried point queue is emptied after the data transmission is completed.
6) And for the merged error type buried point queue, the merged error type buried point queue is directly sent to a server after splicing, for the successfully sent buried points, the successfully sent buried points are cleared from the queue, and for the unsuccessfully sent buried points, the unsuccessfully sent buried points participate in the calculation of the next dynamic sampling algorithm to wait for the next sending.
In the foregoing embodiment, a buried point data reporting method is provided, and correspondingly, the present application also provides a buried point data reporting device. The apparatus corresponds to an embodiment of the method described above.
Please refer to fig. 3, which is a schematic diagram of an embodiment of a buried point data reporting apparatus according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
The device for reporting buried point data in this embodiment includes:
the buried point acquisition unit 301 is configured to store the triggered buried points in the page to a buried point queue to be sent;
the buried point sampling unit 303 is configured to, when a preset condition is met, dynamically sample buried points in the buried point queue through a preset dynamic sampling algorithm to obtain buried point data packets corresponding to the sampling buried points;
a request sending unit 305, configured to send a buried point data processing request for the page to a server, where the buried point data processing request includes the buried point data packet.
Optionally, the type of the buried point includes a common type or an error type;
the buried point sampling unit 303 includes:
a random extraction subunit, configured to randomly extract a common type of buried point from the buried point queue; randomly extracting the buried points with the error types from the buried point queue;
the embedded point data splicing subunit is used for splicing the extracted serialized character strings corresponding to the embedded points of the common type to the maximum extent by taking the maximum length of the request data packet allowed by the transmission protocol as the limit to obtain a sampling embedded point data packet of the common type; splicing the extracted serialized character strings corresponding to the buried points of the error types to the maximum extent to obtain sampling buried point data packets of the error types;
the data packet generating subunit is used for generating a first data packet according to the sampling buried point data packet of the common type and a buried point sampling ratio, wherein the buried point sampling ratio is a ratio of the total number of the buried points of the common type to the number of the buried points which are successfully spliced; generating a second data packet according to the sampling buried point data packet of the error type;
the request transmission unit 305 includes:
a first request sending subunit, configured to send a first embedded data processing request for the page to the server, where the first embedded data processing request includes the first data packet;
and the second request sending subunit is configured to send a second embedded data processing request for the page to the server, where the second embedded data processing request includes the second data packet.
Optionally, the buried point sampling unit 303 further includes:
the same-name buried point merging subunit is used for merging the same-name buried points in the buried point queue and recording the merging number;
the random extraction subunit is specifically configured to randomly extract a common type of buried point from the buried point queue after merging of the same-name buried points; randomly extracting the buried points with the wrong types from the buried point queue after the buried points with the same name are merged;
the buried point sampling unit 303 further includes:
a merging quantity writing unit, configured to write the merging quantity corresponding to the successfully spliced common-type buried point into a first data packet; and writing the merging number corresponding to the buried point of the successfully spliced error type into a second data packet.
Optionally, the apparatus further comprises:
the buried point queue cleaning unit is used for removing the buried points of the error types which are successfully sent from the buried point queue; and removing the common type of buried point from the buried point queue.
Optionally, the preset condition is as follows:
in the page loading process, after the resource file included in the page is downloaded, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a first preset time interval;
and in the page display process, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a second preset time interval.
Optionally, the buried point data packet includes: the information of the buried point and the information of the page to which the buried point belongs.
Optionally, the information of the buried point at least includes a name of the buried point, and includes at least one of the following information: the type of the buried point, the father buried point, the root buried point, the merging quantity and the acquisition time.
Optionally, the information of the page at least includes a page identifier, and includes at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
Please refer to fig. 4, which is a schematic diagram of an embodiment of an electronic device according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
An electronic device of the present embodiment includes: a processor 401; and a memory 403, configured to store a program for implementing the buried point data reporting method, where after the device is powered on and the processor runs the program for implementing the buried point data reporting method, the following steps are performed: storing the triggered embedded points in the page to an embedded point queue to be sent; when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to obtain buried point data packets corresponding to the sampling buried points; and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet.
Corresponding to the buried point data reporting method, the application also provides a buried point data processing method. Please refer to fig. 5, which is a flowchart illustrating an embodiment of a method for processing buried point data according to the present application, and details of the same portions in this embodiment as those in the first embodiment are not repeated, please refer to corresponding portions in the first embodiment. The buried point data processing method provided by the application comprises the following steps:
step S501: receiving a plurality of buried point data processing requests aiming at a specific page and sent by at least one client, wherein the buried point data processing requests comprise buried point data packets.
The specific page includes, but is not limited to, an HTML page, an HTML5 page, or a native page of a mobile APP, the HTML page is displayable through a browser installed in a personal computer, the HTML5 page is displayable through a browser installed in a smart phone, and the native page is displayable through a mobile APP installed in a smart phone.
A large number of users access the specific page (e.g., the head page of a mobile phone skatecat, etc.), and then send the buried point data to the server through the network. In this embodiment, the server first stores these network requests (including the sent data related to the embedded point, the terminal type, the user id, the user IP, the data time, etc.) to the local server, and triggers the real-time computing service by sending a message. After receiving the message, the real-time computing service disassembles and analyzes the original data of the network requests to generate data convenient for subsequent alarm processing and website display, for example, the analyzed buried point data includes: the system comprises a page identifier, a buried point name, a father buried point name, a root buried point name, a warehouse version, a merging number, sending time, a browser, an operating system and the like, and is stored in a column database for convenient query.
According to the embedded point data processing method provided by the embodiment of the application, after the embedded point data processing request sent by the client is received, the embedded point data packet needs to be analyzed, and the embedded point data included in the embedded point data packet is obtained. The embedded data packet of the specific page may be an embedded data packet reported by a fixed sampling ratio in the prior art, or an embedded data packet formed by dynamic sampling by the embedded data reporting method in the above embodiment. When the buried point data packet is a data packet reported by a fixed sampling ratio, the buried point data packet usually only comprises one buried point data; when the buried point data packet is a data packet reported by a dynamic sampling algorithm, the buried point data packet usually includes a plurality of buried point data.
In this embodiment, the buried point data packet is formed by dynamic sampling according to the buried point data reporting method in the first embodiment. When the received buried point data packet is a common type buried point data packet, the buried point data packet comprises a plurality of buried point data and buried point sampling ratios of common types; in this case, parsing the buried point data packet may include the steps of: 1) determining the occurrence frequency of each buried point in the buried point data packet according to the buried point sampling ratio; 2) and forming a plurality of buried point data corresponding to the buried point data packet according to each buried point and the occurrence frequency.
If the client side carries out merging processing on the same-name buried points before sampling, the buried point data packet also comprises information of the merging number of each buried point; in this case, the product of the buried-point sampling ratio and the combined number of the buried points is taken as the number of occurrences of the buried points.
When the received buried point data packet is an error type buried point data packet, the buried point data packet comprises a plurality of error type buried point data, and simultaneously can also comprise the merging number of each buried point; in this case, the parsing of the buried point data packet can be implemented as follows: and forming a plurality of embedded point data corresponding to the embedded point data packet according to the plurality of embedded points and the merging number of each embedded point.
The information of the buried point at least comprises a buried point name and at least one of the following information: the type of the buried point, the father buried point, the root buried point, the number of the merged points, the acquisition time and the like.
The buried point data packet may include information of the specific page. The information of the specific page at least comprises a page identification and at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
In order to facilitate subsequent alarm processing and website display data, the method for processing the data of the buried points, provided by the application, can further comprise the step of storing the plurality of data of the buried points obtained through analysis. In specific implementation, the information of the specific page can be stored corresponding to each buried point data, and the information of the client can be stored corresponding to each buried point data. The information of the client comprises at least one of the following information: device type, browser, operating system, user identification, IP address.
Step S503: and according to a preset time interval, counting the buried point data in the plurality of buried point data packets of the specific page received in the time interval range to obtain the current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point.
Different from the mode that the alarm is triggered when the occurrence frequency of the buried point reaches the preset frequency threshold value in the prior art, the buried point data processing method provided by the application firstly determines the alarm probability of the buried point according to the historical data related to the preset monitoring buried point through a preset alarm probability algorithm, and then compares the determined alarm probability with the preset alarm probability threshold value to determine the preset monitoring buried point needing to be alarmed.
In order to calculate and obtain the alarm probability of the preset monitoring buried point, the data of the preset index related to the alarm probability algorithm corresponding to the preset monitoring buried point needs to be obtained first.
According to the buried point data processing method, according to a preset time interval, the buried point data in the plurality of buried point data packets of the specific page received in the time interval range are counted, and the current data of the preset index related to the alarm probability algorithm corresponding to the preset monitoring buried point is obtained.
The preset time interval is set according to the service requirement, for example, the preset time interval can be set to be 5 minutes, that is, the buried point needing to be alarmed is determined every 5 minutes.
The specific page comprises a plurality of buried points, only a part of the buried points may be buried points needing monitoring and alarming, the buried points needing monitoring and alarming are called preset monitoring buried points, for example, a sub-buried point with successful downloading and a sub-buried point with failed downloading are set for the buried points downloading a certain resource file in a head page of a Tianmao, and if the condition of successful downloading of the resource is desired to be monitored, the sub-buried point with successful downloading can be set as the preset monitoring buried point.
The alarm probability algorithm may be related to only one preset index, or may be related to a plurality of preset indexes. In specific implementation, a user can set which buried points need to be monitored (i.e., designate preset monitoring buried points) through a configuration page, and can also designate an applicable alarm probability algorithm for each preset monitoring buried point through the configuration page.
Step S505: and determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index.
The alarm probability algorithm may be related to only one preset index, for example, the preset index related to the alarm probability algorithm includes, but is not limited to, any one of the following indexes: the number of accumulations, the ratio of the number of accumulations of parent-child buried points (parent buried points and child buried points), the ratio of the number of accumulations of root-child buried points (root buried points and child buried points), the daily-to-ring ratio of the number of accumulations, the circumferential ratio of the number of accumulations, and the like.
In the case where the alarm probability algorithm is only associated with one preset index, the alarm probability algorithm may include the following specific steps: 1) acquiring the average value and the variance of the preset index according to the current data and the historical data of the preset index; 2) and acquiring the alarm probability of the preset index through a Gaussian model according to the data of the preset index, the average value and the variance, and taking the alarm probability as the alarm probability of the preset monitoring buried point. For example, a certain service may be alarmed by selecting the accumulated number of times, so that if the difference between the accumulated number of times and the historical average value is large, an alarm is triggered.
In specific implementation, the embedded data in the multiple embedded data packets of the specific page received in the time interval range may be firstly counted to generate a summary data, which includes: page identification, buried point name, warehouse name, time period and accumulated quantity in the time period; then, calculating to obtain data of each preset index (namely the current data of the preset index) related to the preset monitoring buried point; then, the average value and the variance of the preset indexes can be obtained according to the current data and the historical data of the preset indexes; and finally, according to the data of the preset index, the average value and the variance, obtaining the alarm probability of the preset index through a Gaussian model, and using the alarm probability as the alarm probability of the preset monitoring buried point.
The alarm probability algorithm may also be related to a plurality of preset indicators, for example, the preset indicators related to the alarm probability algorithm include at least two of the following indicators: the accumulated times, the ratio of the accumulated times of the father-son buried points, the ratio of the accumulated times of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times. In this case, alarm probability weights need to be set for each preset index, and the sum of the alarm probability weights corresponding to each preset index is 1.
In the case where the alarm probability algorithm is associated with a plurality of preset indicators, the alarm probability algorithm may include the following specific steps: 1) acquiring the average value and the variance of each preset index according to the current data and the historical data of the preset index; 2) obtaining the alarm probability of each preset index through a Gaussian model according to the data, the average value and the variance of the preset index; 3) and acquiring the alarm probability of the preset monitoring buried point according to the preset alarm probability weight corresponding to each preset index and the alarm probability of each preset index.
For example, a certain service, namely, the flow changes periodically, a daily-to-annular ratio and a weekly-to-annular ratio can be selected for alarming, and the weights of the daily-to-annular ratio and the weekly-to-annular ratio can be distributed in a ratio of 7:3, so that if data is different from yesterday, an alarm is easier to trigger, and if the data is different from last week, an alarm is also triggered, but the priority is lower; if the accumulated times of the buried point A are reported to be 100 in the time period of 10:05-10:10 on 2016/12/2 days; reporting the accumulated times of the buried point A to be 70 in a time period of 10:05-10:10 on 2016/12/1 days; reporting that the accumulated times of the buried point A is 80 in 2016/11/2 days of 10:05-10:10, wherein the daily-to-annular ratio of the buried point A is 100/70, and the cyclic ratio of the buried point A is 100/80; according to the date-to-ring ratio historical data and the current data (100/70) of the buried point A in the time period of 10:05-10:10, calculating to obtain the date-to-ring ratio mean value of the buried point A in the time period of 10:05-10:10, and if the date-to-ring ratio mean value is 100/70 and the date-to-ring ratio variance is 10%, the alarm probability is 0 because the current date-to-ring ratio data of the buried point A is the same as the historical mean value; in addition, according to the historical data of the peripherical ratio of the buried point A in the time period of 10:05-10:10 and the current data (100/80), the mean value of the peripherical ratio of the buried point A in the time period of 10:05-10:10 can be calculated, and if the mean value of the peripherical ratio is 100/50 and the variance of the peripherical ratio is 10%, the alarm probability is higher (for example, 80%) because the current peripherical ratio of the buried point A is far from the historical mean value.
Please refer to fig. 6, which is a flowchart illustrating an embodiment of a buried point data processing method according to the present application. In this embodiment, the average value and the variance of the preset index obtained each time are recorded, and when the current data is processed, the average value and the variance of the preset index can be directly calculated according to the current data of the preset index and the average value and the variance obtained by the previous calculation; by adopting the processing mode, the calculation amount can be effectively reduced; therefore, the consumption of computing resources can be effectively reduced. In the present embodiment, the average and variance of the preset index are calculated using an incremental calculation framework.
Step S507: and determining a preset monitoring buried point needing alarming according to the alarming probability and a preset alarming probability threshold value.
After the alarm probability of the preset monitoring buried point is determined, whether the preset monitoring buried point needs to be alarmed or not can be determined according to the alarm probability and a preset alarm probability threshold value. In this embodiment, when the alarm probability exceeds a preset alarm probability threshold, the preset monitoring buried point is determined as a buried point that needs to be alarmed.
The preset alarm probability threshold value can be set according to experience and specific service requirements. In specific implementation, different alarm probability threshold values can be set for different preset monitoring buried points.
In addition, the buried point data processing method provided by the application can further comprise the following steps: 1) generating a corresponding alarm message for the preset monitoring buried point needing to be alarmed; 2) acquiring an alarm person and an alarm mode corresponding to the preset monitoring buried point needing to be alarmed; 3) and providing the alarm message to the alarm person according to the alarm mode. By adopting the processing mode, the alarm messages of different burial points can be pushed to the user in charge of the burial point.
In this embodiment, the local database at the server side stores the alarm person information and the alarm mode information respectively corresponding to each preset monitoring buried point. After the corresponding alarm message is generated for the preset monitoring buried point needing to be alarmed, the alarm person and the alarm mode respectively corresponding to each preset monitoring buried point can be obtained in a database inquiring mode, and then each alarm message is pushed to the user in charge of the buried point.
The alarm modes include but are not limited to: short message of mobile phone, mail, instant communication message, etc.
In the foregoing embodiment, a buried point data processing method is provided, and correspondingly, the present application also provides a buried point data processing apparatus. The apparatus corresponds to an embodiment of the method described above.
Please refer to fig. 7, which is a schematic diagram of an embodiment of a buried point data processing apparatus according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
A buried point data processing apparatus of this embodiment includes:
a request receiving unit 701, configured to receive multiple buried point data processing requests for a specific page sent by at least one client, where the buried point data processing requests include buried point data packets;
the data statistics unit 703 is configured to perform statistics on buried point data in the multiple buried point data packets of the specific page received within the time interval range according to a preset time interval, so as to obtain current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point;
an alarm probability calculation unit 705, configured to determine, according to the current data and the historical data of the preset index, an alarm probability of the preset monitoring buried point through the alarm probability algorithm;
and an alarm buried point determining unit 707, configured to determine a preset monitoring buried point that needs to be alarmed according to the alarm probability and a preset alarm probability threshold.
Optionally, the preset index related to the alarm probability algorithm includes any one of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
the alarm probability calculation unit 705 includes:
the mean value and variance obtaining subunit is used for obtaining the mean value and the variance of the preset index according to the current data and the historical data of the preset index;
and the alarm probability obtaining subunit is used for obtaining the alarm probability of the preset index through a Gaussian model according to the data of the preset index, the average value and the variance, and the alarm probability is used as the alarm probability of the preset monitoring buried point.
Optionally, the preset indexes related to the alarm probability algorithm include at least two of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
the alarm probability calculation unit 705 includes:
the mean value and variance obtaining subunit is used for obtaining the mean value and variance of each preset index according to the current data and the historical data of the preset index;
the first alarm probability obtaining subunit is used for obtaining the alarm probability of each preset index through a Gaussian model according to the current data, the average value and the variance of the preset index;
and the second alarm probability obtaining subunit is used for obtaining the alarm probability of the preset monitoring buried point according to the preset alarm probability weight corresponding to each preset index and the alarm probability of each preset index.
Optionally, the preset monitoring buried points include a plurality of preset monitoring buried points, different preset monitoring buried points correspond to different alarm probability algorithms, and different preset monitoring buried points correspond to different alarm probability thresholds.
Optionally, the apparatus further comprises:
the alarm message generating unit is used for generating a corresponding alarm message for the preset monitoring buried point needing alarming;
the alarm person and alarm mode acquisition unit is used for acquiring the alarm person and the alarm mode corresponding to the preset monitoring buried point needing alarming;
and the alarm message providing unit is used for providing the alarm message for the alarm person according to the alarm mode.
Optionally, the alarm mode includes: short message of mobile phone, mail, instant communication.
Optionally, the apparatus further comprises:
and the buried point data packet analyzing unit is used for analyzing the buried point data packet to obtain buried point data corresponding to the buried point data packet.
Optionally, the buried point data packet includes a plurality of buried point data of a common type and a buried point sampling ratio;
the buried point data packet analysis unit comprises:
the appearance frequency determining subunit is used for determining the appearance frequency of each buried point in the buried point data packet according to the buried point sampling ratio;
and the buried point data forming subunit is used for forming a plurality of buried point data corresponding to the buried point data packet according to each buried point and the occurrence frequency.
Optionally, the buried point data packet includes a merging number of each buried point;
and the occurrence frequency determining subunit is specifically configured to use a product of the buried point sampling ratio and the combined number of the buried points as the occurrence frequency of the buried points.
Optionally, the buried point data packet includes a plurality of buried point data of an error type and a merging number of each buried point;
the embedded data packet analysis unit is specifically configured to form a plurality of embedded data packets corresponding to the embedded data packet according to the plurality of embedded points and the number of merged embedded points.
Optionally, the information of the buried point at least includes a name of the buried point, and includes at least one of the following information: the type of the embedded points, the belonged father embedded points, the merging quantity and the acquisition time.
Optionally, the buried point data packet includes information of the specific page.
Optionally, the information of the specific page at least includes a page identifier, and includes at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
Optionally, the apparatus further comprises:
and the buried point data storage unit is used for storing the plurality of buried point data obtained by analysis.
Optionally, the apparatus further comprises:
and the page information storage unit is used for storing the information of the specific page corresponding to each buried point data.
Optionally, the apparatus further comprises:
and the client information storage unit is used for storing the information of the client corresponding to each buried point data.
Optionally, the information of the client includes at least one of the following information: device type, browser, operating system, user identification, IP address.
Please refer to fig. 8, which is a diagram illustrating an embodiment of an electronic device according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
An electronic device of the present embodiment includes: a processor 801; and a memory 803 for storing a program for implementing the buried point data processing method, wherein the following steps are executed after the apparatus is powered on and the program for the buried point data processing method is executed by the processor: receiving a plurality of buried point data processing requests aiming at a specific page, which are sent by at least one client, wherein the buried point data processing requests comprise buried point data packets; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point; determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index; and determining a preset monitoring buried point needing alarming according to the alarming probability and a preset alarming probability threshold value.
An embodiment of the present application further provides a buried point data processing system, as shown in fig. 9, the system includes the buried point data processing device 901 and the buried point data reporting device 902 described in the foregoing embodiment. The buried point data processing device is usually deployed in a server, but is not limited to the server, and may be any device capable of implementing the buried point data processing method; the buried point data reporting device is usually deployed in terminal devices such as mobile communication equipment, personal computers, PADs, ipads and the like.
For example, the embedded point data reporting device is deployed on a smart phone, and can store a triggered embedded point in a page to an embedded point queue to be sent, dynamically sample the embedded point in the embedded point queue through a preset dynamic sampling algorithm when a preset condition is met, form an embedded point data packet of the page, and send an embedded point data processing request for the page to a server, where the embedded point data processing request includes the embedded point data packet; the embedded data processing device is deployed on a server, and is used for receiving a plurality of embedded data processing requests aiming at a page and sent by at least one client, wherein the embedded data processing requests comprise embedded data packets, counting the embedded data in the embedded data packets of the page received in a time interval range according to a preset time interval to obtain the current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring embedded point, then determining the alarm probability of the preset monitoring embedded point through the alarm probability algorithm according to the current data and historical data of the preset index, and finally determining the preset monitoring embedded point needing alarming according to the alarm probability and a preset alarm probability threshold.
Please refer to fig. 10, which is a detailed diagram of an embodiment of a buried point data processing system according to the present application. In this embodiment, the whole work flow of reporting the data of the embedded point by the client and performing the embedded point alarm processing by the server involves the following steps:
1) the page developer introduces a buried point tool library on a page, the buried point tool library (SDK) provides a buried point method for directly calling page codes, the page codes call the buried point method of the buried point tool library and transmit information such as buried point names to be sent and father buried point names, the buried point tool library adds buried point data to be sent into a buried point queue to be sent, and finally sent buried point data is generated by sampling from the buried point queue by adopting a dynamic sampling algorithm at intervals.
2) The operation of code construction is executed before the page code is issued, the relevant information (such as warehouse name, warehouse version and the like) of the page code is injected into the page code in the construction process, and the page code sends the data when calling the method of the buried point tool library in the running process.
3) When a large number of users access the page, the codes of the buried points are operated, at the moment, the data are sent to a data collection application through a network, the data collection application stores the network requests (including the sent data related to the buried points, the terminal type, the user id, the user IP, the data time and the like) to the local part of the server, and the Galaxy is triggered to calculate in real time by sending a message. Galaxy is an incremental computation model, not a stream computation, which is a stateful computation. Batch calculation is used, the output result of each time is only related to the data scanned in by the total amount, and the calculation is idempotent; by using incremental calculation, each batch of calculation results is calculated by the batch of data and historical batch results, so that the consumption of calculation resources can be effectively reduced.
4) After Galaxy calculates and receives the message in real time, the original data of the network requests are disassembled and analyzed to generate data (such as page identification, node name, father node name, warehouse version, merging number, sending time, browser, operating system and the like) which are convenient for subsequent alarm monitoring and website display, and the data are stored in a column database for convenient query. Meanwhile, a piece of summarized data (page identification, node name, warehouse name, time period, total number in the time period) is generated according to a preset time interval and is delivered to a stability (namely, alarm monitoring) application (CodeTrack-worker).
5) After the stability application (CodeTrack-worker) acquires the summarized data calculated by Galaxy in real time, the data is combined with a stability algorithm (for example, an algorithm according to an alarm probability and a preset alarm probability threshold) to determine whether an alarm needs to be sent.
6) If the stability algorithm considers that the data flow is abnormal, the alarm messages are pushed to a message queue, meanwhile, the data display application monitors the messages, finds a configured alarm person list and configured alarm modes (nailing, vigorous, short messages, mails and the like) after the messages are obtained, and sends the alarm.
7) After receiving the alarm, the alarm person can enter a data display application (CodeTrack-web), the data display application can acquire stored data from the column-type database and display the data to the alarm person in a graph curve mode, and the alarm person looks up a data fluctuation curve today and a data fluctuation curve yesterday to determine the time when the problem occurs. And then, positioning the specific stable scene and the influence range by checking data such as the injection version, the terminal type, the error information and the like.
8) And after the page developer corrects the problems according to the information, the page code is re-released online.
Corresponding to the above buried point data reporting method, the present application also provides another buried point data processing method. Please refer to fig. 11, which is a flowchart illustrating another embodiment of a method for processing buried point data according to the present application, wherein parts of this embodiment that are the same as those of the first embodiment are not repeated, and refer to corresponding parts in the first embodiment. The application provides another buried point data processing method, which comprises the following steps:
step S1101: receiving a plurality of buried point data processing requests aiming at a specific page and sent by at least one client, wherein the buried point data processing requests comprise buried point data packets.
Step S1103: and according to a preset time interval, counting the data of the embedded points in the multiple embedded point data packets of the specific page received in the time interval range to obtain the current data of a preset index corresponding to a preset monitoring embedded point.
The preset index includes any one of the following indexes: the ratio of the number of times of integration of father-son buried points, the ratio of the number of times of integration of root-son buried points, the daily-ring ratio of the number of times of integration, and the weekly-ring ratio of the number of times of integration.
Step S1105: and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
For example, a certain service may select the ratio of the number of times of the parent-child buried point to alarm, and an alarm may be triggered if the ratio of the number of times of the current time exceeds a preset threshold, for example, if the parent buried point is a buried point corresponding to the resource file to be loaded, and the child buried point is a buried point corresponding to the successful loading, the client will report data of the parent buried point and data of the child buried point.
In the above embodiments, another buried point data processing method is provided, and correspondingly, another buried point data processing device is also provided in the present application. The apparatus corresponds to an embodiment of the method described above.
Please refer to fig. 12, which is a schematic diagram of another embodiment of a buried point data processing apparatus according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
Another buried point data processing apparatus of this embodiment includes:
a request receiving unit 1201, configured to receive a buried point data processing request for a specific page sent by at least one client, where the buried point data processing request includes a buried point data packet;
a data counting unit 1203, configured to count, according to a preset time interval, data of embedded points in multiple embedded point data packets of the specific page received within the time interval range, so as to obtain current data of a preset index corresponding to a preset monitoring embedded point;
an alarm buried point determining unit 1205 is configured to determine a preset monitoring buried point that needs to be alarmed according to the current data of the preset index and a preset threshold corresponding to the preset index.
Optionally, the preset index includes any one of the following indexes: the ratio of the number of times of integration of father-son buried points, the ratio of the number of times of integration of root-son buried points, the daily-ring ratio of the number of times of integration, and the weekly-ring ratio of the number of times of integration.
Please refer to fig. 13, which is a diagram illustrating an embodiment of an electronic device according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
An electronic device of the present embodiment includes: a processor 1301; and a memory 1303 for storing a program for implementing the buried point data processing method, wherein the following steps are executed after the device is powered on and the program for implementing the buried point data processing method is executed by the processor: receiving a buried point data processing request aiming at a specific page and sent by at least one client, wherein the buried point data processing request comprises a buried point data packet; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index corresponding to a preset monitoring buried point; and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
The embodiment of the present application further provides another buried point data processing system, as shown in fig. 14, the system includes another buried point data processing device 1401 and a buried point data reporting device 1402, which are described in the foregoing embodiment. The another buried point data processing device is usually deployed in a server, but is not limited to the server, and may be any device capable of implementing the another buried point data processing method; the buried point data reporting device is usually deployed in terminal devices such as mobile communication equipment, personal computers, PADs, ipads and the like.
For example, the embedded point data reporting device is deployed on a smart phone, and can store a triggered embedded point in a page to an embedded point queue to be sent, dynamically sample the embedded point in the embedded point queue through a preset dynamic sampling algorithm when a preset condition is met, form an embedded point data packet of the page, and send an embedded point data processing request for the page to a server, where the embedded point data processing request includes the embedded point data packet; the other buried point data processing device is deployed on a server and receives a buried point data processing request which is sent by at least one client and aims at a specific page, and the buried point data processing request comprises a buried point data packet; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the specific page received in the time interval range to obtain current data of a preset index corresponding to a preset monitoring buried point; and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
1. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
2. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims (33)

1. A buried point data reporting method is characterized by comprising the following steps:
storing the triggered embedded points in the page to an embedded point queue to be sent;
when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to form a buried point data packet of the page;
the preset conditions are as follows:
in the page loading process, after the resource file included in the page is downloaded, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a first preset time interval;
in the page display process, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a second preset time interval; and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet.
2. The buried point data reporting method of claim 1, wherein the type of the buried point includes a normal type or an error type;
the preset dynamic sampling algorithm comprises the following steps:
randomly extracting a common type of buried points from the buried point queue; randomly extracting the buried points with the error types from the buried point queue;
taking the maximum length of the request data packet allowed by the transmission protocol as a limit, splicing the extracted serialized character strings corresponding to the embedded points of the common type to the maximum extent, and obtaining a sampling embedded point data packet of the common type; splicing the extracted serialized character strings corresponding to the buried points of the error types to the maximum extent to obtain sampling buried point data packets of the error types;
generating a first data packet according to the sampling buried point data packet of the common type and a buried point sampling ratio, wherein the buried point sampling ratio is a ratio of the total number of the buried points of the common type to the number of the buried points which are successfully spliced; generating a second data packet according to the sampling buried point data packet of the error type;
the sending of the buried point data processing request for the data packet to the server includes:
sending a first buried point data processing request aiming at the page to the server, wherein the first buried point data processing request comprises the first data packet; and sending a second embedded data processing request aiming at the page to the server, wherein the second embedded data processing request comprises the second data packet.
3. The buried point data reporting method of claim 2, wherein before randomly extracting a normal type of buried point from the buried point queue and before randomly extracting an error type of buried point from the buried point queue, the method further comprises:
merging the buried points with the same name in the buried point queue, and recording the merging number;
randomly extracting a common type of buried point from the buried point queue by adopting the following method:
randomly extracting common type buried points from a buried point queue after the buried points with the same name are merged;
randomly extracting the buried points with error types from the buried point queue by adopting the following method:
randomly extracting buried points of error types from the buried point queue after the buried points with the same name are merged;
the method further comprises the following steps:
writing the combined number corresponding to the successfully spliced buried points of the common type into a first data packet; and writing the merging number corresponding to the buried point of the successfully spliced error type into a second data packet.
4. The buried point data reporting method of claim 2, further comprising:
after the common type of embedded points are reported successfully, all the common type of embedded points need to be eliminated from the embedded point queue; and after the error type buried point reports successfully, only removing the successfully sent error type buried point from the buried point queue.
5. The buried point data reporting method of claim 1, wherein the buried point data packet comprises: the information of the buried point and the information of the page to which the buried point belongs.
6. The buried point data reporting method of claim 5, wherein the information of the buried point at least includes a name of the buried point and at least one of the following information: the type of the buried point, the father buried point, the root buried point, the merging quantity and the acquisition time.
7. The buried point data reporting method of claim 5, wherein the information of the page at least includes a page identifier and at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
8. A buried point data reporting device is characterized by comprising:
the embedded point acquisition unit is used for storing the triggered embedded points in the page to an embedded point queue to be sent;
the embedded point sampling unit is used for dynamically sampling the embedded points in the embedded point queue through a preset dynamic sampling algorithm when a preset condition is met, and acquiring an embedded point data packet corresponding to the sampling embedded point;
the preset conditions are as follows:
in the page loading process, after the resource file included in the page is downloaded, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a first preset time interval;
in the page display process, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a second preset time interval;
and the request sending unit is used for sending a buried point data processing request aiming at the page to a server, and the buried point data processing request comprises the buried point data packet.
9. An electronic device, comprising:
a processor; and
the device is powered on and executes the program of the buried point data reporting method through the processor, and then the following steps are executed: storing the triggered embedded points in the page to an embedded point queue to be sent; when a preset condition is met, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm to obtain buried point data packets corresponding to the sampling buried points; the preset conditions are as follows:
in the page loading process, after the resource file included in the page is downloaded, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a first preset time interval;
in the page display process, dynamically sampling the buried points in the buried point queue through a preset dynamic sampling algorithm according to a second preset time interval;
and sending a buried point data processing request aiming at the page to a server, wherein the buried point data processing request comprises the buried point data packet.
10. A buried point data processing method is characterized by comprising the following steps:
receiving a plurality of buried point data processing requests aiming at a page, which are sent by at least one client, wherein the buried point data processing requests comprise buried point data packets;
according to a preset time interval, counting buried point data in a plurality of buried point data packets of the page received in the time interval range to obtain current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point;
the preset indexes related to the alarm probability algorithm comprise at least one of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index;
and determining a preset monitoring buried point needing alarming according to the alarming probability and a preset alarming probability threshold value.
11. The buried point data processing method according to claim 10, wherein the preset indexes related to the alarm probability algorithm include one index:
the alarm probability algorithm comprises:
acquiring the average value and the variance of the preset index according to the current data and the historical data of the preset index;
and acquiring the alarm probability of the preset index through a Gaussian model according to the data of the preset index, the average value and the variance, and taking the alarm probability as the alarm probability of the preset monitoring buried point.
12. The buried point data processing method according to claim 10, wherein the preset indexes related to the alarm probability algorithm include at least two indexes: the alarm probability algorithm comprises:
acquiring the average value and the variance of each preset index according to the current data and the historical data of the preset index;
obtaining the alarm probability of each preset index through a Gaussian model according to the data, the average value and the variance of the preset index;
and acquiring the alarm probability of the preset monitoring buried point according to the preset alarm probability weight corresponding to each preset index and the alarm probability of each preset index.
13. The buried point data processing method according to claim 10, wherein the preset monitoring buried points include a plurality of preset monitoring buried points, different preset monitoring buried points correspond to different alarm probability algorithms, and different preset monitoring buried points correspond to different alarm probability thresholds.
14. The buried point data processing method according to claim 10, further comprising:
generating a corresponding alarm message for the preset monitoring buried point needing to be alarmed;
acquiring an alarm person and an alarm mode corresponding to the preset monitoring buried point needing to be alarmed;
and providing the alarm message to the alarm person according to the alarm mode.
15. The buried point data processing method according to claim 14, wherein the alarm mode includes: short message of mobile phone, mail, instant communication.
16. The buried point data processing method according to claim 10, further comprising, after receiving the buried point data processing request:
and analyzing the buried point data packet to obtain buried point data corresponding to the buried point data packet.
17. The buried point data processing method according to claim 16, wherein the buried point data packet includes a plurality of buried point data of a common type and a buried point sampling ratio;
the analyzing the buried point data packet and obtaining the buried point data corresponding to the buried point data packet includes:
determining the occurrence frequency of each buried point in the buried point data packet according to the buried point sampling ratio;
and forming a plurality of buried point data corresponding to the buried point data packet according to each buried point and the occurrence frequency.
18. The buried point data processing method according to claim 17, wherein the buried point data packet includes a merging number of each buried point;
and determining the occurrence frequency of each buried point in the buried point data packet according to the buried point sampling ratio by adopting the following mode:
and taking the product of the sampling ratio of the buried points and the combined number of the buried points as the occurrence number of the buried points.
19. The buried point data processing method according to claim 16, wherein the buried point data packet includes a plurality of buried point data of an error type and a merging number of each buried point;
analyzing the buried point data packet, and obtaining buried point data corresponding to the buried point data packet by adopting the following mode:
and forming a plurality of buried point data corresponding to the buried point data packet according to the plurality of buried points and the merging number of each buried point.
20. The buried point data processing method according to claim 10, wherein the information of the buried point at least includes a name of the buried point, and includes at least one of: the type of the embedded points, the belonged father embedded points, the merging quantity and the acquisition time.
21. The buried point data processing method according to claim 10, wherein the buried point data packet includes information of the page.
22. The buried point data processing method of claim 21, wherein the information of the page at least includes a page identifier and at least one of the following information: the name of the repository to which the page belongs and the version of the repository to which the page belongs.
23. The buried point data processing method according to claim 16, further comprising:
and storing the plurality of buried point data obtained by analysis.
24. The buried point data processing method according to claim 23, further comprising:
and storing the information of the page corresponding to each buried point data.
25. The buried point data processing method according to claim 23, further comprising:
and storing the information of the client corresponding to each buried point data.
26. The buried point data processing method of claim 25, wherein the information of the client includes at least one of: device type, browser, operating system, user identification, IP address.
27. A buried point data processing apparatus, comprising:
the system comprises a request receiving unit, a processing unit and a processing unit, wherein the request receiving unit is used for receiving a plurality of buried point data processing requests aiming at a page and sent by at least one client, and the buried point data processing requests comprise buried point data packets;
the data statistics unit is used for carrying out statistics on the buried point data in the multiple buried point data packets of the page received in the time interval range according to a preset time interval to obtain the current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point;
the preset indexes related to the alarm probability algorithm comprise at least one of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
the alarm probability calculation unit is used for determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index;
and the alarm buried point determining unit is used for determining the preset monitoring buried point needing alarming according to the alarm probability and a preset alarm probability threshold value.
28. An electronic device, comprising:
a processor; and
a memory for storing a program for implementing the buried point data processing method, wherein the following steps are executed after the device is powered on and the program for implementing the buried point data processing method is executed by the processor: receiving a plurality of buried point data processing requests aiming at a page, which are sent by at least one client, wherein the buried point data processing requests comprise buried point data packets; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the page received in the time interval range to obtain current data of a preset index related to an alarm probability algorithm corresponding to a preset monitoring buried point; the preset indexes related to the alarm probability algorithm comprise at least one of the following indexes: the accumulated times, the accumulated time ratio of the father-son buried points, the accumulated time ratio of the root-son buried points, the daily-ring ratio of the accumulated times and the cyclic-ring ratio of the accumulated times;
determining the alarm probability of the preset monitoring buried point through the alarm probability algorithm according to the current data and the historical data of the preset index; and determining a preset monitoring buried point needing alarming according to the alarming probability and a preset alarming probability threshold value.
29. A buried point data processing system, comprising: the buried point data reporting device as claimed in claim 9, the buried point data processing device as claimed in claim 28.
30. A buried point data processing method is characterized by comprising the following steps:
receiving a data embedding processing request aiming at a page and sent by at least one client, wherein the data embedding processing request comprises a data embedding packet;
according to a preset time interval, counting buried point data in a plurality of buried point data packets of the page received in the time interval range to obtain current data of a preset index corresponding to a preset monitoring buried point;
the preset index includes any one of the following indexes: the ratio of the number of times of the father-son buried points, the ratio of the number of times of the root-son buried points, the daily-ring ratio of the number of times of accumulation and the circumferential-ring ratio of the number of times of accumulation;
and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
31. A buried point data processing apparatus, comprising:
the system comprises a request receiving unit, a processing unit and a processing unit, wherein the request receiving unit is used for receiving a buried point data processing request aiming at a page and sent by at least one client, and the buried point data processing request comprises a buried point data packet;
the data statistics unit is used for carrying out statistics on the buried point data in the multiple buried point data packets of the page received in the time interval range according to a preset time interval to obtain the current data of a preset index corresponding to a preset monitoring buried point;
the preset index includes any one of the following indexes: the ratio of the number of times of the father-son buried points, the ratio of the number of times of the root-son buried points, the daily-ring ratio of the number of times of accumulation and the circumferential-ring ratio of the number of times of accumulation;
and the alarm buried point determining unit is used for determining the preset monitoring buried point needing alarming according to the data of the preset index and the preset threshold corresponding to the preset index.
32. An electronic device, comprising:
a processor; and
a memory for storing a program for implementing the buried point data processing method, wherein the following steps are executed after the device is powered on and the program for implementing the buried point data processing method is executed by the processor: receiving a data embedding processing request aiming at a page and sent by at least one client, wherein the data embedding processing request comprises a data embedding packet; according to a preset time interval, counting buried point data in a plurality of buried point data packets of the page received in the time interval range to obtain current data of a preset index corresponding to a preset monitoring buried point; the preset index includes any one of the following indexes: the ratio of the number of times of the father-son buried points, the ratio of the number of times of the root-son buried points, the daily-ring ratio of the number of times of accumulation and the circumferential-ring ratio of the number of times of accumulation;
and determining a preset monitoring buried point needing to be alarmed according to the data of the preset index and a preset threshold corresponding to the preset index.
33. A buried point data processing system, comprising: the buried point data reporting device as claimed in claim 8, the buried point data processing device as claimed in claim 31.
CN201611102576.4A 2016-12-05 2016-12-05 Buried point data reporting method and device and electronic equipment Active CN108156006B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611102576.4A CN108156006B (en) 2016-12-05 2016-12-05 Buried point data reporting method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611102576.4A CN108156006B (en) 2016-12-05 2016-12-05 Buried point data reporting method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN108156006A CN108156006A (en) 2018-06-12
CN108156006B true CN108156006B (en) 2021-10-12

Family

ID=62470906

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611102576.4A Active CN108156006B (en) 2016-12-05 2016-12-05 Buried point data reporting method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN108156006B (en)

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921400A (en) * 2018-06-14 2018-11-30 万翼科技有限公司 Statistical method, server and the storage medium of house property information
CN108966041B (en) * 2018-07-18 2020-10-20 创新先进技术有限公司 Audio and video playing point burying method, device, equipment and computer readable storage medium
CN109767259A (en) * 2018-12-15 2019-05-17 深圳壹账通智能科技有限公司 Based on operation event promotion method, apparatus, equipment and the medium for burying point data
CN109474619B (en) * 2018-12-17 2023-06-23 中国平安财产保险股份有限公司 Data encryption reporting method and device and data decryption method and device
CN110135911A (en) * 2019-05-16 2019-08-16 重庆八戒传媒有限公司 Ad data statistical method, system, medium and electronic equipment
CN110347582B (en) * 2019-05-21 2024-05-28 平安银行股份有限公司 Buried point testing method and device
CN111984896A (en) * 2019-05-24 2020-11-24 上海哔哩哔哩科技有限公司 Buried point data acquisition method and device, computer equipment and readable storage medium
CN110245051B (en) * 2019-06-14 2023-07-04 上海中通吉网络技术有限公司 Data embedding method, device, equipment and storage medium
CN111767185A (en) * 2019-06-20 2020-10-13 北京京东尚科信息技术有限公司 Data point burying method and device
CN112433920A (en) * 2019-08-26 2021-03-02 北京京东尚科信息技术有限公司 Buried point data reporting method, device, equipment and medium
CN110825731B (en) * 2019-09-18 2023-10-24 平安科技（深圳）有限公司 Data storage method, device, electronic equipment and storage medium
CN110908879B (en) * 2019-10-16 2023-10-20 陕西合友网络科技有限公司 Reporting method, reporting device, reporting terminal and recording medium of buried point data
CN110716848A (en) * 2019-10-18 2020-01-21 广州华多网络科技有限公司 Data collection method and device, electronic equipment and storage medium
CN110941528B (en) * 2019-11-08 2022-04-08 支付宝(杭州)信息技术有限公司 Log buried point setting method, device and system based on fault
CN110929192A (en) * 2019-11-12 2020-03-27 北京城市网邻信息技术有限公司 Front-end request processing method and device
CN112905427A (en) * 2019-12-04 2021-06-04 北京沃东天骏信息技术有限公司 Data processing method and device
CN111866071A (en) * 2020-06-08 2020-10-30 北京百度网讯科技有限公司 Data uploading method and device of business application, electronic equipment and storage medium
CN111898036B (en) * 2020-06-24 2023-10-20 方欣科技有限公司 Behavior data collecting and processing system and method
CN111752803A (en) * 2020-06-28 2020-10-09 厦门美柚股份有限公司 Method, device and medium for collecting and reporting buried point data
CN112131180B (en) * 2020-09-25 2024-02-06 京东科技控股股份有限公司 Data reporting method, device and storage medium
CN112422445A (en) * 2020-10-10 2021-02-26 四川新网银行股份有限公司 Kafka-based real-time acquisition, calculation and storage method for buried point data
CN112416974A (en) * 2020-11-05 2021-02-26 上海幻电信息科技有限公司 Data processing method, device and equipment and readable storage medium
CN113138905A (en) * 2021-05-11 2021-07-20 北京京东拓先科技有限公司 Software function monitoring method and device
CN113158118A (en) * 2021-05-20 2021-07-23 中国工商银行股份有限公司 Page buried point data acquisition method, device and system
CN113590447B (en) * 2021-08-02 2024-04-19 北京百度网讯科技有限公司 Buried point processing method and device
CN114579416B (en) * 2022-03-09 2024-05-03 北京字节跳动网络技术有限公司 Index determination method, device, server and medium
CN115022388A (en) * 2022-05-31 2022-09-06 济南浪潮数据技术有限公司 Data acquisition method, device, equipment and medium
CN114692058B (en) * 2022-06-01 2022-08-02 天津市普迅电力信息技术有限公司 Automatic point burying method and system based on VUE (virtual environment) architecture and electronic equipment

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102054000B (en) * 2009-10-28 2012-07-25 中国移动通信集团公司 Data querying method, device and system
US8606907B1 (en) * 2011-07-20 2013-12-10 Google Inc. Multi-tiered system for receiving and reporting web site traffic data
CN104348650B (en) * 2013-08-05 2019-07-16 腾讯科技（深圳）有限公司 Monitoring method, service apparatus and the system of website
CN104572043B (en) * 2013-10-16 2018-01-19 阿里巴巴集团控股有限公司 A kind of control to client application carries out burying method and device a little in real time
CN106156212A (en) * 2015-04-22 2016-11-23 阿里巴巴集团控股有限公司 A kind of collecting method and system and terminal thereof and server
CN106066826A (en) * 2016-05-24 2016-11-02 北京金山安全软件有限公司 Automatic testing method and device for buried point and terminal

Also Published As

Publication number Publication date
CN108156006A (en) 2018-06-12

Similar Documents

Publication Publication Date Title
CN108156006B (en) Buried point data reporting method and device and electronic equipment
US20180365085A1 (en) Method and apparatus for monitoring client applications
CN110347716B (en) Log data processing method, device, terminal equipment and storage medium
CN106656536B (en) Method and equipment for processing service calling information
US20150170070A1 (en) Method, apparatus, and system for monitoring website
CN105376335B (en) Collected data uploading method and device
CN106815254B (en) Data processing method and device
CN109560989B (en) Link monitoring system
CN109039817B (en) Information processing method, device, equipment and medium for flow monitoring
CN111405030B (en) Message pushing method and device, electronic equipment and storage medium
CN105553770B (en) Data acquisition control method and device
CN111130944B (en) System monitoring method and system
CN113746703A (en) Abnormal link monitoring method, system and device
CN109409948B (en) Transaction abnormity detection method, device, equipment and computer readable storage medium
CN108334429A (en) Method, apparatus and system for investigating front end page problem
CN110119334B (en) Page script monitoring method and device
US10616306B2 (en) System and method for large-scale capture and tracking of web-based application parameters
CN108694107A (en) Backlog monitoring method, device, readable medium and the electronic equipment of message queue
CN110020166B (en) Data analysis method and related equipment
CN115277355A (en) Method, device, equipment and medium for processing state code data of monitoring system
EP3289446A1 (en) Classification of application events using call stacks
CN110968561B (en) Log storage method and distributed system
CN114531361A (en) Service topology analysis method and device of distributed system and storage medium
CN113254313A (en) Monitoring index abnormality detection method and device, electronic equipment and storage medium
CN112395155A (en) Service monitoring method and device, storage medium and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant