CN115563423A - Data acquisition method and device, computer equipment and storage medium - Google Patents
Data acquisition method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115563423A CN115563423A CN202211369752.6A CN202211369752A CN115563423A CN 115563423 A CN115563423 A CN 115563423A CN 202211369752 A CN202211369752 A CN 202211369752A CN 115563423 A CN115563423 A CN 115563423A
- Authority
- CN
- China
- Prior art keywords
- data
- buried point
- page
- preset
- point data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
Abstract
The invention discloses a data acquisition method, a data acquisition device, computer equipment and a storage medium, and relates to the computer application technology. The method comprises the following steps: monitoring an interactive behavior triggered by a user on a page element in a Web end; judging whether the page elements accord with a preset acquisition rule set or not; if the page elements accord with a preset collection rule set, executing an event monitoring function bound with the page elements, wherein the event monitoring function is used for collecting buried point data corresponding to the interactive behaviors; and caching the obtained buried point data, and uploading the buried point data according to a preset time interval. The method can collect the interaction behavior on the page elements, and determine whether to execute the event monitoring function bound with the page elements according to the condition that the page elements with the interaction behavior are consistent with the preset collection rule set. And then, the user behaviors of the page elements which accord with the preset acquisition rule set are acquired, and the accuracy of the data of the buried point is improved.
Description
Technical Field
The embodiment of the invention relates to a computer application technology, in particular to a data acquisition method, a data acquisition device, computer equipment and a storage medium.
Background
When the internet enterprise obtains the user behavior data, the data is obtained in a point burying mode.
The buried point deployment process comprises the following steps: and (4) raising a demand, combing out a buried point scheme according to the demand, programming each buried point data after confirming the buried point scheme, and after testing, putting the buried points on line.
However, the above method requires the configuration of each buried point data in advance. How to provide an accurate full-buried point scheme to achieve targeted acquisition of buried point data becomes an urgent problem to be solved.
Disclosure of Invention
The invention provides a data acquisition method, a data acquisition device, computer equipment and a storage medium, which are used for realizing accurate user behavior acquisition in a full-buried point and improving acquisition accuracy.
In a first aspect, an embodiment of the present invention provides a data acquisition method, including:
monitoring interaction behaviors triggered by a user on page elements in a Web end;
judging whether the page elements accord with a preset collection rule set or not;
if the page element accords with a preset collection rule set, executing an event monitoring function bound by the page element, wherein the event monitoring function is used for collecting buried point data corresponding to the interaction behavior;
caching the obtained buried point data, and uploading the buried point data according to a preset time interval.
In a second aspect, an embodiment of the present invention further provides a data acquisition apparatus, including:
the monitoring module is used for monitoring the interaction behavior triggered by the user in the Web end on the page element;
the acquisition rule judging module is used for judging whether the page elements accord with a preset acquisition rule set or not;
the event monitoring execution module is used for executing an event monitoring function bound with the page elements if the page elements accord with a preset acquisition rule set, and the event monitoring function is used for acquiring buried point data corresponding to the interaction behavior;
the cache module is used for caching the obtained buried point data;
and the reporting module is used for uploading the data of the buried point according to a preset time interval.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the data acquisition method according to the embodiment of the present invention.
In a fourth aspect, the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are used to perform the data acquisition method according to the embodiment of the present invention.
The data acquisition method provided by the embodiment of the invention monitors the interaction behavior triggered by the user in the Web end on the page element; judging whether the page elements accord with a preset collection rule set or not; if the page element accords with a preset collection rule set, executing an event monitoring function bound by the page element, wherein the event monitoring function is used for collecting buried point data corresponding to the interaction behavior; caching the obtained buried point data, and uploading the buried point data according to a preset time interval. Compared with the existing point burying mode which has redundancy and the existing full-buried points which cannot be accurately collected, the data collection method provided by the embodiment of the invention can collect the interaction behavior on the page elements, and determine whether to execute the event monitoring function bound by the page elements according to the coincidence condition of the page elements which interact with the interaction behavior and the preset collection rule set. Therefore, the user behaviors of the page elements which accord with the preset acquisition rule set are acquired, and the accuracy of the buried point data is improved. Meanwhile, the user behavior of page elements which do not belong to a preset collection rule set is avoided, the calculation amount and the data amount of full-buried point collection are reduced, and data redundancy is reduced.
Drawings
FIG. 1 is a schematic view of a buried point process in an embodiment of the present invention;
FIG. 2 is an event driven schematic in an embodiment of the invention;
FIG. 3 is a schematic diagram of an event trap bubble mechanism in an embodiment of the invention;
FIG. 4 is a first flowchart of a data collection method in an embodiment of the invention;
FIG. 5 is a second flowchart of a data collection method in an embodiment of the invention;
FIG. 6 is a schematic structural diagram of a data acquisition device in an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a computer device in the embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
A buried point, also called event tracing, is a way of data acquisition, and the buried point is used as a monitoring point set in advance in a web page. For a Web application, the most common form of embedding points is to write JavaScript codes to embed points at the front end, collect user traffic data and behavior data, and send the collected data to a corresponding data analysis display platform through a data cross-domain technology to complete further analysis.
The buried site deployment flow includes multiple phases, as shown in fig. 1. Firstly, a demand stage is provided, and in the stage, business personnel such as markets, operation managers and product managers define acquisition indexes based on own business scenes and initiate buried point demands. Then entering a buried point scheme design stage, and carrying out buried point scheme design by a data analyst on the basis of clear analysis requirements. And next, determining a duty stage, wherein a development team carries out corresponding system design and module development according to a buried point scheme, and a test team checks whether the event name, the attribute and the data are complete or not and whether the trigger time is correct or not. And finally, in a buried point online stage, a service party can monitor and analyze reported buried point data according to the data analysis platform. According to the technical scheme, the full-embedded point scheme can comprehensively collect the user behaviors of all interactive elements in a client page under the condition of not changing a project code, realizes automatic full collection, shortens a data collection period, reduces development cost and communication cost, enables developers and business parties to concentrate on user analysis rather than data acquisition, and is a data acquisition scheme which is mainly applied at present.
The scheme of full-buried point data acquisition adopts the idea of an event-driven programming model. The event-driven model is a responsive event model, as shown in fig. 2, and generally consists of three parts: an event source, an event distributor and an event handler. The event source and the event handler are not aware of each other's existence and are linked together through the event distributor. Event sources may be registered bound to the event dispatcher and delivered to the thread pool to execute corresponding event handlers. The method has the advantages that the event source code does not need to be modified one by one when the event handler is modified, and the event handler is not influenced by modifying the event source, so that the goals of high cohesion and low coupling are achieved.
There are generally three cases when the data of the Web end buried point is collected:
1. and when the user clicks the webpage element, acquiring user behavior information.
2. And when the page loading is finished, acquiring user access information.
3. And when the front end receives the data returned by the back end, acquiring the success or failure information of the key operation of the user.
In order to realize uniform point burying of page elements, the scheme of full-buried point data acquisition utilizes an event capturing and bubbling mechanism of a JavaScript language, as shown in fig. 3, the mechanism treats an event as propagation of a stream and is divided into three stages according to a time sequence: an event capture phase, a target phase, and an event bubble phase. When an event occurs, the event stream passes from the parent level to the child level one layer, and after an actual target element is reached, the event stream carries the attribute information of the element and bubbles upwards until the event stream returns to the starting point of the event stream. By using the mechanism of processing events by using the high-level element proxy and the low-level element, an event handler can be added in the bubble stage of a time stream, and when the triggering of the event is monitored, a callback function of the event handler is immediately executed to process the monitored event.
In order to monitor all user interaction events globally, reduce the code amount of the buried point codes and realize the full-amount acquisition of the buried point data, for example, the full-buried point technology adds a monitoring function on a document body object. Body object, also called content subject object, is one of indispensable objects in the web page structure, and this object contains almost all page visible elements and is a parent object of all visible elements. Due to the existence of a mechanism for processing events by high-level element agents and low-level elements, the monitoring function is arranged on the label, so that the proxy agent of the global event can be realized.
However, the total-buried-point technical solution adopts a total-volume collection method, so that the collected data volume is too large, the data redundancy is high, the pressure on the browser and the server is high under the condition of high concurrency, and the performance of the system is affected. How to provide an accurate full-buried point scheme to achieve targeted acquisition of buried point data becomes an urgent problem to be solved.
Fig. 4 is a first flowchart of a data acquisition method according to an embodiment of the present invention, where this embodiment is applicable to a situation where data acquisition of a specific page element is performed in a Web-end full-buried point scene, and the method may be executed by a server or a terminal and other computer devices, and specifically includes the following steps:
step 110, monitoring the interaction behavior triggered by the user on the page element in the Web end.
The Web end can be a webpage application of the PC end. A user interacts with page elements within a Web application. The interaction behavior can be user clicking on page elements, page browsing, mouse moving, keyboard loading and the like.
And step 120, judging whether the page elements accord with a preset acquisition rule set.
Whether the triggered element type is in an acquisition rule set defined by a user or not can be judged, and if the triggered element type is not the element concerned by the user, acquisition and reporting of the data of the buried point are abandoned.
Optionally, the determining whether the page element meets a preset collection rule set may be implemented by:
if the page element is not configured with a response mode, sending the interaction behavior to a father node or a node with higher hierarchy of the page element in the Web end;
until the information is sent to the node configured with a response mode, wherein the response mode comprises the following steps: and judging whether the page elements accord with a preset acquisition rule set or not.
The monitored events are processed, due to the existence of a JavaScript bubbling mechanism, a monitoring function does not need to be bound to each webpage label, and the triggering of the event function can be completed only by setting the monitoring function on a father node or a node with higher level of the webpage label, so that the configuration workload of the monitoring function is reduced, and the configuration efficiency is improved.
The method comprises the steps of adopting an event capturing and bubbling mechanism of JavaScript to realize global monitoring of an event, judging whether buried point data meets a predefined delivery rule before buried point triggering, and forbidding sending of the buried point data when the buried point data does not meet the delivery rule.
And step 130, if the page element accords with a preset collection rule set, executing an event monitoring function bound with the page element, wherein the event monitoring function is used for collecting buried point data corresponding to the interaction behavior.
The actual data types collected by the buried point data can be divided into two main types, one type is collection of flow data, and the flow data can be understood as data information generated when a user visits or leaves a webpage. Such as the visitor volume, the visit volume, the page loading time, the residence time of the user on the web page, the user environment information, and the like of the website, the information belongs to static information, and the trigger time of collection is usually after the user successfully visits the web page. The other type is directed at the collection of event data, the event data refers to behavior information generated when a user interacts with a webpage element control, the information can be basically detected by JavaScript and belongs to dynamic information, and the collection triggering time is usually when the event type bound to the element on the webpage document is triggered.
And if the page element accords with the preset collection rule set, executing an event monitoring function bound by the page element.
And step 140, caching the obtained buried point data, and uploading the buried point data according to a preset time interval.
Optionally, the obtained buried point data is cached, which may be implemented by the following manner:
determining picture data according to the buried point data, wherein the picture data is used for representing the buried point data;
and caching the embedded data in a mode of the picture data.
And packaging the obtained buried point data into a picture format. The buried point data can be spliced into a request after being converted into the format of the picture, and can be subsequently reported to a server side for storage. The size of the optional picture data may be 1 × 1 pixel.
The data are reported by using the mode of requesting 1 x 1 pixel git pictures, and the mode utilizes the characteristic that the browser can finish picture loading before closing the page, thereby improving the success rate of data reporting. In addition, when the client sends the GET request to complete data reporting in the mode, the server does not need to respond to the client after receiving the data, and overload pressure of the server during full collection is relieved. Compared with the prior art, the method for requesting the picture can also avoid the problem of interception by a browser caused by cross-domain, and the git format is used because the git format occupies a smaller volume than other picture formats such as jpeg, png and the like, so that network resources can be saved.
Optionally, the caching of the embedded data in the mode of the picture data may be implemented in the following modes:
caching the buried point data to a target cache region of a client in a mode of the picture data;
judging whether the target cache region reaches the maximum capacity;
and if the maximum capacity is reached, reporting the buried point data in real time.
The target cache region may be LocalStorage. Before the LocalStorage does not reach the maximum capacity, the data collected this time can be added to the LocalStorage for storage. Furthermore, in order to avoid data loss, the invention can automatically check whether data exist in the cache when entering a new page with the same domain name, and if so, the cached data can be continuously reported.
On the basis of the above embodiment, the uploading the buried point data according to the preset time interval may be implemented as:
determining uploading time according to a preset time interval;
and when the uploading time is reached, reporting the buried point data in the target cache region in batch.
When the buried point data is frequently sent, the buried point data is cached before transmission, and meanwhile, the fixed-time sending of a batch of data is realized, so that the request times are reduced, and the request pressure to a server is reduced. The specific implementation mode is that a request is written into a client cache Localstorage, and the client SDK triggers and sends data regularly within a set time interval of T seconds. The maximum capacity of Localstorage is 200 data, and when the maximum capacity is reached, the batch sending function is disabled, and a real-time sending mode is changed. In order to avoid data loss, when entering a new page with the same domain name, the client side SDK automatically checks whether data exists in the cache, and if so, the client side SDK continues to send the cached data. Compared with the traditional technology which adopts a reporting mode of receiving and sending, the mode can obviously reduce the request pressure to the server and improve the acquisition efficiency.
The data acquisition method provided by the embodiment of the invention monitors the interaction behavior triggered by the user in the Web end on the page element; judging whether the page elements accord with a preset acquisition rule set or not; if the page element accords with a preset collection rule set, executing an event monitoring function bound by the page element, wherein the event monitoring function is used for collecting buried point data corresponding to the interaction behavior; caching the obtained buried point data, and uploading the buried point data according to a preset time interval. Compared with the existing point burying mode which has redundancy and the existing full-buried points which cannot be accurately collected, the data collection method provided by the embodiment of the invention can collect the interaction behavior on the page elements, and determine whether to execute the event monitoring function bound by the page elements according to the coincidence condition of the page elements which interact with the interaction behavior and the preset collection rule set. And then, the user behaviors of the page elements which accord with the preset acquisition rule set are acquired, and the accuracy of the data of the buried point is improved. Meanwhile, the user behavior of collecting page elements which do not belong to a preset collection rule set is avoided, the calculated amount and the data amount of full-buried point collection are reduced, and data redundancy is reduced.
Fig. 5 is a second flowchart of a data acquisition method according to an embodiment of the present invention, which is used to further describe the foregoing embodiment, and includes:
step 210, monitoring the interaction behavior triggered by the user on the page element in the Web end.
And step 220, judging whether the page element accords with a preset acquisition rule set.
And step 230, if the page element accords with a preset collection rule set, rendering the page through a browser, and then constructing a DOM tree.
DOM, shorthand of the document object model. The process of rendering a page by a browser can be seen as a modeling process of organizing an HTML document into an object model from top to bottom. The HTML page can be constructed into a DOM tree after rendering is finished, elements, attributes, texts, annotations and the like in the original HTML are mapped into nodes in the DOM tree, the nodes can also keep the original hierarchical relationship in the DOM tree, and the organization structure facilitates the programming of an API (application programming interface) to directly manage and control various elements in the HTML in a browser memory.
And step 240, when the DOM number structure is changed, acquiring a plurality of changed element information.
And step 250, determining a first screen rendering score according to the first screen variation of the plurality of changed element information.
Optionally, the first screen rendering score is determined according to the first screen variation of the plurality of changed element information, and the first screen rendering score may be implemented in the following manner:
traversing the plurality of changed element information;
and if the label body of the changed object is an element, generating a first screen rendering score according to the variable quantity of the element node.
Step 260, when the first screen rendering score is increased for the first time, acquiring the first content drawing time from the rendering of the first page element to the screen;
step 270, determining first meaningful drawing time according to the maximum value of the first screen variation;
and step 280, determining whether the page is loaded completely according to the first content drawing time, the first meaningful drawing time and a preset loading interval.
And 290, caching the obtained buried point data, and uploading the buried point data according to a preset time interval.
As an example of the above embodiment, the real-time may be performed by:
step S1: after the Web page is rendered by the browser, the Web page can be constructed into a DOM tree, and the change of the DOM tree structure and the content is observed through a MutationObserver api (variable monitoring function interface).
Step S2: when the DOM tree structure changes, the MutationObserver interface does not immediately execute a callback function, but packs the related information of the variable elements into a set and returns the set.
And step S3: traversing the elements in the set returned in the step S42, when the element is an element in the < body > tag, it indicates that the element is a first screen element, at this time, the change of the DOM node may be converted into a score, and then the loading condition of the current page is determined according to the change condition of the score value, for example, the node change may be called a one-time effective change and is marked as 1 score; when the element is not an element in the < body > tag, the process returns to step S41 to continue listening.
And step S4: if the rendering score is increased for the First time, the time of rendering the First page element to the screen at this time is recorded as the First content rendering time (FCP).
Step S5: the rendering score continues to increase, the time point with the maximum score change in the interval with the maximum score change is recorded as the time when the page main body element is rendered on the screen, and the user can observe that the main body of the page appears at the time and record the time as the First Meaningful drawing time (FMP).
Step S6: and judging whether the DOM tree is built completely, if so, entering the next step, otherwise, returning to the step S41, and continuing to monitor.
Step S7: and comparing the execution intervals of the two threads, wherein the interval exceeds the set time, and indicating that all elements and data of the page are loaded completely.
Step S8: at this Time, the user can smoothly complete the interaction with the page, and the Time To Interactive (TTI) is recorded.
The data acquisition method provided by the embodiment of the invention can realize webpage index acquisition based on the DOM tree and improve the acquisition efficiency.
Fig. 6 is a schematic structural diagram of a data acquisition device according to an embodiment of the present invention, where the embodiment is applicable to a situation where data acquisition of a specific page element is performed in a Web-end full-buried-point scene, and the data acquisition device may be executed by an electronic device such as a server or a terminal, and specifically includes: the monitoring module 310, the collection rule determining module 320, the event monitoring executing module 330, the caching module 340, and the reporting module 350.
The monitoring module 310 is configured to monitor an interaction behavior triggered by a user on a page element in a Web;
an acquisition rule determining module 320, configured to determine whether the page element meets a preset acquisition rule set;
an event monitoring execution module 330, configured to execute an event monitoring function bound to the page element if the page element meets a preset collection rule set, where the event monitoring function is used to collect buried point data corresponding to the interaction behavior;
the caching module 340 is configured to cache the obtained buried point data;
and a reporting module 350, configured to upload the buried point data according to a preset time interval.
On the basis of the foregoing embodiment, the collection rule determining module 320 is configured to:
if the page element is not configured with a response mode, the interaction behavior is sent to a father node or a node with higher hierarchy of the page element in the Web end;
until the information is sent to the node configured with a response mode, wherein the response mode comprises the following steps: and judging whether the page elements accord with a preset acquisition rule set or not.
On the basis of the foregoing embodiment, the caching module 340 is configured to:
determining picture data according to the buried point data, wherein the picture data is used for representing the buried point data;
and caching the embedded data in a mode of the picture data.
On the basis of the foregoing embodiment, the caching module 340 is configured to:
caching the buried point data to a target cache region of a client in a mode of the picture data;
judging whether the target cache region reaches the maximum capacity or not;
and if the maximum capacity is reached, reporting the buried point data in real time.
On the basis of the foregoing embodiment, the reporting module 350 is configured to:
determining uploading time according to a preset time interval;
and when the uploading time is up, reporting the buried point data in the target cache region in batch.
On the basis of the foregoing embodiment, the event snoop execution module 330 is configured to:
after a page is rendered through a browser, a DOM tree is constructed;
when the DOM number structure is changed, acquiring a plurality of changed element information;
determining a first screen rendering score according to the first screen variation of the plurality of changed element information;
when the first screen rendering score is increased for the first time, acquiring the first content drawing time from the first page element to the screen;
determining the first meaningful drawing time according to the maximum value of the first screen variation;
and determining whether the page is completely loaded according to the first content drawing time, the first meaningful drawing time and a preset loading interval.
On the basis of the foregoing embodiment, the event snoop execution module 330 is configured to:
traversing the plurality of changed element information;
and if the label body of the changed object is an element, generating a first screen rendering score according to the variable quantity of the element node.
The data acquisition device provided by the embodiment of the invention comprises a monitoring module 310, a processing module and a display module, wherein the monitoring module 310 is used for monitoring the interaction behavior triggered by a user in a Web end on a page element; an acquisition rule determining module 320, configured to determine whether the page element meets a preset acquisition rule set; an event monitoring executing module 330, configured to execute an event monitoring function bound to the page element if the page element meets a preset collection rule set, where the event monitoring function is used to collect buried point data corresponding to the interaction behavior; the caching module 340 is configured to cache the obtained buried point data; and a reporting module 350, configured to upload the buried point data according to a preset time interval. Compared with the existing point burying mode which has redundancy and the existing full buried point can not be accurately collected, the data collection method provided by the embodiment of the invention can collect the interaction behavior on the page element, and determines whether to execute the event monitoring function bound by the page element according to the page element acted by the interaction behavior and the coincidence condition conforming to the preset collection rule set. Therefore, the user behaviors of the page elements which accord with the preset acquisition rule set are acquired, and the accuracy of the buried point data is improved. Meanwhile, the user behavior of page elements which do not belong to a preset collection rule set is avoided, the calculation amount and the data amount of full-buried point collection are reduced, and data redundancy is reduced.
The data acquisition device provided by the embodiment of the invention can execute the data acquisition method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Fig. 7 is a schematic structural diagram of a computer apparatus according to an embodiment of the present invention, as shown in fig. 7, the computer apparatus includes a processor 40, a memory 41, an input device 42, and an output device 43; the number of processors 40 in the computer device may be one or more, and one processor 40 is taken as an example in fig. 7; the processor 40, the memory 41, the input device 42 and the output device 43 in the computer apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 7.
The memory 41 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the data acquisition method in the embodiment of the present invention (for example, the monitoring module 310, the acquisition rule determining module 320, the event monitoring executing module 330, the caching module 340, and the reporting module 350 in the data acquisition apparatus). The processor 40 executes software programs, instructions and modules stored in the memory 41, so as to execute various functional applications of the computer device and data processing, namely, to realize the data acquisition method.
The memory 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 41 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 41 may further include memory located remotely from processor 40, which may be connected to a computer device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 42 is operable to receive input numeric or character information and to generate key signal inputs relating to user settings and function controls of the computer apparatus. The output device 43 may include a display device such as a display screen.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a data acquisition method, the method including:
monitoring an interactive behavior triggered by a user on a page element in a Web end;
judging whether the page elements accord with a preset acquisition rule set or not;
if the page element accords with a preset acquisition rule set, executing an event monitoring function bound by the page element, wherein the event monitoring function is used for acquiring buried point data corresponding to the interaction behavior;
caching the obtained buried point data, and uploading the buried point data according to a preset time interval.
On the basis of the foregoing embodiment, the determining whether the page element meets a preset collection rule set includes:
if the page element is not configured with a response mode, the interaction behavior is sent to a father node or a node with higher hierarchy of the page element in the Web end;
until the information is sent to the node configured with a response mode, wherein the response mode comprises the following steps: and judging whether the page elements accord with a preset acquisition rule set or not.
On the basis of the above embodiment, caching the obtained buried point data includes:
determining picture data according to the buried point data, wherein the picture data is used for representing the buried point data;
and caching the buried point data in a mode of the picture data.
On the basis of the above embodiment, the caching the embedded data in the form of the picture data includes:
caching the buried point data to a target cache region of a client in a mode of the picture data;
judging whether the target cache region reaches the maximum capacity or not;
and if the maximum capacity is reached, reporting the buried point data in real time.
On the basis of the foregoing embodiment, the uploading the buried point data according to a preset time interval includes:
determining uploading time according to a preset time interval;
and when the uploading time is reached, reporting the buried point data in the target cache region in batch.
On the basis of the above embodiment, the executing the event listening function of the page element binding includes:
after a page is rendered through a browser, a DOM tree is constructed;
when the DOM number structure changes, acquiring a plurality of changed element information;
determining a first screen rendering score according to the first screen variation of the plurality of changed element information;
when the first screen rendering score is increased for the first time, acquiring the first content drawing time from the first page element to the screen;
determining the first meaningful drawing time according to the maximum value of the first screen variation;
and determining whether the page is loaded completely according to the first content drawing time, the first meaningful drawing time and a preset loading interval.
On the basis of the foregoing embodiment, determining a first-screen rendering score according to a first-screen variation amount of a plurality of changed element information includes:
traversing the plurality of changed element information;
and if the label body of the changed object is an element, generating a first screen rendering score according to the variable quantity of the element node.
Of course, the storage medium provided by the embodiments of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also perform related operations in the data acquisition method provided by any embodiments of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the data acquisition device, each included unit and each included module are only divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method of data acquisition, comprising:
monitoring an interactive behavior triggered by a user on a page element in a Web end;
judging whether the page elements accord with a preset acquisition rule set or not;
if the page element accords with a preset acquisition rule set, executing an event monitoring function bound by the page element, wherein the event monitoring function is used for acquiring buried point data corresponding to the interaction behavior;
caching the obtained buried point data, and uploading the buried point data according to a preset time interval.
2. The method of claim 1, wherein the determining whether the page element complies with a preset collection rule set comprises:
if the page element is not configured with a response mode, the interaction behavior is sent to a father node or a node with higher hierarchy of the page element in the Web end;
until the information is sent to the node configured with a response mode, wherein the response mode comprises the following steps: and judging whether the page elements accord with a preset collection rule set or not.
3. The method of claim 1, wherein caching the obtained buried point data comprises:
determining picture data according to the buried point data, wherein the picture data is used for representing the buried point data;
and caching the embedded data in a mode of the picture data.
4. The method according to claim 3, wherein the caching the embedded data in a mode of the picture data comprises:
caching the buried point data to a target cache region of a client in a mode of the picture data;
judging whether the target cache region reaches the maximum capacity;
and if the maximum capacity is reached, reporting the buried point data in real time.
5. The method of claim 4, wherein uploading the buried point data according to the preset time interval comprises:
determining uploading time according to a preset time interval;
and when the uploading time is reached, reporting the buried point data in the target cache region in batch.
6. The method of claim 1, wherein the performing the event snoop function for the page element binding comprises:
after a page is rendered through a browser, a DOM tree is constructed;
when the DOM number structure is changed, acquiring a plurality of changed element information;
determining a first screen rendering score according to the first screen variation of the plurality of changed element information;
when the first screen rendering score is increased for the first time, acquiring the first content drawing time from the first page element to the screen;
determining the first meaningful drawing time according to the maximum value of the first screen variation;
and determining whether the page is loaded completely according to the first content drawing time, the first meaningful drawing time and a preset loading interval.
7. The method of claim 6, wherein determining the first screen rendering score according to the first screen variation amount of the plurality of changed element information comprises:
traversing the plurality of changed element information;
and if the label body of the changed object is an element, generating a first screen rendering score according to the variable quantity of the element node.
8. A data acquisition device, comprising:
the monitoring module is used for monitoring the interaction behavior triggered by the user on the page element in the Web end;
the acquisition rule judging module is used for judging whether the page elements accord with a preset acquisition rule set or not;
the event monitoring execution module is used for executing an event monitoring function bound by the page elements if the page elements accord with a preset collection rule set, and the event monitoring function is used for collecting buried point data corresponding to the interaction behavior;
the cache module is used for caching the obtained buried point data;
and the reporting module is used for uploading the data of the buried point according to a preset time interval.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the data acquisition method of any one of claims 1 to 7 when executing the program.
10. A storage medium containing computer executable instructions for performing the data acquisition method of any one of claims 1-7 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211369752.6A CN115563423A (en) | 2022-11-03 | 2022-11-03 | Data acquisition method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211369752.6A CN115563423A (en) | 2022-11-03 | 2022-11-03 | Data acquisition method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115563423A true CN115563423A (en) | 2023-01-03 |
Family
ID=84768536
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211369752.6A Pending CN115563423A (en) | 2022-11-03 | 2022-11-03 | Data acquisition method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115563423A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016629A (en) * | 2023-03-22 | 2023-04-25 | 创云融达信息技术(天津)股份有限公司 | Front-end page event extraction method |
-
2022
- 2022-11-03 CN CN202211369752.6A patent/CN115563423A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016629A (en) * | 2023-03-22 | 2023-04-25 | 创云融达信息技术(天津)股份有限公司 | Front-end page event extraction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107562620B (en) | Automatic buried point setting method and device | |
| CN108363602B (en) | Intelligent UI (user interface) layout method and device, terminal equipment and storage medium | |
| US8640037B2 (en) | Graphical overlay related to data mining and analytics | |
| CN107239389B (en) | Method and device for determining user operation record in hybrid APP | |
| US20200357007A1 (en) | Page data acquisition method, apparatus, server, electronic device and computer readable medium | |
| US20160170943A1 (en) | Dom snapshot capture | |
| CN103443781A (en) | Data delivery | |
| CN108334641B (en) | Method, system, electronic equipment and storage medium for collecting user behavior data | |
| CN109829121B (en) | Method and device for reporting click behavior data | |
| CN111901192A (en) | Statistical method and device for page access data | |
| CN111813629A (en) | Method, device and equipment for generating monitoring data of Web page | |
| CN113010395A (en) | Data acquisition method, apparatus, device, medium, and program product | |
| CN113254320A (en) | Method and device for recording user webpage operation behaviors | |
| CN113553269A (en) | Page buried point reporting method and related device | |
| CN115563423A (en) | Data acquisition method and device, computer equipment and storage medium | |
| CN109240664B (en) | Method and terminal for collecting user behavior information | |
| CN112817817B (en) | Buried point information query method, buried point information query device, computer equipment and storage medium | |
| CN113158118A (en) | Page buried point data acquisition method, device and system | |
| CN112861059A (en) | Visual component generation method and device, computer equipment and readable storage medium | |
| CN110188308B (en) | Client automatic dotting reporting method, storage medium, equipment and system | |
| CN110674426A (en) | Webpage behavior reporting method and device | |
| CN113515715B (en) | Buried point event code generation method, buried point event code processing method and related equipment | |
| CN110554892A (en) | Information acquisition method and device | |
| CN114579415A (en) | Method, device, equipment and medium for configuring and acquiring buried point data | |
| CN113961835A (en) | Data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |