CN112948542A - Buried point data acquisition method, system, equipment and storage medium - Google Patents
Buried point data acquisition method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN112948542A CN112948542A CN202110150606.3A CN202110150606A CN112948542A CN 112948542 A CN112948542 A CN 112948542A CN 202110150606 A CN202110150606 A CN 202110150606A CN 112948542 A CN112948542 A CN 112948542A
- Authority
- CN
- China
- Prior art keywords
- buried point
- point data
- data
- time
- buried
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
Abstract
The invention provides a buried point data acquisition method, a system, equipment and a storage medium, wherein the buried point data acquisition method comprises the following steps: acquiring first buried point data of a terminal at time t from a first data source; acquiring recorded second buried point data at the t-1 moment from a second data source; comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t; and adding third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t. The method realizes real-time recording of the acquired actual on-line buried point data by comparing the actual on-line buried point data with the recorded buried point data in real time, improves the acquisition efficiency of the buried point data, reduces the error of the buried point information by automatic acquisition, and improves the overall quality and integrity of the buried point data.
Description
Technical Field
The invention relates to the field of internet, in particular to a buried point data acquisition method, a buried point data acquisition system, buried point data acquisition equipment and a storage medium.
Background
At present, the analysis requirement of an internet company on personal behavior data reaches a state of being bland, a point burying technology can record user behavior data when a user uses a product, and undoubtedly, data point burying is a good privatization deployment data acquisition mode. A buried data system is a system where large data underlying platforms are very important. However, when the number of buried points used by a company for the collective management of all product buried points and the daily monitoring and analysis of buried point data exceeds 1 ten thousand, and the amount of buried point data generated per day reaches 30 hundred million, the buried points are effectively managed to avoid the situation that the buried point data cannot be effectively utilized as a pain point of a buried point data system.
The existing site management means of many companies are mainly divided into the following:
1. recording the embedded point information by simply using Excel or WIKI;
2. manually inputting buried point information into a buried point management system;
3. meanwhile, manual input and batch import into the buried point management system are supported.
The above method has disadvantages of low operation efficiency; the page responsible for each service product principal is more, or the omission of buried point entry caused by work handover does not enter buried point metadata; and manual entry of the buried point information has errors and the like.
How to design a method for automatically collecting buried points to realize the automatic collection of the buried points to a buried point data system, reduce buried point information errors and improve buried point input efficiency becomes a technical problem to be solved urgently.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.
Disclosure of Invention
The invention aims to provide a buried point data acquisition method, a buried point data acquisition system, buried point data acquisition equipment and a storage medium, wherein the buried point data acquisition method improves the acquisition efficiency of the buried point data and reduces the error rate of the buried point information, so that the overall quality and the integrity of the buried point data are improved.
Some embodiments of the present invention provide a buried point data acquisition method, including the steps of:
acquiring first buried point data of the terminal at the time t from a first data source, wherein the first buried point data is data generated by the terminal responding to at least one operation;
acquiring recorded second buried point data at the t-1 moment from a second data source;
comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t;
and adding the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t.
According to some examples of this invention, the first buried point data comprises at least one buried point page information;
the third buried point data comprises at least one buried point page information;
the second buried point data comprises a buried point page table;
the adding the third buried point data to the second buried point data at the time t-1 to obtain the second buried point data at the time t comprises:
and adding at least one piece of embedded point page information included in the third embedded point data to the embedded point page table at the time t-1 to obtain an embedded point page table at the time t.
According to some examples of the present invention, after the step of obtaining the third buried point data, which is not present in the second buried point data at time t-1, in the first buried point data at time t, the method further includes the steps of:
and creating at least one corresponding embedded point element table according to at least one embedded point page information of the third embedded point data, wherein each embedded point element table comprises an element information list of an embedded point page.
According to some examples of this invention, the first buried point data further includes buried point element information of at least one buried point page;
the third buried point data also comprises at least one buried point element information;
the second buried point data also comprises at least one buried point element table;
the adding the third buried point data to the second buried point data at the time t-1 to obtain the second buried point data at the time t comprises:
and adding at least one piece of embedded point element information included in the third embedded point data to the embedded point element table at the time t-1 to obtain the embedded point element table at the time t.
According to some examples of the present invention, after the step of obtaining the third buried point data, which is not present in the second buried point data at time t-1, in the first buried point data at time t, the method further includes the steps of:
and creating at least one corresponding embedded point parameter according to at least one embedded point element information of the third embedded point data, wherein each embedded point parameter table comprises a parameter information list of an embedded point page.
According to some examples of the invention, the first buried point data further comprises buried point parameter information of at least one buried point element;
the third buried point data also comprises at least one buried point parameter information;
the second buried point data also comprises at least one buried point parameter table;
the adding the third buried point data to the second buried point data at the time t-1 to obtain the second buried point data at the time t comprises:
and adding at least one piece of embedded point parameter information included in the third embedded point data to the embedded point element table at the time t-1 to obtain an embedded point parameter table at the time t.
According to some examples of the invention, the first data source is a buried point log table.
According to some examples of the invention, the first data source is stored in a HIVE data warehouse
According to some examples of the invention, the second data source is stored in an ODS data store layer of the HIVE data warehouse.
According to some examples of the invention, the second data source is synchronized from the MySQL service library to the ODS data store layer.
According to some examples of the invention, after the step of adding the third buried point data to the second buried point data at time t-1 to form the second buried point data at time t, the method further includes the steps of:
and synchronizing the second buried point data to a MySQL service library.
Some embodiments of the present invention further provide a buried point data acquisition system, which is used for implementing the buried point data acquisition method, and includes an acquisition module, a comparison module, and an execution module, where:
the acquisition module is used for acquiring first buried point data of the terminal at the time t from a first data source, wherein the first buried point data is data generated by the terminal in response to at least one operation; and
the second embedded point data is used for acquiring the t-1 moment recorded in the system from a second data source;
the comparison module is used for comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t;
the execution module is used for adding the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t.
An embodiment of the present invention further provides an electronic device, including:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the buried point data collection method via execution of the executable instructions.
An embodiment of the present invention further provides a computer-readable storage medium for storing a program, where the program is executed to implement the steps of the buried data collecting method.
The method realizes real-time recording of the acquired actual on-line buried point data by comparing the actual on-line buried point data with the recorded buried point data in real time, improves the acquisition efficiency of the buried point data, reduces the error of the buried point information by automatic acquisition, and improves the overall quality and integrity of the buried point data.
Drawings
Other features, objects, and advantages of the invention will be apparent from the following detailed description of non-limiting embodiments, which proceeds with reference to the accompanying drawings and which is incorporated in and constitutes a part of this specification, illustrating embodiments consistent with the present application and together with the description serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a flow chart of a buried point data collection method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a buried point data collection method according to an embodiment of the present invention;
FIG. 3 is an exemplary diagram of a landed page table in accordance with one embodiment of the present invention;
FIG. 4 is an exemplary diagram of a buried point element table in accordance with one embodiment of the present invention;
FIG. 5 is a diagram illustrating an exemplary buried point parameter table according to an embodiment of the present invention;
FIG. 6 is a block diagram of a buried point data acquisition system according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a buried point data acquisition device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
Fig. 1 is a flowchart of a buried point data acquisition method according to an embodiment of the present invention, and specifically, the buried point data acquisition method includes the following steps:
s100: acquiring first buried point data of the terminal at the time t from a first data source, wherein the first buried point data is data generated by the terminal responding to at least one operation; the terminal here can be various terminals, such as web pages, mobile terminals, applets, etc.;
s200: acquiring recorded second buried point data at the t-1 moment from a second data source;
s300: comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t;
s400: and adding the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t, wherein the second buried point data is recorded data which can be used for subsequent query and analysis.
The step S100 of the above embodiment may obtain the first buried point data of the terminal at the time t from the first data source, and the step S200 may obtain the second buried point data at the time t-1 recorded from the second data source, which may be real-time periodic, and accordingly, the step S300 may compare the first buried point data at the time t with the second buried point data at the time t-1 to obtain the third buried point data that is not present in the second buried point data at the time t-1 in the first buried point data at the time t; and the step S400 of adding the third embedded point data to the second embedded point data at the time t-1 to form the second embedded point data at the time t is also real-time and periodic, namely in the method of the invention, the system compares the embedded point data at the product end which is actually operated with the embedded point data recorded in the system in real time, and the embedded point data which is not recorded in the system is recorded in the system, so that the timeliness of acquiring the embedded point data is realized, and meanwhile, the steps are automatically executed by the system, so that the acquisition efficiency of the embedded point data is improved, the error rate of the embedded point data is reduced, and the integral quality and the integrity of the embedded point data are improved.
The buried point data collection method of the present invention will be described in detail with reference to specific examples.
In an embodiment, the first data source may be a buried point log table, and when a user uses a product after completing the burying of the product, related user behavior data or buried point access data may be recorded in the product to form a corresponding buried point log table, where the buried point log table records related data of all latest buried points of the product, where it is noted that the buried point log table refers to a plurality of products to be monitored, and even a buried point log table of products used by a company.
The first data source can be stored in an HIVE data warehouse, the HIVE data warehouse is a data warehouse tool based on Hadoop, can be used for data extraction, conversion and loading, the HIVE is a mechanism capable of storing, inquiring and analyzing large-scale data stored in the Hadoop, is suitable for big data processing, and has the advantages of low learning cost, support for user-defined functions, capability of realizing own functions according to own requirements and the like.
However, the HIVE data warehouse has a slow query speed due to the storage of mass data, and cannot perform millisecond-level query of a Relational Database (RDBMS). In contrast, the MySQL service library is a relational database management system, and the relational database stores data in different tables, rather than putting all data in one large library, thereby increasing query speed and flexibility. Meanwhile, the MySQL service library can be used as a single application program in a client server network environment, and can also be used as a library to be embedded in other software, so that the MySQL service library is widely applied. In actual use, the data in the result table in the HIVE data warehouse can be written into the MySQL service library so as to query the data in the result table from the MySQL service library. Meanwhile, the HIVE data warehouse is arranged on a Hadoop, and the MySQL service library stores data in the equipment or the local system.
In the embodiment, for the case that the second buried point source for subsequent query and analysis is stored in the MySQL service library, the buried point data collection method of the present invention may be specifically as shown in fig. 2, where the step S300 compares the first buried point data at the time t with the second buried point data at the time t-1, and then may be implemented by synchronizing the second buried point source in the MySQL service library to the ODS data storage layer of the HIVE data warehouse in real time, that is, when comparing the actual buried point data and the recorded second buried point data on the real-time line, the actual operation is:
the HIVE data warehouse obtains first buried point data in real time from the buried point log table, and the first buried point data at the moment is defined as the first buried point data at the moment t;
through real-time synchronization of the MySQL service library and the ODS data storage layer, the ODS data storage layer acquires second buried point data at the t-1 moment in real time;
comparing the first buried point data with the second buried point data, and when the data which does not exist in the first buried point data and the second buried point data exists in the first buried point data, considering the data as actually generated and unrecorded buried point data, and defining the data as third buried point data;
then, step S400 is executed: and adding the third embedding point data to the second embedding point data at the time t-1 to obtain the second embedding point data at the time t, namely finishing the updating of the second embedding point data.
Certainly, after the second buried point data is updated, the second buried point data at the time t is synchronized to the MySQL service library from the HIVE data warehouse in real time and serves as an object for comparing the actual buried point data on the real-time line at the time t + 1.
The first buried point data obtained from the first data source in step S100 may include all information of buried points, such as information of buried point pages, information of buried point elements of the buried point pages, information of buried point parameters of the buried point elements, and the like.
The embedded point page information includes a page name, a domain name, an IP, a page description, and the like, when the first embedded point data includes at least one embedded point page information, the embedded point page information included in the first embedded point data at the time t is compared with the embedded point page information included in the second embedded point data at the time t-1 in the step S300, at this time, a plurality of embedded point page information which does not exist in the second embedded point data at the time t-1 in the first embedded point data at the time t is obtained, and correspondingly, the third embedded point data includes the plurality of embedded point page information.
The second buried point data at this time includes a buried point page table, and the step of executing S400 includes adding a plurality of pieces of buried point page information included in the third buried point data to the buried point page table at time t-1 to form the buried point page table at time t, and an exemplary diagram of the buried point page table of an embodiment is shown in fig. 3.
In an embodiment, in step S300, at least one piece of embedded point page information in the first embedded point data at time t and not in the second embedded point data at time t-1 is obtained, that is, after it is found that an embedded point page exists and is not recorded, the newly found embedded point page is recorded, and in order to subsequently update data of each element included in the embedded point page, a corresponding embedded point element table of the newly found embedded point page needs to be created, where each embedded point element table includes an element information list of an embedded point page.
The embedded point element information comprises an element event type, embedded point page information id, an element name and the like, and when the first embedded point data further comprises embedded point element information of at least one embedded point page; in step S300, the buried point element information included in the first buried point data at time t is compared with the buried point element information included in the second buried point data at time t-1, at this time, a plurality of pieces of buried point element information that are not present in the second buried point data at time t-1 in the first buried point data at time t are obtained, and correspondingly, the third buried point data further includes the plurality of pieces of buried point element information.
The second buried point data at this time includes a buried point element table, and the step S400 is executed to add a plurality of pieces of buried point element information included in the third buried point data to the buried point element table at time t-1 to form the buried point element table at time t, and an exemplary diagram of the buried point element table in an embodiment is shown in fig. 4.
In an embodiment, in step S300, at least one piece of embedded point element information in the second embedded point data that is not present at time t-1 in the first embedded point data at time t is obtained, that is, after it is found that the embedded point element is not recorded, the newly-found embedded point element is recorded, and in order to subsequently update data of each parameter included in the embedded point element, a embedded point parameter table corresponding to the newly-found embedded point element needs to be created, where each embedded point parameter table includes an embedded point parameter information list of an embedded point page.
The embedded point element information comprises a parameter name, a parameter type, related information of creation and modification and the like, and when the first embedded point data further comprises embedded point parameter information of at least one embedded point page; in step S300, the buried point parameter information included in the first buried point data at time t is compared with the buried point parameter information included in the second buried point data at time t-1, at this time, a plurality of pieces of buried point parameter information that are not present in the second buried point data at time t-1 in the first buried point data at time t are obtained, and correspondingly, the third buried point data further includes the plurality of pieces of buried point parameter information.
The second buried point data includes a buried point parameter table, and the step S400 is executed to add a plurality of buried point parameter information included in the third buried point data to the buried point element table at time t-1 to form the buried point parameter table at time t, where an exemplary diagram of the buried point parameter table in an embodiment is shown in fig. 5.
In the embodiment, the HIVE data warehouse is adopted to collect actually generated buried point data in real time, the actually generated buried point data is compared with the recorded buried point data, the recorded buried point data is updated, meanwhile, the updated buried point data comprises buried point page information, buried point element information of a buried point page, buried point parameter information of a buried point element and the like, and the integrity and timeliness of the buried point data are kept. In addition, the updated buried point data is stored in the MySQL service library, so that the speed of subsequently inquiring the buried point data is improved, and the user behavior analysis based on the buried point data is better served.
Some embodiments of the present invention further provide a buried point data acquisition system, which is used for implementing the buried point data acquisition method.
Fig. 6 is a schematic block diagram of a buried point data acquisition system according to an embodiment of the present invention, where the system includes an acquisition module M100, a comparison module M200, and an execution module M300, where:
the acquisition module M100 is configured to acquire first buried point data of the terminal at time t from a first data source, where the first buried point data is data generated by the terminal in response to at least one operation; and
the second embedded point data is used for acquiring the t-1 moment recorded in the system from a second data source;
the comparison module M200 is configured to compare the first buried point data at the time t with the second buried point data at the time t-1, and obtain third buried point data that does not exist in the second buried point data at the time t-1 in the first buried point data at the time t;
the execution module M300 is configured to add the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t.
The function implementation manner of each functional module in the buried point data acquisition system of the embodiment can be implemented by adopting the specific implementation manner of each step in the buried point data acquisition method. For example, the acquisition module M100, the comparison module M200, and the execution module M300 may respectively adopt the specific implementation manners of the steps S100 to S400 to implement the functions thereof, which are not described herein again. The embedded point data acquisition system realizes real-time recording of the acquired online actual embedded point data, and simultaneously improves the acquisition efficiency of the embedded point data and the accuracy of the embedded point data, thereby obtaining the embedded point data with high overall quality and good integrity.
An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 600 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code which can be executed by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention as described in the above-mentioned method section of the present specification. For example, processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
The embodiment of the invention also provides a computer readable storage medium for storing a program, and the program is executed to realize the steps of the buried point data acquisition method. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention described in the method part above of this description when said program product is run on the terminal device.
Referring to fig. 8, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In summary, the present invention provides a buried point data collecting method, which includes the following steps: acquiring first buried point data of the terminal at the time t from a first data source, wherein the first buried point data is data generated by the terminal responding to at least one operation; acquiring recorded second buried point data at the t-1 moment from a second data source; comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t; and adding the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t. The method realizes real-time recording of the acquired actual on-line buried point data by comparing the actual on-line buried point data with the recorded buried point data in real time, improves the acquisition efficiency of the buried point data, reduces the error of the buried point information by automatic acquisition, and improves the overall quality and integrity of the buried point data.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (14)
1. A buried point data acquisition method is characterized by comprising the following steps:
acquiring first buried point data of the terminal at the time t from a first data source, wherein the first buried point data is data generated by the terminal responding to at least one operation;
acquiring recorded second buried point data at the t-1 moment from a second data source;
comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t;
and adding the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t.
2. The buried point data collection method according to claim 1, characterized in that:
the first buried point data comprises at least one buried point page information;
the third buried point data comprises at least one buried point page information;
the second buried point data comprises a buried point page table;
the adding the third buried point data to the second buried point data at the time t-1 to obtain the second buried point data at the time t comprises:
and adding at least one piece of embedded point page information included in the third embedded point data to the embedded point page table at the time t-1 to obtain an embedded point page table at the time t.
3. The buried point data collection method according to claim 2, wherein after the step of obtaining third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t, the method further comprises the steps of:
and creating at least one corresponding embedded point element table according to at least one embedded point page information of the third embedded point data, wherein each embedded point element table comprises an element information list of an embedded point page.
4. The buried point data collection method according to claim 3, characterized in that:
the first buried point data also comprises buried point element information of at least one buried point page;
the third buried point data also comprises at least one buried point element information;
the second buried point data also comprises at least one buried point element table;
the adding the third buried point data to the second buried point data at the time t-1 to obtain the second buried point data at the time t comprises:
and adding at least one piece of embedded point element information included in the third embedded point data to the embedded point element table at the time t-1 to obtain the embedded point element table at the time t.
5. The buried point data collection method according to claim 4, wherein after the step of obtaining third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t, the method further comprises the steps of:
and creating at least one corresponding embedded point parameter according to at least one embedded point element information of the third embedded point data, wherein each embedded point parameter table comprises a parameter information list of an embedded point page.
6. The buried point data collection method according to claim 5, characterized in that:
the first buried point data also comprises buried point parameter information of at least one buried point element;
the third buried point data also comprises at least one buried point parameter information;
the second buried point data also comprises at least one buried point parameter table;
the adding the third buried point data to the second buried point data at the time t-1 to obtain the second buried point data at the time t comprises:
and adding at least one piece of embedded point parameter information included in the third embedded point data to the embedded point element table at the time t-1 to obtain an embedded point parameter table at the time t.
7. The buried point data collection method of claim 1, wherein the first data source is a buried point log table.
8. The buried data collection method of claim 7, wherein the first data source is stored in a HIVE data warehouse.
9. The buried data collection method of claim 8, wherein the second data source is stored in an ODS data storage layer of the HIVE data warehouse.
10. The buried point data collection method of claim 9, wherein the second data source is synchronized from a MySQL service library to the ODS data store layer.
11. The buried point data collection method according to claim 10, wherein after the step of adding the third buried point data to the second buried point data at time t-1 to form the second buried point data at time t, the method further comprises the steps of:
and synchronizing the second buried point data to a MySQL service library.
12. A buried point data acquisition system for implementing the buried point data acquisition method of any one of claims 1 to 11, comprising an acquisition module, a comparison module and an execution module, wherein:
the acquisition module is used for acquiring first buried point data of the terminal at the time t from a first data source, wherein the first buried point data is data generated by the terminal in response to at least one operation; and
the second embedded point data is used for acquiring the t-1 moment recorded in the system from a second data source;
the comparison module is used for comparing the first buried point data at the time t with the second buried point data at the time t-1 to obtain third buried point data which does not exist in the second buried point data at the time t-1 in the first buried point data at the time t;
the execution module is used for adding the third buried point data to the second buried point data at the time t-1 to form the second buried point data at the time t.
13. An electronic device, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the buried point data acquisition method of any one of claims 1 to 11 via execution of the executable instructions.
14. A computer-readable storage medium storing a program which, when executed by a processor, implements the steps of the buried data collection method of any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110150606.3A CN112948542A (en) | 2021-02-03 | 2021-02-03 | Buried point data acquisition method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110150606.3A CN112948542A (en) | 2021-02-03 | 2021-02-03 | Buried point data acquisition method, system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112948542A true CN112948542A (en) | 2021-06-11 |
Family
ID=76243366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110150606.3A Withdrawn CN112948542A (en) | 2021-02-03 | 2021-02-03 | Buried point data acquisition method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112948542A (en) |
-
2021
- 2021-02-03 CN CN202110150606.3A patent/CN112948542A/en not_active Withdrawn
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11487539B2 (en) | Systems and methods for automating and monitoring software development operations | |
| CN111709527A (en) | Operation and maintenance knowledge map library establishing method, device, equipment and storage medium | |
| CN110807067A (en) | Data synchronization method, device and equipment for relational database and data warehouse | |
| CN110705235A (en) | Information input method and device for business handling, storage medium and electronic equipment | |
| CN111506603B (en) | Data processing method, device, equipment and storage medium | |
| CN109473178B (en) | Method, system, device and storage medium for medical data integration | |
| CN109460363B (en) | Automatic testing method and device, electronic equipment and computer readable medium | |
| CN109582906B (en) | Method, device, equipment and storage medium for determining data reliability | |
| CN109493928B (en) | Patient data screening method, system, equipment and storage medium based on condition tree | |
| CN110716804A (en) | Method and device for automatically deleting useless resources, storage medium and electronic equipment | |
| US10129328B2 (en) | Centralized management of webservice resources in an enterprise | |
| CN112507075A (en) | Case data searching method, system, equipment and storage medium | |
| CN117076457A (en) | Data processing method, device, equipment and storage medium | |
| CN111796833A (en) | Code language conversion method, system, equipment and storage medium | |
| CN112948542A (en) | Buried point data acquisition method, system, equipment and storage medium | |
| CN103809915A (en) | Read-write method and device of magnetic disk files | |
| CN116244387A (en) | Entity relationship construction method, device, electronic equipment and storage medium | |
| CN113127496B (en) | Method and device for determining change data in database, medium and equipment | |
| CN110543520B (en) | Data migration method and device | |
| CN113722296A (en) | Agricultural information processing method and device, electronic equipment and storage medium | |
| CN112925840A (en) | Data export method, system, device and storage medium | |
| CN110727457A (en) | Component management method, device, storage medium and electronic equipment | |
| CN110928898A (en) | Data acquisition method, data acquisition device, storage medium and electronic equipment | |
| CN112597220B (en) | Data file reading method, device, electronic equipment and medium | |
| US20120233224A1 (en) | Data processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210611 |
|
| WW01 | Invention patent application withdrawn after publication |