CN113190411A - Data processing method and device, electronic equipment and storage medium - Google Patents
Data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113190411A CN113190411A CN202110519755.2A CN202110519755A CN113190411A CN 113190411 A CN113190411 A CN 113190411A CN 202110519755 A CN202110519755 A CN 202110519755A CN 113190411 A CN113190411 A CN 113190411A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- rule
- statistics
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computer Hardware Design (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention relates to a data processing method, a data processing device, electronic equipment and a storage medium, wherein the data processing method comprises the following steps: determining at least one target client, and sending data statistical rules to the at least one target client; receiving data statistical information sent by at least one target client, wherein the data statistical information is obtained by counting buried point data acquired by the target client according to a data statistical rule by the target client; and performing service processing based on the data statistical information. Therefore, the calculation cost of the server side can be reduced, the data transmission quantity between the client side and the server side can be reduced, and the network resources can be saved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.
Background
At present, a script is embedded in a client page to detect user operation, when the client detects the user operation, the client acquires Pingback data, and then the client reports the acquired Pingback data to a server, and the server aggregates required report data according to the finest granularity Pingback data. Here, the Pingback data may be, for example, information on the user's dwell time on the page, mouse click area, or the like.
However, with the rapid development of the internet, the internet user scale is increasingly large, and the user operation is also increasingly frequent, so that the client side reports a large amount of Pingback data generated to the server every day, which not only greatly occupies the storage resource of the server side, but also causes the server to pay a huge calculation cost when performing data statistics.
Disclosure of Invention
In view of this, in order to solve the technical problem that the server side needs to pay a huge calculation cost due to the fact that data statistics is performed on the server side from the finest granularity data, embodiments of the present invention provide a data processing method, an apparatus, an electronic device, and a storage medium.
In a first aspect, an embodiment of the present invention provides a data processing method applied to a server, where the method includes:
determining at least one target client, and sending a data statistical rule to at least one target client;
receiving data statistical information sent by at least one target client, wherein the data statistical information is obtained by the target client through statistics on buried point data acquired by the target client according to the data statistical rule;
and performing service processing based on the data statistical information.
In a possible embodiment, the determining at least one target client includes:
analyzing a user identifier from a data statistical rule to be sent;
and determining the client corresponding to the user identification as a target client.
In a possible embodiment, the determining at least one target client includes:
analyzing user characteristics from a data statistical rule to be sent;
determining a target user matched with the user characteristics;
and determining the client corresponding to the target user as a target client.
In a possible implementation, after the receiving data statistics information sent by at least one of the target clients, the method further includes:
determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the data statistical information;
and writing the data statistical information into the target storage medium.
In a possible implementation manner, before the determining, according to the data statistics rule corresponding to the data statistics information, a target storage medium for storing the data statistics information, the method further includes:
storing the data statistical information into a preset message middleware;
determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the data statistical information; writing the data statistics to the target storage medium, including:
reading the data statistical information from the preset message middleware according to a preset data reading strategy;
when the data statistical information is read, determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the currently read data statistical information; and writing the currently read data statistical information into the target storage medium.
In a second aspect, an embodiment of the present invention provides a data processing method, which is applied to a client, where the method includes:
receiving a data statistical rule sent by a server;
when a buried point triggering event is detected, collecting corresponding buried point data;
when data statistics is determined, performing data statistics on the buried point data collected in a set historical time period according to the data statistics rule to obtain data statistics information;
and sending the data statistical information to a server so that the server performs service processing based on the data statistical information.
In a possible implementation manner, the performing data statistics on the buried point data collected in a set historical time period according to the data statistics rule includes:
searching target buried point data matched with the data statistical rule from the buried point data collected in a set historical time period;
and performing data statistics on the target buried point data according to the data statistics rule.
In a possible embodiment, the data statistics rule includes an aggregation dimension, and the method further includes:
if the target buried point data does not contain dimension information, sending a data query request to the server, wherein the data query request is used for indicating to obtain the dimension information corresponding to the target buried point data;
receiving target data returned by the server, wherein the target data comprises dimension information corresponding to the target buried point data;
the data statistics of the target buried point data according to the data statistics rule comprises the following steps:
and performing data statistics on the target buried point data based on the target data according to the data statistics rule.
In a third aspect, an embodiment of the present invention provides a data processing apparatus, which is applied to a server, and the apparatus includes:
the distribution module is used for determining at least one target client and sending a data statistical rule to the at least one target client;
the receiving module is used for receiving data statistical information sent by at least one target client, wherein the data statistical information is obtained by counting buried point data acquired by the target client according to the data statistical rule of the target client;
and the processing module is used for processing the service based on the data statistical information.
In a fourth aspect, an embodiment of the present invention provides a data processing apparatus, which is applied to a client, where the apparatus includes:
the rule receiving module is used for receiving the data statistical rule sent by the server;
the data acquisition module is used for acquiring corresponding buried point data when a buried point triggering event is detected;
the data statistics module is used for carrying out data statistics on the buried point data collected in a set historical time period according to the data statistics rule when the data statistics is determined to be carried out, so that data statistics information is obtained;
and the data reporting module is used for sending the data statistical information to a server so that the server performs service processing based on the data statistical information.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including: a processor and a memory, the processor being configured to execute a data processing program stored in the memory to implement the data processing method of any one of the first aspect or the second aspect.
In a sixth aspect, an embodiment of the present invention provides a storage medium, where the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the data processing method according to any one of the first aspect or the second aspect.
According to the technical scheme provided by the embodiment of the invention, partial data statistics work is transferred from the server side to the client side, and the client side sends the data statistics information to the server side, so that the server side does not need to start calculation from bottom data any more, but calculates on the basis of the data statistics information sent by the client side, and therefore, the calculation cost of the server side is reduced.
Drawings
FIG. 1 is a diagram of a system architecture according to an embodiment of the present invention;
fig. 2 is a flowchart of an embodiment of a data processing method according to the present invention;
FIG. 3 is a flowchart illustrating another data processing method according to an embodiment of the present invention;
FIG. 4 is a block diagram of an embodiment of a data processing apparatus according to the present invention;
FIG. 5 is a block diagram of an embodiment of another data processing apparatus according to the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For the understanding of the embodiments of the present invention, the following description first illustrates a system architecture according to the embodiments of the present invention with reference to the accompanying drawings:
fig. 1 is a schematic diagram of a system architecture according to an embodiment of the present invention. The system architecture shown in fig. 1 includes: server 11, client 12 ~ 13. The server 11 and the clients 12 and 13 both have a communication link, which may include a wired link, a wireless link, and the like, and the embodiment of the present invention is not limited thereto.
The clients 12-13 refer to programs installed on the terminal equipment and corresponding to the server 11 for providing local services for the users. Here, the local service may include, but is not limited to: human-computer interaction service, local data acquisition and maintenance service, communication service between local and server, etc. The terminal device may include a mobile phone, a tablet computer, a notebook computer, a palm computer, a mobile internet device, a wearable device (e.g., a smart band, a smart watch, etc.), and the like.
It will be appreciated that the number of devices in figure 1 is merely illustrative. According to actual needs, the system architecture shown in fig. 1 may include any number of servers and clients, and the embodiment of the present invention is not limited thereto.
The data processing method provided by the present invention is further explained in terms of a client and a server respectively by the following specific embodiments with reference to the drawings, and the embodiments do not limit the embodiments of the present invention.
Referring to fig. 2, a flowchart of an embodiment of a data processing method according to an embodiment of the present invention is provided. As an embodiment, the method may be applied to a client, such as the clients 12-13 illustrated in FIG. 1, and the process may include the following steps:
In an embodiment of the present invention, the data statistics rule may include, but is not limited to, the following: a user group [ GA, GB, GC. ] a statistical period [ Data1, Data2 ], a convergence dimension [ WA, WB, WC. ] an index [ MA, MB, MC,. ] a calculation rule [ sum, count, max, min. ] a time interval T. The specific significance of the data statistical rule is as follows: in a statistical period [ Data1, Data2 ], Data statistics is performed on indexes MA, MB, MC and the like of users GA, GB, GC and the like in T time from a plurality of aggregation dimensions such as WA, WB, WC and the like at intervals of time T according to a calculation rule [ sum, count, max, min.
For example, if the aggregate dimension indicates a video type and the indicator includes click volume, the data statistics rule indicates: and performing Data statistics on the sum (or the maximum value, the minimum value, the average value and the like) of the click amounts of the various types of videos in the T time for the user GA or all users matched with the GA in the statistical period [ Data1, Data2 ] at intervals of T.
In the embodiment of the invention, the server can send the data statistical rule to the client, so that the client can receive the data statistical rule sent by the server. The server may send the data statistics rule to the client more than once, and may send one or more data statistics rules each time the data statistics rule is sent to the client, which is not limited in the embodiment of the present invention. As to how the server sends the data statistics to the client, reference may be made to the following description in the embodiment shown in fig. 3, which is not described in detail here.
In addition, it should be noted that, in the application, the data statistical rule may also be set on the client in advance by a user (such as a developer). Of course, in the case where the user sets the data statistics rule on the client in advance, the client may also receive the data statistics rule sent by the server.
It should be further noted that when the client receives the data statistics rule sent by the server for a non-first time or when the client statically sets the data statistics rule, the client may add the data statistics rule received this time locally according to the actual service requirement (which means that the client may store multiple data statistics rules locally at the same time), or replace the existing data statistics rule locally with the data statistics rule received this time, which may implement dynamic setting of the data statistics rule locally stored by the client.
When the server sends the data statistical rule, the server can send the processing mode corresponding to the data statistical rule together, so that the client processes the data statistical rule according to the processing mode, wherein the processing refers to the newly added data statistical rule or replacing the locally existing data statistical rule.
In the embodiment of the invention, a developer or a data analyst can bury points on the client in advance. The essential technology of the embedded point is to monitor a specific event or a specific user behavior in the running process of the software application, and when the specific event and/or the specific user behavior is monitored to occur, data collection is carried out. Here, the specific user behavior may be, for example, a click behavior, an approval behavior, a comment behavior, and the like, and the specific event may be, for example, an exposure event, a presentation event, a play event, and the like.
Based on the above description, in the embodiment of the present invention, when the client detects the buried point trigger event, the client acquires corresponding buried point data.
And 203, when determining to perform data statistics, the client performs data statistics on the buried point data collected in the set historical time period according to the data statistics rule to obtain data statistics information.
As an embodiment, the client may perform data statistics periodically, that is, the client determines to perform data statistics each time a set period is reached.
As another embodiment, the client may also be triggered to perform data statistics by means of a buried point, that is, the client determines to perform data statistics when detecting a buried point trigger event for instructing to perform data statistics.
It should be noted that the above two embodiments are only two implementation manners for determining to perform data statistics by the client, and in application, the client may also determine to perform data statistics by other manners, for example, the client may determine to perform data statistics when receiving an instruction sent from the outside and used for instructing to perform data statistics, which is not limited in this embodiment of the present invention.
Based on the data statistics rule described in step 201, in the embodiment of the present invention, when determining to perform data statistics, the client searches for target buried point data matched with the current data statistics rule from the buried point data collected in the set historical time period, and then performs data statistics on the target buried point data according to the current data statistics rule. Here, the set history period may be a period of a preset time period (e.g., T) before the current time.
For example, the client searches target buried point data collected when the user GA clicks the video from the buried point data collected in the set historical time period, so as to count the sum (or the maximum value, the minimum value, the average value, and the like) of the click amounts of the user GA to various types of videos in the set historical time period according to the searched target buried point data.
It should be noted that, the current data statistics rule described above refers to a data statistics rule currently stored locally at the client, and as can be seen from the above description, multiple data statistics rules may be stored locally at the client at the same time, so in this step 202, the client may perform, according to each data statistics rule, a step of performing data statistics on buried point data collected in a set history time period according to the data statistics rule to obtain data statistics information.
In addition, it should be noted that, in practical applications, different data statistics rules may relate to partially the same calculation rule, for example, both data statistics rules relate to performing data statistics on the indexes MA, MB, MC, and the like of the users GA, GB, GC, and the like within the time T according to the calculation rule [ sum ], in this case, the client may analyze a plurality of data statistics rules in advance to analyze a calculation rule common to the plurality of data statistics rules, and perform statistical calculation only once for the common calculation rule, which may improve the data statistics efficiency of the client and save the calculation resources of the client.
In addition, in practical applications, there may be a case where the target buried point data does not include the dimension information. In this case, the client may send a data query request to the server, where the data query request is used to instruct to obtain the dimension information corresponding to the target buried point data, and then the client receives the dimension information (hereinafter referred to as target data) returned by the server, and performs statistics on the target buried point data according to the data statistics rule stored in the client, and obtains the data statistics information.
For example, if the target buried point data does not include a video type, the client may send a data query request to the server, where the data query request is used to indicate a video type corresponding to the obtained buried point data, that is, a video type of a video clicked by the user GA, and the server responds to the data query request and returns the video type of the video clicked by the user GA to the client.
And step 204, the client sends the data statistical information to the server so that the server performs service processing based on the data statistical information.
In the embodiment of the present invention, after the client executes step 203 and obtains the data statistical information, the client may send the data statistical information to the server in real time, so that the server performs service processing based on the data statistical information.
According to the technical scheme provided by the embodiment of the invention, partial data statistics work is transferred from the server side to the client side, and the client side sends the data statistics information to the server side, so that the server side does not need to start calculation from bottom data any more, but calculates on the basis of the data statistics information sent by the client side, and therefore, the calculation cost of the server side is reduced.
Referring to fig. 3, a flowchart of an embodiment of a data processing method according to an embodiment of the present invention is provided. As an embodiment, the method may be applied to a server, such as the server 11 shown in fig. 1, and the process may include the following steps:
In the embodiment of the invention, a user (such as an operator) can set the corresponding data statistical rule on the server side according to the actual service requirement, and the data statistical rules corresponding to different services are different. For example, in an exemplary business scenario in which an operator plans to count the daily viewing duration of an art program by a college student, the data statistics rules may include the following: the data statistics rule comprises a user group (college student), a statistics period (every day), a polymerization dimension (comprehensive programs) and an index (watching duration), and the specific significance of the data statistics rule is as follows: counting the watching time of the university student group to the comprehensive programs every day; in another exemplary business scenario, where an operator plans to count the stay time of women in a first-line city on different video channels per day, the data statistics rules may include the following: a user group [ first-line city and women ], a statistical period [ daily ], an aggregation dimension [ channel 1, channel 2,. the.. the.. the stay duration ], and the specific significance of the data statistical rule is as follows: women in the first-line city are counted for stay time under different video channels every day.
The server may perform step 301 after receiving an instruction indicating a statistical rule for the distribution data. Here, the instruction for instructing to distribute the data statistics rule may be triggered manually by the user, or may be triggered automatically by the server according to the configuration of the user, for example, the user configures a plurality of data statistics rules in advance on the server, and configures the sending time of each data statistics rule, so that the server may trigger the instruction automatically when the sending time of any data statistics rule arrives, and then step 301 is executed.
In the embodiment of the present invention, the server may determine at least one target client for each data statistical rule to be sent, and send the data statistical rule to the at least one target client. It should be noted that the server may determine different target clients according to different data statistics rules, which means that different target clients may receive different data statistics rules.
Specifically, as an embodiment, the server may determine at least one target client by: the server analyzes the user identification from the data statistical rule to be sent, and determines the client corresponding to each analyzed user identification as a target client.
As another example, the server may determine at least one target client by: the server analyzes the user characteristics from the data statistical rules to be sent, determines target users matched with the analyzed user characteristics from all stock users, and determines the client corresponding to the target users as target clients.
It should be noted that the two embodiments of determining at least one target client by the server described above are merely exemplary, and in application, the server may also determine the target client by other ways, which is not limited in this embodiment of the present invention.
In the embodiment of the present invention, a Dynamic computation Rule distribution service (DCRD) may be deployed on the server, and based on the DCRD, the server may issue a data statistics Rule to the target client.
In the embodiment of the present invention, how the target client performs statistics on the buried point data collected by the target client according to the data statistics rule to obtain the data statistics information and send the data statistics information to the server may refer to the description in the flow illustrated in fig. 2, and details are not described here.
And step 303, performing service processing based on the data statistical information.
In the embodiment of the invention, after the server receives the data statistical information sent by at least one target client and before the service processing is carried out based on the data statistical information, the data statistical information can be stored in the corresponding storage medium, so that when the server carries out the service processing based on the data statistical information, the data statistical information can be obtained from the corresponding storage medium according to the service to be processed. For example, the server may store the data statistics information in an offline table, so that a business process performs data statistics by using the data in the offline table. For example, the server may store the data statistics information into a real-time service flow, so that a certain service process may use the data in the real-time service flow to make a real-time data warehouse, a real-time flow report, and the like.
As an embodiment, the server may determine a target storage medium (e.g., a certain offline table) for storing the data statistics according to a data statistics rule corresponding to the data statistics, and then write the data statistics into the target storage medium. Here, the data statistics rule corresponding to the data statistics information is: and the data statistical information is obtained by the target client side according to the corresponding data statistical rule. In application, the storage medium corresponding to the data statistical rule can be set manually by the maker of the data statistical rule.
In addition, as an embodiment, after receiving the data statistics information sent by at least one client, the server does not write the data statistics information into a target storage medium immediately, but stores the data statistics information into one or more preset message middleware (e.g., a message queue), then reads the data statistics information from the one or more preset message middleware according to a preset data reading policy (e.g., a polling reading policy), determines a target storage medium for storing the data statistics information according to a data statistics rule corresponding to the currently read data statistics information when the data statistics information is read, and writes the currently read data statistics information into the target storage medium. By this processing, asynchronous processing between reception of the data statistics information and the disk dropping (the disk dropping is referred to as writing to the target storage medium) can be realized, which eliminates the need for clock synchronization between the process responsible for reception of the data statistics information and the process responsible for the disk dropping, and can macroscopically improve the operating efficiency of the process in the server.
According to the technical scheme provided by the embodiment of the invention, partial data statistics work is transferred from the server side to the client side, and the client side sends the data statistics information to the server side, so that the server side does not need to start calculation from bottom data any more, but calculates on the basis of the data statistics information sent by the client side, and therefore, the calculation cost of the server side is reduced.
Corresponding to the foregoing embodiments of the data processing method, the present invention also provides a block diagram of an embodiment of a data processing apparatus.
Referring to fig. 4, a block diagram of an embodiment of a data processing apparatus according to an embodiment of the present invention is provided. As an embodiment, the device can be applied to a client, such as the clients 12-13 illustrated in FIG. 1, and as shown in FIG. 4, the device comprises:
a rule receiving module 41, configured to receive a data statistics rule sent by a server;
the data acquisition module 42 is configured to acquire corresponding buried point data when a buried point trigger event is detected;
the data statistics module 43 is configured to perform data statistics on the buried point data collected in a set historical time period according to the data statistics rule when determining to perform data statistics, so as to obtain data statistics information;
and a data reporting module 44, configured to send the data statistics information to a server, so that the server performs service processing based on the data statistics information.
In a possible implementation, the data statistics module 43 includes:
the searching submodule is used for searching target buried point data matched with the data statistical rule from the buried point data collected in a set historical time period;
and the statistic submodule is used for carrying out data statistics on the target buried point data according to the data statistics rule.
In a possible embodiment, the device further comprises (not shown in the figures):
the query module is used for sending a data query request to the server if the target buried point data does not contain dimension information, wherein the data query request is used for indicating to obtain the dimension information corresponding to the target buried point data;
the data receiving module is used for receiving target data returned by the server, and the target data comprises dimension information corresponding to the target buried point data;
the statistics submodule is specifically configured to: and performing data statistics on the target buried point data based on the target data according to the data statistics rule.
Referring to fig. 5, a block diagram of another data processing apparatus according to an embodiment of the present invention is shown. As an embodiment, the apparatus may be applied to a server, for example, the server 11 illustrated in fig. 1, and as shown in fig. 5, the apparatus includes:
a distribution module 51, configured to determine at least one target client, and send a data statistics rule to the at least one target client;
the receiving module 52 is configured to receive data statistics information sent by at least one client, where the data statistics information is obtained by the target client performing statistics on buried point data collected by the target client according to the data statistics rule;
and the processing module 53 is configured to perform service processing based on the data statistics information.
In a possible implementation, the distribution module 51 is specifically configured to:
analyzing a user identifier from a data statistical rule to be sent;
and determining the client corresponding to the user identification as a target client.
In a possible implementation, the distribution module 51 is specifically configured to:
analyzing user characteristics from a data statistical rule to be sent;
determining a target user matched with the user characteristics;
and determining the client corresponding to the target user as a target client.
In a possible embodiment, the device further comprises (not shown in the figures):
a determining module, configured to determine, after the receiving of the data statistics information sent by at least one target client, a target storage medium for storing the data statistics information according to the data statistics rule corresponding to the data statistics information;
and the first storage module is used for writing the data statistical information into the target storage medium.
In a possible embodiment, the device further comprises (not shown in the figures):
the second storage module is used for storing the data statistical information into a preset message middleware before the target storage medium for storing the data statistical information is determined according to the data statistical rule corresponding to the data statistical information;
the determining module is specifically configured to: reading the data statistical information from the preset message middleware according to a preset data reading strategy; when the data statistical information is read, determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the currently read data statistical information;
the first storage module is specifically configured to write the currently read data statistics information into the target storage medium.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 600 shown in fig. 6 includes: at least one processor 601, memory 602, at least one network interface 604, and other user interfaces 603. The various components in the electronic device 600 are coupled together by a bus system 605. It is understood that the bus system 605 is used to enable communications among the components. The bus system 605 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 605 in fig. 6.
The user interface 603 may include, among other things, a display, a keyboard, or a pointing device (e.g., a mouse, trackball, touch pad, or touch screen, among others.
It will be appreciated that the memory 602 in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be a Read-only memory (ROM), a programmable Read-only memory (PROM), an erasable programmable Read-only memory (erasabprom, EPROM), an electrically erasable programmable Read-only memory (EEPROM), or a flash memory. The volatile memory may be a Random Access Memory (RAM) which functions as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (staticiram, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (syncronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM ), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and direct memory bus RAM (DRRAM). The memory 602 described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
In some embodiments, memory 602 stores the following elements, executable units or data structures, or a subset thereof, or an expanded set thereof: an operating system 6021 and application programs 6022.
The operating system 6021 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks. The application program 6022 includes various application programs such as a media player (MediaPlayer), a Browser (Browser), and the like, and is used to implement various application services. A program implementing the method of an embodiment of the invention can be included in the application program 6022.
In the embodiment of the present invention, by calling a program or an instruction stored in the memory 602, specifically, a program or an instruction stored in the application program 6022, the processor 601 is configured to execute the method steps provided by the method embodiments, for example, including:
determining at least one target client, and sending a data statistical rule to at least one target client; receiving data statistical information sent by at least one target client, wherein the data statistical information is obtained by the target client through statistics on buried point data acquired by the target client according to the data statistical rule; and performing service processing based on the data statistical information.
Or, receiving a data statistical rule sent by the server; when a buried point triggering event is detected, collecting corresponding buried point data; when data statistics is determined, performing data statistics on the buried point data collected in a set historical time period according to the data statistics rule to obtain data statistics information; and sending the data statistical information to a server so that the server performs service processing based on the data statistical information.
The method disclosed by the above-mentioned embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 601. The processor 601 may be a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software elements in the decoding processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, among other storage media that are well known in the art. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the method in combination with the hardware thereof.
It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units configured to perform the functions described herein, or a combination thereof.
For a software implementation, the techniques described herein may be implemented by means of units performing the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
The electronic device provided in this embodiment may be the electronic device shown in fig. 6, and may perform all the steps of the data processing method shown in fig. 2 to 3, so as to achieve the technical effect of the data processing method shown in fig. 2 to 3, and for brevity, it is specifically described with reference to fig. 2 to 3, and no further description is provided here.
The embodiment of the invention also provides a storage medium (computer readable storage medium). The storage medium herein stores one or more programs. Among others, the storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, a hard disk, or a solid state disk; the memory may also comprise a combination of memories of the kind described above.
When one or more programs in the storage medium are executable by one or more processors, the data processing method executed on the electronic device side is realized.
The processor is used for executing the data processing program stored in the memory to realize the following steps of the data processing method executed on the electronic equipment side:
determining at least one target client, and sending a data statistical rule to at least one target client; receiving data statistical information sent by at least one target client, wherein the data statistical information is obtained by the target client through statistics on buried point data acquired by the target client according to the data statistical rule; and performing service processing based on the data statistical information.
Or, receiving a data statistical rule sent by the server; when a buried point triggering event is detected, collecting corresponding buried point data; when data statistics is determined, performing data statistics on the buried point data collected in a set historical time period according to the data statistics rule to obtain data statistics information; and sending the data statistical information to a server so that the server performs service processing based on the data statistical information.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (12)
1. A data processing method is applied to a server, and the method comprises the following steps:
determining at least one target client, and sending a data statistical rule to at least one target client;
receiving data statistical information sent by at least one target client, wherein the data statistical information is obtained by the target client through statistics on buried point data acquired by the target client according to the data statistical rule;
and performing service processing based on the data statistical information.
2. The method of claim 1, wherein determining at least one target client comprises:
analyzing a user identifier from a data statistical rule to be sent;
and determining the client corresponding to the user identification as a target client.
3. The method of claim 1, wherein determining at least one target client comprises:
analyzing user characteristics from a data statistical rule to be sent;
determining a target user matched with the user characteristics;
and determining the client corresponding to the target user as a target client.
4. The method of claim 1, wherein after said receiving data statistics sent by at least one of said target clients, said method further comprises:
determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the data statistical information;
and writing the data statistical information into the target storage medium.
5. The method of claim 4, wherein before determining the target storage medium for storing the data statistics according to the data statistics rule corresponding to the data statistics, the method further comprises:
storing the data statistical information into a preset message middleware;
determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the data statistical information; writing the data statistics to the target storage medium, including:
reading the data statistical information from the preset message middleware according to a preset data reading strategy;
when the data statistical information is read, determining a target storage medium for storing the data statistical information according to the data statistical rule corresponding to the currently read data statistical information; and writing the currently read data statistical information into the target storage medium.
6. A data processing method is applied to a client, and the method comprises the following steps:
receiving a data statistical rule sent by a server;
when a buried point triggering event is detected, collecting corresponding buried point data;
when data statistics is determined, performing data statistics on the buried point data collected in a set historical time period according to the data statistics rule to obtain data statistics information;
and sending the data statistical information to a server so that the server performs service processing based on the data statistical information.
7. The method of claim 6, wherein performing data statistics on the buried point data collected in a set historical time period according to the data statistics rule comprises:
searching target buried point data matched with the data statistical rule from the buried point data collected in a set historical time period;
and performing data statistics on the target buried point data according to the data statistics rule.
8. The method of claim 7, wherein the data statistics rule comprises an aggregation dimension, the method further comprising:
if the target buried point data does not contain dimension information, sending a data query request to the server, wherein the data query request is used for indicating to obtain the dimension information corresponding to the target buried point data;
receiving target data returned by the server, wherein the target data comprises dimension information corresponding to the target buried point data;
the data statistics of the target buried point data according to the data statistics rule comprises the following steps:
and performing data statistics on the target buried point data based on the target data according to the data statistics rule.
9. A data processing apparatus, applied to a server, the apparatus comprising:
the distribution module is used for determining at least one target client and sending a data statistical rule to the at least one target client;
the receiving module is used for receiving data statistical information sent by at least one target client, and the data statistical information is obtained by counting buried point data acquired by the target client according to the data statistical rule by the target client;
and the processing module is used for processing the service based on the data statistical information.
10. A data processing apparatus, applied to a client, the apparatus comprising:
the rule receiving module is used for receiving the data statistical rule sent by the server;
the data acquisition module is used for acquiring corresponding buried point data when a buried point triggering event is detected;
the data statistics module is used for carrying out data statistics on the buried point data collected in a set historical time period according to the data statistics rule when the data statistics is determined to be carried out, so that data statistics information is obtained;
and the data reporting module is used for sending the data statistical information to a server so that the server performs service processing based on the data statistical information.
11. An electronic device, comprising: a processor and a memory, the processor being configured to execute a data processing program stored in the memory to implement the data processing method of any one of claims 1 to 5 or 6 to 8.
12. A storage medium storing one or more programs executable by one or more processors to implement the data processing method of any one of claims 1 to 5 or 6 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110519755.2A CN113190411A (en) | 2021-05-12 | 2021-05-12 | Data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110519755.2A CN113190411A (en) | 2021-05-12 | 2021-05-12 | Data processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113190411A true CN113190411A (en) | 2021-07-30 |
Family
ID=76981407
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110519755.2A Pending CN113190411A (en) | 2021-05-12 | 2021-05-12 | Data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113190411A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113645214A (en) * | 2021-08-03 | 2021-11-12 | 北京百度网讯科技有限公司 | Data detection method and device, electronic equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598868A (en) * | 2016-12-24 | 2017-04-26 | 上海亿账通互联网科技有限公司 | Dynamic point burying method and system for application program of client |
| CN107566148A (en) * | 2016-07-01 | 2018-01-09 | 北京京东尚科信息技术有限公司 | Analysis method, system, device and the terminal of terminal applies operation data |
| CN109740089A (en) * | 2018-11-30 | 2019-05-10 | 东软集团股份有限公司 | Collecting method, device, system, readable storage medium storing program for executing and electronic equipment |
| CN111813629A (en) * | 2020-07-13 | 2020-10-23 | 赞同科技股份有限公司 | Method, device and equipment for generating monitoring data of Web page |
| CN112311686A (en) * | 2020-09-27 | 2021-02-02 | 长沙市到家悠享网络科技有限公司 | Data processing method and device, electronic equipment and storage medium |
-
2021
- 2021-05-12 CN CN202110519755.2A patent/CN113190411A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107566148A (en) * | 2016-07-01 | 2018-01-09 | 北京京东尚科信息技术有限公司 | Analysis method, system, device and the terminal of terminal applies operation data |
| CN106598868A (en) * | 2016-12-24 | 2017-04-26 | 上海亿账通互联网科技有限公司 | Dynamic point burying method and system for application program of client |
| CN109740089A (en) * | 2018-11-30 | 2019-05-10 | 东软集团股份有限公司 | Collecting method, device, system, readable storage medium storing program for executing and electronic equipment |
| CN111813629A (en) * | 2020-07-13 | 2020-10-23 | 赞同科技股份有限公司 | Method, device and equipment for generating monitoring data of Web page |
| CN112311686A (en) * | 2020-09-27 | 2021-02-02 | 长沙市到家悠享网络科技有限公司 | Data processing method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113645214A (en) * | 2021-08-03 | 2021-11-12 | 北京百度网讯科技有限公司 | Data detection method and device, electronic equipment and readable storage medium |
| CN113645214B (en) * | 2021-08-03 | 2023-08-15 | 北京百度网讯科技有限公司 | Data detection method, device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11609839B2 (en) | Distributed code tracing system | |
| US11782989B1 (en) | Correlating data based on user-specified search criteria | |
| US10614132B2 (en) | GUI-triggered processing of performance data and log data from an information technology environment | |
| US10019496B2 (en) | Processing of performance data and log data from an information technology environment by using diverse data stores | |
| WO2021174694A1 (en) | Operation and maintenance monitoring method and apparatus based on data center, device, and storage medium | |
| US10318541B2 (en) | Correlating log data with performance measurements having a specified relationship to a threshold value | |
| US10225136B2 (en) | Processing of log data and performance data obtained via an application programming interface (API) | |
| US11119982B2 (en) | Correlation of performance data and structure data from an information technology environment | |
| US20170286499A1 (en) | Query-Triggered Processing of Performance Data and Log Data from an Information Technology Environment | |
| US10248674B2 (en) | Method and apparatus for data quality management and control | |
| CN108509313B (en) | Service monitoring method, platform and storage medium | |
| CN111522711B (en) | Data monitoring processing system, method, execution end, monitoring end and electronic equipment | |
| CN110569222B (en) | Link tracking method and device, computer equipment and readable storage medium | |
| CN111740860A (en) | Log data transmission link monitoring method and device | |
| CN110650146A (en) | Anti-cheating method and device and electronic equipment | |
| CN110266555B (en) | Method for analyzing website service request | |
| CN115273191A (en) | Face document gathering method, face recognition method, device, equipment and medium | |
| CN113190411A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111158926A (en) | Service request analysis method, device and equipment | |
| CN112631879A (en) | Data acquisition method and device, computer readable medium and electronic equipment | |
| CN111046240B (en) | Gateway traffic statistics method, device, computer equipment and storage medium | |
| WO2023045434A1 (en) | Access detection method, system, and apparatus | |
| KR102464688B1 (en) | Method and apparatus for detrmining event level of monitoring result | |
| CN110020166A (en) | A kind of data analysing method and relevant device | |
| CN106547788B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |