CN114722078A - Data statistical method, device, equipment, storage medium and program product - Google Patents

Data statistical method, device, equipment, storage medium and program product Download PDF

Info

Publication number
CN114722078A
CN114722078A CN202210262152.3A CN202210262152A CN114722078A CN 114722078 A CN114722078 A CN 114722078A CN 202210262152 A CN202210262152 A CN 202210262152A CN 114722078 A CN114722078 A CN 114722078A
Authority
CN
China
Prior art keywords
data
statistical
target
historical
target data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210262152.3A
Other languages
Chinese (zh)
Inventor
黎丙成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Singapore Pte Ltd
Original Assignee
Bigo Technology Singapore Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bigo Technology Singapore Pte Ltd filed Critical Bigo Technology Singapore Pte Ltd
Priority to CN202210262152.3A priority Critical patent/CN114722078A/en
Publication of CN114722078A publication Critical patent/CN114722078A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24564—Applying rules; Deductive queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2358—Change logging, detection, and notification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24539—Query rewriting; Transformation using cached or materialised query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462—Approximate or statistical queries

Abstract

The embodiment of the application discloses a data statistical method, a device, equipment, a storage medium and a program product, and relates to the technical field of data processing. The method comprises the following steps: the method comprises the steps of obtaining a log file of a database, analyzing the log file, obtaining statistical configuration information, combining target data with historical statistical data stored in a first storage space based on the statistical configuration information to obtain target statistical data, updating and storing the target statistical data into the first storage space, wherein the first storage space is used for storing the statistical data for inquiry. Target data are directly obtained by analyzing log files of the database, the target data are counted according to the counting configuration information, and the counting result is stored in advance, so that the data with the query requirement can be counted only, the storage resource of a computer is saved, the query efficiency is improved, and the query delay is reduced by storing the counting result in advance.

Description

Data statistical method, device, equipment, storage medium and program product
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data statistics method, apparatus, device, storage medium, and program product.
Background
In a live broadcast scene, a large amount of data is generated at every moment based on object behaviors, and accordingly, statistical query requirements for the data are generated.
In the related technology, the real-time generated flow data is stored in a database, when a server receives an inquiry request of a service party, database inquiry languages are spliced according to the inquiry request, then the database is requested to be inquired according to the spliced database inquiry languages, and finally statistical results obtained by inquiry are fed back to the service party.
However, the above method has large performance loss to the database when the data volume is large and frequent query is required, and the query response time is long, so that the query requirement for massive data cannot be met.
Disclosure of Invention
The embodiment of the application provides a data statistical method, a device, equipment, a storage medium and a program product. The technical scheme is as follows:
according to an aspect of an embodiment of the present application, there is provided a data statistics method, including:
acquiring a log file of a database, wherein the log file is used for recording data updating of the database;
analyzing the log file to obtain target data in the log file, wherein the target data is updated data in the log file;
acquiring statistical configuration information, wherein the statistical configuration information is a preset data statistical scheme;
combining the target data with historical statistical data stored in a first storage space based on the statistical configuration information to obtain target statistical data;
and updating and storing the target statistical data into the first storage space, wherein the first storage space is used for storing the statistical data for inquiry.
According to an aspect of an embodiment of the present application, there is provided a data statistics apparatus, the apparatus including:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a log file of a database, and the log file is used for recording data updating of the database;
the analysis module is used for analyzing the log file to obtain target data in the log file, wherein the target data is updated data in the log file;
the acquisition module is further configured to acquire statistical configuration information, where the statistical configuration information is a preset data statistical scheme;
the processing module is used for combining the target data with historical statistical data stored in a first storage space based on the statistical configuration information to obtain target statistical data;
the processing module is further configured to update and store the target statistical data into the first storage space, where the first storage space is used to store statistical data for querying.
According to an aspect of embodiments of the present application, there is provided a computer device, which includes a processor and a memory, where at least one instruction, at least one program, a code set, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the data statistics method described in any one of the above embodiments.
According to an aspect of the embodiments of the present application, there is provided a computer-readable storage medium having at least one program code stored therein, the program code being loaded and executed by a processor to implement the data statistics method of any one of the above embodiments.
According to an aspect of the embodiments of the present application, there is provided a computer program product, the computer program product including computer instructions, the computer instructions being stored in a computer-readable storage medium, and a processor reading and executing the computer instructions from the computer-readable storage medium to implement the above data statistical method.
The technical scheme provided by the embodiment of the application at least comprises the following beneficial effects:
target data are directly obtained by analyzing log files of the database, the target data are counted according to the counting configuration information, and the counting result is stored in advance, so that the data with the query requirement can be counted only, the storage resource of a computer is saved, the query efficiency is improved, and the query delay is reduced by storing the counting result in advance.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic illustration of an environment for implementing an embodiment provided by an embodiment of the present application;
FIG. 2 is a flow chart of a data statistics method provided by an embodiment of the present application;
FIG. 3 is a flow chart of a data statistics method provided by another embodiment of the present application;
FIG. 4 is a flow chart of a data statistics method provided by another embodiment of the present application;
FIG. 5 is a diagram illustrating a non-periodic class statistics structure, according to an embodiment of the present application;
FIG. 6 is a diagram illustrating a periodic class statistics structure, according to an embodiment of the present application;
FIG. 7 is a technical schematic provided by one embodiment of the present application;
FIG. 8 is a block diagram of a data statistics apparatus provided in one embodiment of the present application;
fig. 9 is a block diagram of a data statistics apparatus according to another embodiment of the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the present application clearer, the following will describe embodiments of the present application in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," and the like, in this application, are used for distinguishing between similar items and items that have substantially the same function or similar functionality, and it is to be understood that "first" and "second" do not have a logical or temporal dependency, nor do they define a quantity or order of execution.
According to the traditional data statistics query technical scheme, database languages are spliced according to the requirements of a service party, the database is requested to be queried according to the spliced database languages, and then the query result is fed back to the service party. The embodiment of the application provides a data statistics method, which is characterized in that the running water data is written into a database, the updated running water data, namely target data, is obtained by analyzing a log file of the database, the target data is counted according to statistical configuration information, and a statistical result is written into a memory to complete pre-statistics of the data. According to the embodiment of the application, the data needing to be counted is configured individually by setting the counting configuration information, so that the data can be counted as required, the storage space of a computer can be saved, and the query efficiency is improved; meanwhile, the statistical result is pre-calculated and written into the storage in the embodiment of the application, so that the query delay is reduced, and a higher-frequency query scene can be dealt with.
Referring to fig. 1, a schematic diagram of an implementation environment of an embodiment of the present application is shown. Illustratively, the implementation environment includes a first terminal 100, a second terminal 110, a server 120, and a communication network 130. Optionally, the implementation environment includes a plurality of different first terminals 100 and second terminals 110, and fig. 1 illustrates only one first terminal 100 and second terminal 110 as an example.
In some optional embodiments, the first terminal 100 has installed and run a target application that can generate the streaming data in real time, illustratively, the target application includes a live application, a social application, a game application, a shopping application, and the like, which is not limited in this embodiment.
The first terminal 100 uploads the running data to the server 120 in real time when running the target application, optionally, after receiving the running data, the server 120 writes the running data into the database first, during the writing process of the running data, the database generates a corresponding log file for recording data update of the database, and the server 120 listens to the log file to obtain the target data, for example: when new streaming data is written into the database, the log file records the update action of the database, and the server 120 monitors the update action and analyzes the update action to obtain the target data. After the target data is obtained, the server 120 performs statistical processing on the target data according to the preconfigured statistical configuration information, and writes the statistical result into the storage space.
In some optional embodiments, the second terminal 110 runs a query system, and may query the statistical result. The second terminal 110 sends a statistical query request to the server 120, and in response to the statistical query request, the server 120 queries the storage space and feeds back the statistical result to the second terminal 110.
In some alternative embodiments, the first terminal 100 and the second terminal 110 are, but not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart home appliance, a smart car terminal, a smart speaker, and the like. Alternatively, the first terminal 100 and the second terminal 110 may be two independent devices or may be the same device, which is not limited in this application.
In some optional embodiments, the server 120 may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing basic cloud computing services such as cloud services, a cloud database, cloud computing, and the like, and the server 120 may also be implemented as a node in a blockchain system, which is not limited herein.
The first terminal 100, the second terminal 110 and the server 120 are connected through a communication network 130, and in some alternative embodiments, the communication network 130 may be a wired network or a wireless network, which is not limited herein.
It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals referred to in this application are authorized by the user or fully authorized by various parties, and the collection, use and processing of the relevant data are subject to relevant laws and regulations and standards in relevant countries and regions. For example, the pipelined data referred to in this application is obtained with sufficient authorization.
The data statistical method provided by the embodiment of the application can be at least applied to the following application scenes:
in the live broadcast application program, based on the behaviors of the object (such as the behaviors of recharging, gift sending and the like), a large amount of data is generated at every moment, and after the object is authorized, the data can be obtained and stored in a database. According to the data statistical method, a complex data engine is not needed to be deployed, log files of a database are monitored, real-time streaming data are analyzed and obtained, the data can be pre-counted through standardized configuration and data processing, technical personnel can configure statistical configuration files according to actual requirements, storage resources of a computer are saved, meanwhile, due to the adoption of the pre-counting scheme, live broadcast product strategies can directly return cached statistical values during query, and query delay is greatly reduced.
It should be noted that the above application scenarios are only illustrative examples, and the embodiments of the present application do not limit other application scenarios of the data statistics method.
With reference to the above description and implementation environments, fig. 2 is a flowchart of a data statistics method provided in an embodiment of the present application, and as shown in fig. 2, the method includes:
step 201, obtaining a log file of a database.
The log file is used for recording data update of the database.
Optionally, the database is a relational database management system (My structured query Language, MySQL), illustratively, the log file corresponding to MySQL is a Binary log (Binlog), the Binlog log can be directly enabled in the configuration file of MySQL, and the Binlog log is automatically updated when data in MySQL is updated (for example, data is added, deleted, changed, and the like).
Optionally, before obtaining the log file of the database, the following steps should be further included:
acquiring running water data; and writing the running water data into a database. The pipeline data refers to real-time data generated when an object uses an application program, and includes: in the live broadcast application program, the object can give the virtual gift to the main broadcast in a live broadcast room, the process of giving the virtual gift corresponds to a piece of streaming data each time, the object can carry out authorization management on the data in the setting interface of the live broadcast application program, and the object agrees to upload the streaming data generated in real time to the server after agreeing to authorization.
In some optional embodiments, the database comprises a pipelined database corresponding to at least two service servers. Illustratively, the implementation of the usage scenario of the flow database corresponding to the at least two service servers includes at least one of the following scenarios:
scene one: writing the flow data into different flow databases, illustratively, each flow database is connected with the same/different number of service terminals, the same/different application programs are installed in the service terminals, and the flow data generated in each service terminal is written into the flow database connected with the service terminal; alternatively, each pipeline database is connected with different modules of the application program, and the pipeline data generated in each module is written into the pipeline database connected with the module, for example: when the application is implemented as a shopping application, the order information for which the resource exchange has been implemented and the commodity information for which the shopping cart is added can be written into different flow databases, respectively.
Scene two: determining a main flow database in a plurality of flow databases, wherein other flow databases except the main flow database are all subordinate flow databases, flow data are all written into the main flow database, and the subordinate flow databases are read when the flow data need to be inquired, so that read-write separation is realized.
Step 202, analyzing the log file to obtain the target data in the log file.
The target data is data with updates in the log file. Illustratively, a piece of running water data is written into the database, and the database is updated, so that the running water data is the target data.
In some optional embodiments, the process of obtaining the target data further comprises the following steps:
monitoring a log file; and responding to the content update of the log file, and analyzing the log file to obtain target data in the log file.
Illustratively, when the log file is implemented as a Binlog log, optionally, an execution statement of MySQL is recorded in the Binlog log. When a piece of stream data is written into MySQL, the written execution statement is added into the Binlog log, it is monitored that the content of the Binlog log is added with an execution statement, and the Binlog analyzer analyzes the execution statement, so that target data in the execution statement is obtained.
Optionally, after the target data is acquired, the target data is written into a message system to wait for processing, and illustratively, the target data may be written into a message queue of a distributed message system (e.g., Apache Kafka) for caching.
Optionally, the target data includes an object identifier, and before processing the target data, the target data may be further subjected to split processing. The splitting processing means that the target data with the same object identifier is split to the same queue or a downstream node, and optionally, the target data is split according to the object identifier based on a hash algorithm, and the target data with the same object identifier is serially processed.
Step 203, obtaining statistical configuration information.
The statistical configuration information is a preset data statistical scheme.
Optionally, the statistical configuration information may be information that is configured in the statistical configuration file in advance by the service party, and illustratively, the service party may be a person skilled in the relevant art for developing the target application program.
Optionally, the data statistics scheme is used to instruct the server to perform statistics on the data in the database according to rules for the target data, for example:
when the target application program is implemented as a main broadcasting application program, the object behavior corresponding to the target data is that the object pays attention to a certain live broadcasting room currently, the data statistical scheme corresponding to the object behavior is that the total number of the live broadcasting rooms paid attention to by the object within 7 days is statistically queried, and then the total number of the live broadcasting rooms paid attention to by the object, which are pushed forward for 7 days from the current time point, is queried in the database.
Optionally, the data statistics scheme is further used to indicate whether the server needs to perform statistics on the target data, and illustratively, the manner for managing the data statistics requirement by the data statistics scheme includes at least one of the following manners:
1. optionally, the statistical configuration information includes specific statistical categories of data (for example, the number of live broadcasting rooms in which the object is concerned, the duration of live broadcasting watching of the object, and the like), account categories of the object in the target application program (for example, a main broadcasting account, a viewer account, and the like), and the service provider may add the statistical requirement switch according to the categories, for example: carrying out statistic demand configuration on the number of live broadcasting rooms concerned by the audience account in 7 days; the statistical configuration information further includes an object identifier, and the service party may also manage statistical requirements of single object data in the statistical configuration information, for example: the method is characterized in that all data of a certain object account are forbidden to be counted, or all data of a certain object account within a specific time period are forbidden to be counted.
Optionally, the demand switch may be set to a real switch button form, and the demand switch being in the "on" state indicates that statistics on the target data is required, and the demand switch being in the "off" state indicates that statistics on the target data is not required; the demand switch may also be set to a simple 0 or 1, with the demand switch set to "0" indicating that statistics on the target data is not required and the demand switch set to "1" indicating that statistics on the target data is required.
2. Optionally, the target data includes an object identifier and a statistical category, corresponding statistical configuration information is queried in the statistical configuration file according to the object identifier and the statistical category, and in response to the statistical configuration file having the corresponding statistical configuration information, it indicates that the service party has a statistical requirement for the target data; and responding to the statistical configuration file that the corresponding statistical configuration information cannot be inquired, wherein the statistical configuration file indicates that the business party has no statistical requirement on the target data.
The above-mentioned manner for managing the statistical requirements is only an illustrative example, and the embodiment of the present application does not limit this.
And step 204, combining the target data with the historical statistical data stored in the first storage space based on the statistical configuration information to obtain target statistical data.
Optionally, the historical statistical data refers to statistical data stored in the first storage space during a historical period of time.
Optionally, in response to that the data statistics scheme in the statistical demand information is that statistics needs to be performed on the target data, the processing condition on the target data includes at least one of the following conditions:
1. and directly combining the target data with the historical statistical data stored in the first storage space to obtain the target statistical data.
2. According to the data statistical scheme of the target data, a result is statistically inquired from a database, and the result is the target statistical data; or, according to the data statistical scheme of the target data, directly storing the target data into the first storage space as the target statistical data.
Optionally, in response to the data statistics scheme in the statistical demand information, the target data does not need to be counted, and the target data is discarded, that is, the data statistics operation is not performed on the target data.
And step 205, updating and storing the target statistical data into the first storage space.
The first storage space is used for storing the statistical data for inquiry.
Illustratively, the first storage space may be implemented as a non-relational database (e.g., Redis cache).
Optionally, the target statistical data is obtained by querying from a database according to the statistical configuration information and stored in the first storage space, or the target statistical data is obtained by combining the target data with the historical statistical data and the historical statistical data in the first storage space is updated to the target statistical data.
To sum up, in the embodiment of the present application, the target data is directly obtained by analyzing the log file of the database, the target data is counted according to the statistical configuration information, and the statistical result is stored in advance, so that only data with query requirements can be counted, the storage resource of the computer is saved, the query efficiency is improved, and the query delay is reduced by storing the statistical result in advance.
Fig. 3 is a flowchart of a data statistics method provided in another exemplary embodiment of the present application, as shown in fig. 3, the method includes:
step 301, obtaining a log file of a database.
The log file is used for recording data update of the database.
The process of obtaining the log file of the database has already been described in step 201, and is not described herein.
Step 302, analyzing the log file to obtain the target data in the log file.
The target data is data with updates in the log file.
The process of acquiring the target data in the log file has already been described in step 202, and is not described herein.
Step 303, matching the target data with historical statistical data of the first storage space.
In some alternative embodiments, the target data corresponds to a first object identifier and first statistical item information, and the historical statistical data corresponds to a second object identifier and second statistical item information.
Optionally, the object logged in the target application corresponds to a unique object identifier, and illustratively, the object identifier may be implemented as an account number in the target application.
Optionally, the first statistical item information is associated with a business behavior contained in the target data, and the second statistical item information is associated with a business behavior contained in the historical statistical data, for example: one of the statistical types corresponding to the behavior of the object concerning the live broadcast room is the number of the object concerning the live broadcast room.
In some alternative embodiments, the first object identification is matched with the second object identification; and matching the first statistical item information with the second statistical item information.
Optionally, the first object identifier and the second object identifier are matched, and if the matching fails, the first statistical item information does not need to be matched with the second statistical item information; and if the matching is successful, matching the first statistical item information with the second statistical item information.
Optionally, first matching the first statistical item information with the second statistical item information, if the matching fails, then the first object identifier does not need to be matched with the second object identifier; and if the matching is successful, matching the first object identifier with the second object identifier.
Step 304, in response to the failure of matching the target data with the historical statistical data, obtaining statistical configuration information.
In some optional embodiments, the historical statistics correspond to status information, and the status information is used to indicate the validity of the historical statistics.
In some optional embodiments, the method of obtaining statistical configuration information comprises at least one of the following methods:
1. in response to a failure to match the first object identification with the second object identification; or, the first statistical item information and the second statistical item information are failed to be matched, and statistical configuration information is obtained.
Illustratively, the case where the matching fails in the mode 1 includes the following cases:
(1) optionally, the first object identifier and the second object identifier refer to an object account number, there is no second object identifier in the first storage space, which is the same as the first object identifier, and the first object identifier and the second object identifier fail to match;
(2) optionally, the first statistical item information and the second statistical item information refer to statistical content, in the first storage space, the statistical content corresponding to the first statistical item information and the second statistical item information is inconsistent, and the first statistical item information fails to match with the second statistical item information.
Illustratively, if the first statistical item information refers to the number of live broadcasting rooms concerned by the object, the second statistical item information refers to the total number of live broadcasting rooms concerned by the object in a period of time, and the first statistical item information and the second statistical item information are matched; if the first statistical item information refers to the number of live broadcasting rooms concerned by the object, the second statistical item information refers to the total duration of the live broadcasting watched by the object in a period of time, and the first statistical item information and the second statistical item information are not matched.
As long as at least one of the above two conditions exists, the target data fails to match the historical statistical data.
2. And responding to the state information corresponding to the historical statistical data to indicate that the historical statistical data is invalid, and acquiring statistical configuration information.
Illustratively, the target data is data generated when the object pays attention to a certain live broadcast room at the current time, the historical statistical data refers to the total number of live broadcast rooms paid attention to by the object within the past 7 days, but if the status information of the historical statistical data is set to be invalid, the target data and the historical statistical data fail to be matched.
And 305, combining the target data with the historical statistical data stored in the first storage space based on the statistical configuration information to obtain target statistical data.
The process of obtaining the target statistical data based on the statistical configuration information is already described in step 204, and is not described herein again.
In some optional embodiments, in response to a successful match between the target data and the historical statistical data, combining the target data with the historical statistical data in a specified combination manner to obtain the target statistical data.
It should be noted that, because the target data is successfully matched with the historical statistical data, the default target data needs to be counted, in general, the default target data may have statistical requirements, optionally, the business side may change the statistical requirements for the target data at any time, and if the statistical requirements for the target data change from the original statistics that is needed to be counted to the statistics that is not needed, the target data needs to be discarded and the historical statistical data associated with the target data needs to be deleted from the first storage space; alternatively, the target data is discarded but the historical statistics associated therewith are not deleted from the first memory space.
Optionally, the successful matching of the target data and the historical statistical data means that the matching of the first object identifier and the second object identifier is successful, the matching of the first statistical item information and the second statistical item information is successful, and the state information corresponding to the historical statistical data is used for indicating that the historical statistical data is normal.
Optionally, the target data includes statistics, and illustratively, if the target object focuses on a live broadcast room at the current time, the statistics is 1, and if the target object cancels the focus on a live broadcast room at the current time, the statistics is-1.
Optionally, wherein specifying the combination comprises at least one of:
the historical statistical data includes a total statistical value and a total record number, illustratively, the number of the target objects in the live broadcasting room in 7 days is 1000 (excluding the number of the target objects in the live broadcasting room), the total statistical value is 1000, the target objects in the live broadcasting room in 7 days have 10 times in total (including the action of canceling the target objects in the live broadcasting room), and the total record number is 10.
The designated combination mode is to perform accumulation calculation on the statistic value in the target data and the total statistic value in the historical statistic data and update the total record number in the historical statistic data.
And secondly, specifying a combination mode refers to associating target data with historical statistical data, but not changing a statistical value in the target data or changing a total statistical value in the historical statistical data, and schematically connecting data generated by the target in a live broadcast room within 7 days to form the target statistical data.
And step 306, updating and storing the target statistical data into the first storage space.
The first storage space is used for storing the statistical data for inquiry.
The method for storing the target statistical data has already been described in step 205, and is not described herein.
In summary, in the embodiment of the present application, the target data is directly obtained by analyzing the log file of the database, the target data is counted according to the statistical configuration information, and the statistical result is stored in advance, so that only the data with the query requirement can be counted, the storage resource of the computer is saved, the query efficiency is improved, and the query delay is reduced by storing the statistical result in advance.
According to the method provided by the embodiment of the application, the target data and the historical statistical data are matched, and if the matching is successful, the target data and the historical statistical data can be directly combined to obtain the target statistical data, so that a database does not need to be inquired, and the performance loss of the database is reduced; and if the matching fails, only the target data of the statistical requirement is counted according to the statistical configuration information, so that the storage space and the computing resources of the computer are saved.
Fig. 4 is a flowchart of a data statistics method provided in another exemplary embodiment of the present application, as shown in fig. 4, the method includes:
step 401, obtaining a log file of a database.
The log file is used for recording data update of the database.
The process of obtaining the log file of the database has already been described in step 201, and is not described herein.
Step 402, analyzing the log file to obtain the target data in the log file.
The target data is data with updates in the log file.
The process of acquiring the target data in the log file has already been described in step 202, and is not described herein.
Step 403, write the target data into the data statistics write service.
The data statistics write service is used for caching the target data before statistics.
Optionally, after the target data write data statistics write service, the server further needs to receive a response result of the current write, where the response result is used to indicate whether the target data write data statistics write service is successful.
Optionally, in the process of writing the target data into the data statistics writing service, if the target data may be lost due to an abnormal condition of the network or other systems, a rollback operation needs to be performed on the target data, and the following is a specific step of the rollback operation of the target data:
writing the target data into a rollback queue in response to the target data write data statistics write service failure; sequentially rolling back the data to be rolled back to the data statistics writing service from the rolling back queue; in response to the target data rollback being successful, the target data is deleted from the rollback queue.
The data to be rolled back refers to target data of write data statistics write service failure.
Wherein the case that the target data write data statistics fails to write the service comprises at least one of the following cases:
1. if the response result of the target data write data statistics write service is not received, representing that the target data write data statistics write service fails;
2. if the response result of the target data write data statistics write service is received and the response result indicates that the target data write data statistics write service is unsuccessful, the write operation is repeated for a plurality of times, and if the response result indicates that the target data write data statistics write service is unsuccessful is still received after the write operation is repeated for a plurality of times, the target data write data statistics write service is failed.
Optionally, the successful rollback of the target data refers to that the data statistics writing service receives the target data, that is, the target data has been successfully written into the cache in the data statistics writing service.
Step 404, obtaining statistical configuration information.
The statistical configuration information is a preset data statistical scheme.
The process of obtaining the statistical configuration information is already described in step 203, and is not described herein.
Step 405, combining the target data with the historical statistical data stored in the first storage space based on the statistical configuration information to obtain target statistical data.
Optionally, the target data includes a time stamp, and the time stamp is used for indicating the generation time of the target data.
Optionally, the historical statistical data includes status information, and the status information is used to indicate validity of the historical statistical data.
The process of obtaining the target statistic based on the target data that is not rolled back is already described in step 204, and is not described herein.
Optionally, the storage structure of the historical statistical data in the cache includes the following two implementation manners:
1. aperiodic class historical statistics data structure
In some optional embodiments, the historical statistical data includes at least one historical statistical data node corresponding to the at least one historical statistical sub-data, and the historical statistical data node corresponds to a node level.
Optionally, the historical statistical subdata refers to statistical data recorded in a historical time period, the historical statistical subdata includes a timestamp, and the timestamps corresponding to the historical statistical subdata are different.
Optionally, the historical statistic data node corresponds to a node level, and the historical statistic data node may be used to store the historical statistic subdata.
Optionally, the historical statistical sub-data with the timestamp closest to the current timestamp is stored in the historical statistical data node with the higher node level.
Referring to fig. 5, the historical statistical data 500 includes three levels of historical statistical data nodes, where the level rule of a node is level 3> level 2> level 1, the latest statistical data is always stored in the historical statistical data node represented by level 3, and different historical statistical sub-data are stored in three historical statistical data nodes of different levels, respectively.
Wherein each historical statistical subdata comprises: num (total number of records), value (total count), ts (last timestamp), status (data state). The total record number refers to the total number of the target data counted from the first time to the last time stamp; the total statistics count refers to the total statistics value from the first statistics to the last timestamp, for example: the target object establishes 1000 live broadcast rooms (excluding the live broadcast room which cancels the attention) from the beginning to the current time, and the total statistics count is 1000; the last timestamp refers to a timestamp of the latest update of the historical statistical sub-data, and specifically may be a timestamp corresponding to the target data when the latest target data statistics are accumulated; the data status is used to indicate whether the historical statistical sub-data is valid data or invalid data.
In some optional embodiments, the historical statistical sub-data in the historical statistical data node with the high node level is periodically saved in the historical statistical data node with the low node level, and the process may include the steps of: acquiring first historical statistical subdata at the ith node level; acquiring second historical statistical subdata at an i +1 th node level, wherein the i +1 th node level is higher than the i +1 th node level; i is a positive integer greater than 1; according to a preset period, updating the first historical statistical subdata of the ith node level to the ith-1 node level, and updating the second historical statistical subdata of the (i + 1) th node level to the ith node level.
Illustratively, referring to fig. 5, every 10 minutes, the historical statistical subdata in the level 2 historical statistical data node is updated to the historical statistical subdata in the level 3 historical statistical data node; and updating the historical statistic subdata in the level 1 historical statistic data node into the historical statistic subdata in the level 2 historical statistic data node.
2. Periodic class historical statistics data structure
Optionally, the historical statistics structure includes a header and at least one window node.
Schematically, referring to fig. 6, the head (head) includes: step _ type, time _ zone, node _ num, last _ ts, and top _ idx (top node number). Wherein, the step type indicates the statistical period of each node (window node), for example: if the step type is month, the running data generated in one month is counted by one window node; the statistical time zone is used for indicating a world time zone; the window node number refers to the number of historical statistical data which can be stored in the historical statistical data structure and is used for determining the space size of the historical statistical data structure; the latest timestamp refers to a timestamp corresponding to the latest updated historical statistical data; the top node sequence number indicates the window node represented by the historical statistics structure, for example: and the top node serial number is 3, the window node represented by the historical statistical data structure is node3, and the sliding of the window in the historical statistical data structure can be realized by changing the top node serial number.
Each window node comprises: num (total number of records), value (total count), status (data state). When the step type is realized as a month, the total record number refers to the number of target data counted in one month; the total statistic is the total statistic value of one month; the data status is used to indicate whether the window node is storing valid data or invalid data.
Optionally, before combining the target data with the historical statistical data stored in the first storage space to obtain the target statistical data, the historical statistical data associated with the target data needs to be queried in the first storage space.
Optionally, in response to querying the historical statistical data associated with the target data, when the target data is data after rollback, the historical statistical data needs to be rolled back.
Optionally, the rollback processing means that, in response to the corresponding timestamp in the historical statistical data being greater than the timestamp of the current target data, the status information corresponding to the historical statistical data is set to be invalid.
Schematically, (1) as shown in fig. 5, when the historical statistical data structure is implemented as the aperiodic historical statistical data structure, the data state of the historical statistical sub-data in the level 3 historical statistical data node is first set to be invalid; and then sequentially checking the timestamps of the historical statistical subdata in the level 2 and level 1 historical statistical data nodes, and if the last timestamp is larger than the timestamp of the current target data, setting the data state of the historical statistical subdata in the level 2 and level 1 historical statistical data nodes as invalid. (2) As shown in fig. 6, when the historical statistic data structure is implemented as a period-class historical statistic data structure, if the latest timestamp stored in the header is greater than the timestamp of the current target data, a window node is found according to the top node sequence number, and the data state in the window node is set to be invalid.
And 406, updating and storing the target statistical data into the first storage space.
The method for storing the target statistical data has already been described in step 205, and is not described herein.
The first storage space is used for storing the statistical data for inquiry.
In some alternative embodiments, the query requester may send a statistical query request requesting for query statistics, and the step of querying the statistics includes at least one of:
1. receiving a statistical data query request; performing data query on the first storage space based on the statistical data query request; and responding to the statistical data corresponding to the statistical data query request queried in the first storage space, and feeding back the statistical data.
2. Receiving a statistical data query request; performing data query on the first storage space based on the statistical data query request; and responding to the fact that the statistical data corresponding to the statistical data query request cannot be queried in the first storage space, statistically querying a statistical result in the database according to the data query request, writing the statistical result into a cache, and feeding back the statistical result.
3. Receiving a statistical data query request; performing data query on the first storage space based on the statistical data query request; in response to the statistical data corresponding to the statistical data query request is queried in the first storage space, and the state information in the statistical data indicates that the statistical data is invalid, the statistical result is queried in the database in a statistical manner according to the data query request and written into the cache, and the statistical result is fed back.
Optionally, the invalid statistical data needs to be repaired, illustratively, when the statistical data structure is implemented as a non-periodic statistical data structure, when the statistical data is repaired, a statistical result is statistically queried in the database according to the data query request, and the statistical result is written into the statistical data node with the highest level in the non-periodic statistical data structure to complete the repair of the statistical data.
In summary, in the embodiment of the present application, the target data is directly obtained by analyzing the log file of the database, the target data is counted according to the statistical configuration information, and the statistical result is stored in advance, so that only the data with the query requirement can be counted, the storage resource of the computer is saved, the query efficiency is improved, and the query delay is reduced by storing the statistical result in advance.
The embodiment of the application designs a data structure of multi-level nodes, and when the target data is abnormal in pushing, repeated statistics is avoided through strict incremental check and rollback operation of the timestamp, so that storage resources are saved, and the working efficiency is improved.
Fig. 7 is a schematic diagram of an operation of a data statistics method according to an exemplary embodiment of the present application, and the operation principle in fig. 7 is described according to a business process, as shown in fig. 7:
step one, writing the flow data generated by the service terminal into a flow database through a service background.
Fig. 7 shows three pipeline databases, wherein the service terminal 711 is connected to the pipeline database 712, that is, pipeline data generated in the service terminal 711 is written into the pipeline database 712, the service terminal 713 is connected to the pipeline database 714, that is, pipeline data generated in the service terminal 713 is written into the pipeline database 714, and the service terminal 715 is connected to the pipeline database 716, that is, pipeline data generated in the service terminal 715 is written into the pipeline database 716.
Step two, the Binlog parser 717 acquires target data in real time by directly listening and parsing the Binlog logs in the pipeline database 712, the pipeline database 714 and the pipeline database 716.
And step three, the Binlog parser 717 writes the obtained target data into an Apache Kafka message queue for caching.
Step four, the real-time pipeline consumption service 718 reads the target data by subscribing Apache Kafka, serially processes the target data with the same object identifier according to the hash route, and distributes the target data to the data statistics write service 719 for statistical processing.
After receiving the target data pushed by the real-time pipeline consumption service 718, the data statistics writing service 719 firstly queries corresponding historical statistics data from the statistics data cache 720, if the historical statistics data exists and the state is normal, accumulates the value of the current target data into the historical statistics data to obtain target statistics data, and then writes the target statistics data back into the statistics data cache 720; if the historical statistical data is not found or found to be invalid, the processing needs to be handled in two cases according to the statistical requirement information in the service statistical configuration 721:
the first condition is as follows: the statistical demand information indicates that the current target data is directly discarded without performing statistics on the target data;
case two: the statistical requirement information indicates that the target data needs to be counted, and the result statistically queried from the pipeline database according to the service statistical configuration 721 is written into the statistical data cache 720.
It is noted that if the current historical statistical data state is normal, but the timestamp of the target data is earlier than the timestamp stored in the historical statistical data, it may be that the current historical statistical data state needs to be set to invalid because duplicate target data is pushed.
The first to fifth steps are basic processes of statistical processing of the running data generated when the service terminal runs the target application program, that is, a pre-statistical process of the target data. In the process of performing target data pre-statistics, the data statistics reading service 722 is responsible for processing the statistics query request of the business side.
After receiving the service statistics query request, the data statistics reading service 722 will try to read the historical statistics data from the statistics data cache 720 according to the request parameters, and if the historical statistics data is read and the historical statistics data is in a normal state, directly return the result to the requester; if the historical statistical data is not read, the data statistics writing service 719 is requested to complete the first statistics, at this time, the data statistics writing service 719 needs to statistically query a result from the pipeline database according to the service statistics configuration 721 and write the result into the statistical data cache 720, and then the data statistics reading service 722 returns the result of the first statistics to the requesting party; if the read historical statistical data is invalid, the data statistics writing service 719 is requested to repair the historical statistical data, and the repaired statistical data is returned to the requester.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Referring to fig. 8, a block diagram of a data statistics apparatus provided in an exemplary embodiment of the present application is shown. The device has the function of realizing the data statistical method, and the function can be realized by hardware or by hardware executing corresponding software. The device can be a computer device and can also be arranged in the computer device. The apparatus 800 may include:
an obtaining module 810, configured to obtain a log file of a database, where the log file is used to record data update of the database;
an analysis module 820, configured to analyze the log file to obtain target data in the log file, where the target data is data that is updated in the log file;
the obtaining module 810 is further configured to obtain statistical configuration information, where the statistical configuration information is a preset data statistical scheme;
a processing module 830, configured to combine the target data with historical statistical data stored in the first storage space based on the statistical configuration information to obtain target statistical data;
the processing module 830 is further configured to update and store the target statistical data into the first storage space, where the first storage space is used to pre-store statistical data for querying.
Referring to fig. 9, in an exemplary embodiment, the obtaining module 810 includes:
a matching sub-module 811 for matching the target data with the historical statistical data of the first storage space;
the obtaining module 810 is further configured to obtain the statistical configuration information in response to a failure in matching the target data with the historical statistical data.
In an exemplary embodiment, the target data corresponds to a first object identifier and first statistical item information, and the historical statistical data corresponds to a second object identifier and second statistical item information; the matching sub-module 811 is further configured to match the first object identifier with the second object identifier; and matching the first statistical item information with the second statistical item information.
Optionally, the obtaining module 810 is further configured to respond to that matching between the first object identifier and the second object identifier fails; or, the first statistical item information and the second statistical item information are failed to be matched, and the statistical configuration information is obtained.
In an exemplary embodiment, the historical statistical data corresponds to status information indicating validity of the historical statistical data; the obtaining module 810 is further configured to obtain the statistical configuration information in response to that the status information corresponding to the historical statistical data is used to indicate that the historical statistical data is invalid.
In an exemplary embodiment, the processing module 830 is further configured to combine the target data with the historical statistical data in a designated combination manner to obtain the target statistical data in response to a successful matching between the target data and the historical statistical data.
In an exemplary embodiment, the historical statistical data includes at least one historical statistical data node corresponding to at least one historical statistical subdata, and the historical statistical data node corresponds to a node level; the obtaining module 810 is further configured to obtain first historical statistical subdata at an ith node level; acquiring second historical statistical subdata at an i +1 th node level, wherein the i +1 th node level is higher than the i +1 th node level; i is a positive integer greater than 1.
Optionally, the apparatus 800 further includes:
an updating module 840, configured to update the first historical statistical subdata of the ith node level to the (i-1) th node level and update the second historical statistical subdata of the (i + 1) th node level to the ith node level according to a preset period.
In an exemplary embodiment, the apparatus 800 further comprises:
a writing module 850, configured to write the target data into a data statistics writing service, where the data statistics writing service is used to perform caching before statistics on the target data.
In an exemplary embodiment, the writing module 850 is further configured to write the target data into a rollback queue in response to the target data being written into the statistics write service failing.
Optionally, the writing module 850 includes:
a rollback sub-module 851 for sequentially rolling back the data to be rolled back from the rollback queue to the data statistics writing service;
a deleting submodule 852, configured to delete the target data from the rollback queue in response to a rollback success of the target data.
In an exemplary embodiment, the obtaining module 810 further includes:
a monitoring sub-module 812, configured to monitor the log file;
the processing sub-module 813 is configured to, in response to the content update of the log file, analyze the log file to obtain the target data in the log file.
In an exemplary embodiment, the database includes a pipelined database corresponding to at least two business servers.
In an exemplary embodiment, the apparatus 800 further comprises:
a receiving module 860, configured to receive a statistical data query request;
a query module 870, configured to perform a data query on the first storage space based on the statistical data query request;
a feedback module 880, configured to perform feedback on the statistical data in response to querying the statistical data corresponding to the statistical data query request in the first storage space.
To sum up, in the embodiment of the present application, the target data is directly obtained by analyzing the log file of the database, the target data is counted according to the statistical configuration information, and the statistical result is stored in advance, so that only data with query requirements can be counted, the storage resource of the computer is saved, the query efficiency is improved, and the query delay is reduced by storing the statistical result in advance.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, the division of each functional module is merely used as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the content structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the data statistics apparatus provided in the foregoing embodiment and the data statistics method embodiment belong to the same concept, and specific implementation processes thereof are described in the method embodiment, and are not described herein again.
In an exemplary embodiment, there is also provided a computer device comprising a processor and a memory, the memory having stored therein a computer program, the computer program being loaded and executed by the processor to implement the above data statistics method.
In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, the computer program being loaded and executed by a processor to implement the above data statistical method. Alternatively, the computer-readable storage medium may be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product comprising computer instructions stored in a computer readable storage medium, from which a processor reads and executes the computer instructions to implement the above data statistical method.
It should be understood that reference herein to "a plurality" means two or more. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.
The above description is only exemplary of the present application and is not intended to limit the present application, and any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (15)

1. A method of data statistics, the method comprising:
acquiring a log file of a database, wherein the log file is used for recording data updating of the database;
analyzing the log file to obtain target data in the log file, wherein the target data is updated data in the log file;
acquiring statistical configuration information, wherein the statistical configuration information is a preset data statistical scheme;
combining the target data with historical statistical data stored in a first storage space based on the statistical configuration information to obtain target statistical data;
and updating and storing the target statistical data into the first storage space, wherein the first storage space is used for storing the statistical data for inquiry.
2. The method of claim 1, wherein the obtaining statistical configuration information comprises:
matching the target data with the historical statistical data of the first storage space;
and responding to the failure of matching between the target data and the historical statistical data, and acquiring the statistical configuration information.
3. The method of claim 2, wherein the target data corresponds to a first object identifier and first statistical item information, and the historical statistical data corresponds to a second object identifier and second statistical item information;
the matching the target data with the historical statistical data of the first storage space comprises:
matching the first object identification with the second object identification; matching the first statistical item information with the second statistical item information;
the obtaining the statistical configuration information in response to a failure in matching the target data with the historical statistical data includes:
in response to a failure of the first object identification to match the second object identification; or, the first statistical item information and the second statistical item information are failed to be matched, and the statistical configuration information is obtained.
4. The method of claim 2, wherein the historical statistics correspond to status information indicating validity of the historical statistics;
the obtaining the statistical configuration information in response to a failure in matching the target data with the historical statistical data includes:
and responding to the state information corresponding to the historical statistical data to indicate that the historical statistical data is invalid, and acquiring the statistical configuration information.
5. The method of any of claims 2 to 4, further comprising:
and in response to the successful matching of the target data and the historical statistical data, combining the target data and the historical statistical data in a specified combination mode to obtain the target statistical data.
6. The method of any of claims 1 to 4, wherein the historical statistical data comprises at least one historical statistical data node corresponding to at least one historical statistical subdata, the historical statistical data node corresponding to a node level;
the method further comprises the following steps:
acquiring first historical statistical subdata at the ith node level; acquiring second historical statistical subdata at an i +1 th node level, wherein the i +1 th node level is higher than the i +1 th node level; i is a positive integer greater than 1;
according to a preset period, updating the first historical statistical subdata of the ith node level to the (i-1) th node level, and updating the second historical statistical subdata of the (i + 1) th node level to the ith node level.
7. The method according to any one of claims 1 to 4, wherein after parsing the log file and obtaining the target data in the log file, the method further comprises:
and writing the target data into a data statistics writing service, wherein the data statistics writing service is used for caching the target data before statistics.
8. The method of claim 7, wherein the writing the target data to a data statistics write service comprises:
writing the target data into a rollback queue in response to the failure of the target data to write to the statistics write service;
sequentially rolling back the data to be rolled back to the data statistics writing service from the rolling back queue;
in response to the target data rollback being successful, deleting the target data from the rollback queue.
9. The method according to any one of claims 1 to 4, wherein the parsing the log file to obtain the target data in the log file comprises:
monitoring the log file;
and responding to the content update of the log file, and analyzing the log file to obtain the target data in the log file.
10. The method according to any one of claims 1 to 4,
the database comprises a flow database corresponding to at least two business servers.
11. The method of any of claims 1 to 4, further comprising:
receiving a statistical data query request;
performing data query on the first storage space based on the statistical data query request;
responding to the statistical data corresponding to the statistical data query request queried in the first storage space, and feeding back the statistical data.
12. A data statistics apparatus, characterized in that the apparatus comprises:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a log file of a database, and the log file is used for recording data updating of the database;
the analysis module is used for analyzing the log file to obtain target data in the log file, wherein the target data is updated data in the log file;
the acquisition module is further configured to acquire statistical configuration information, where the statistical configuration information is a preset data statistical scheme;
the processing module is used for combining the target data with historical statistical data stored in a first storage space based on the statistical configuration information to obtain target statistical data;
the processing module is further configured to update and store the target statistical data into the first storage space, where the first storage space is used to store statistical data for querying.
13. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement a data statistics method as claimed in any of claims 1 to 11.
14. A computer-readable storage medium, having at least one program code stored therein, the program code being loaded and executed by a processor to implement the data statistics method of any one of claims 1 to 11.
15. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the data statistics method of any of claims 1 to 11.
CN202210262152.3A 2022-03-16 2022-03-16 Data statistical method, device, equipment, storage medium and program product Pending CN114722078A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210262152.3A CN114722078A (en) 2022-03-16 2022-03-16 Data statistical method, device, equipment, storage medium and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210262152.3A CN114722078A (en) 2022-03-16 2022-03-16 Data statistical method, device, equipment, storage medium and program product

Publications (1)

Publication Number Publication Date
CN114722078A true CN114722078A (en) 2022-07-08

Family

ID=82237363

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210262152.3A Pending CN114722078A (en) 2022-03-16 2022-03-16 Data statistical method, device, equipment, storage medium and program product

Country Status (1)

Country Link
CN (1) CN114722078A (en)

Similar Documents

Publication Publication Date Title
CN111125260A (en) Data synchronization method and system based on SQL Server
CN108228322B (en) Distributed link tracking and analyzing method, server and global scheduler
CN111309550A (en) Data acquisition method, system, equipment and storage medium of application program
CN112559475B (en) Data real-time capturing and transmitting method and system
CN112506743A (en) Log monitoring method and device and server
CN111030888B (en) Domain name system DNS capacity measuring method, device, equipment and medium
CN112506870B (en) Data warehouse increment updating method and device and computer equipment
CN110780882B (en) Method, device, system, electronic equipment and storage medium for processing code file
CN110912757B (en) Service monitoring method and server
US11650989B2 (en) Efficient aggregation of time series data
US20120323924A1 (en) Method and system for a multiple database repository
CN112632129A (en) Code stream data management method, device and storage medium
CN111984495A (en) Big data monitoring method and device and storage medium
US10346281B2 (en) Obtaining and analyzing a reduced metric data set
CN116737765A (en) Service alarm information processing method and device, electronic equipment and storage medium
CN110795026B (en) Hot spot data identification method, device, equipment and storage medium
CN110309206B (en) Order information acquisition method and system
CN104317820B (en) Statistical method and device for report forms
CN114722078A (en) Data statistical method, device, equipment, storage medium and program product
CN116109322A (en) Data acquisition method, data acquisition device, and computer-readable storage medium
CN112764988B (en) Data segment acquisition method and device
CN115344633A (en) Data processing method, device, equipment and storage medium
CN114579416A (en) Index determination method, device, server and medium
CN114356712A (en) Data processing method, device, equipment, readable storage medium and program product
CN112965793A (en) Data warehouse task scheduling method and system oriented to identification analysis data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination