CN113268487B - Data statistical method, device and computer readable storage medium - Google Patents
Data statistical method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN113268487B CN113268487B CN202110669906.2A CN202110669906A CN113268487B CN 113268487 B CN113268487 B CN 113268487B CN 202110669906 A CN202110669906 A CN 202110669906A CN 113268487 B CN113268487 B CN 113268487B
- Authority
- CN
- China
- Prior art keywords
- data
- statistics
- database
- task
- data statistics
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/221—Column-oriented storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24558—Binary matching operations
- G06F16/2456—Join operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a data statistical method, a device and a computer readable storage medium, wherein the data statistical method comprises the following steps: determining data dimensions of source data, and establishing data indexes corresponding to the data dimensions; when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index; and determining a data statistical result according to the target source data, and storing the data statistical result in a database. The invention can improve the data statistics speed.
Description
Technical Field
The present invention relates to the field of data statistics, and in particular, to a data statistics method, an apparatus, and a computer-readable storage medium.
Background
The conventional data statistical scheme is that data statistical analysis is performed in a mode of generating a statistical task by a plurality of statistical reports based on a preset statistical page, complex task parameter configuration needs to be performed on a plurality of servers under the condition that the number of servers is large, the task execution time depends heavily on the generation time of the statistical task, server resources are easily occupied for a long time under the condition that the link of generating the statistical task is delayed, the data volume required to be traversed by data statistics is large each time, the statistical task cannot be completed in time, and the data statistics speed is low.
Disclosure of Invention
The invention mainly aims to provide a data statistical method, a data statistical device and a computer readable storage medium, aiming at improving the speed of data statistics.
In order to achieve the above object, the present invention provides a data statistical method, including:
determining data dimensions of source data, and establishing data indexes corresponding to the data dimensions;
when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and determining a data statistical result according to the target source data, and storing the data statistical result in a database.
In an embodiment, before the step of detecting that the data statistics task is not executed, searching the source data for the target source data corresponding to the data statistics task according to the data index further includes:
judging whether the database is in an idle state or not;
and when the database is in an idle state, acquiring the data statistics task.
In one embodiment, the step of determining whether the database is in an idle state includes:
acquiring a data request of the current connection of the database;
when the number of the data requests connected currently is smaller than or equal to a preset number, determining that the database is in an idle state;
and when the number of the data requests which are currently connected is larger than a preset number, determining that the database is not in an idle state.
In an embodiment, before the step of searching the source data for the target source data corresponding to the data statistics task according to the data index when the data statistics task is detected not to be executed, the method further includes:
judging whether a data statistics result of the data statistics task exists in the database or not;
and when the data statistics result of the data statistics task does not exist in the database, determining that the data statistics task is not executed, and executing the step of searching the target source data corresponding to the data statistics task in the source data according to the data index.
In an embodiment, the step of determining whether the data statistics result of the data statistics task exists in the database further includes:
and when the data statistics result of the data statistics task exists in the database, determining that the data statistics task is executed, and ending the data statistics task.
In an embodiment, after the step of determining data dimensions of source data and establishing data indexes corresponding to the data dimensions, the method further includes:
acquiring the data statistics task at regular time;
when the data statistics task is detected not to be executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and determining a data statistical result according to the target source data, and storing the data statistical result in a database.
In an embodiment, before the step of searching for the target source data corresponding to the data statistics task in the source data according to the data index, the method further includes:
acquiring a data statistics interface corresponding to the data statistics task;
and calling the data statistics interface, and executing the step of searching the target source data corresponding to the data statistics task in the source data according to the data index.
In addition, to achieve the above object, the present invention also provides a data statistic device, including:
the determining module is used for determining data dimensions of source data and establishing data indexes corresponding to the data dimensions;
the searching module is used for searching target source data corresponding to the data statistics task in the source data according to the data index when the data statistics task is detected not to be executed;
and the storage module is used for determining a data statistical result according to the target source data and storing the data statistical result to a database.
In addition, in order to achieve the above object, the present invention further provides a data statistics apparatus, which includes a memory, a processor, and a data statistics program stored on the memory and operable on the processor, wherein the data statistics program, when executed by the processor, implements the steps of the data statistics method according to any one of the above aspects.
In addition, to achieve the above object, the present invention further provides a computer readable storage medium, on which a data statistics program is stored, the data statistics program, when executed by a processor, implementing the steps of the data statistics method according to any one of the above.
The invention provides a data statistical method, a data statistical device and a computer readable storage medium. According to the scheme, the data dimension division is carried out on the source data, the data index is established, and data statistics is carried out according to the established data index when the data statistics task is not executed, so that single data statistics is effectively reduced, the pressure of a database and a server is reduced, the data statistics time is shortened, and the data statistics speed is increased.
Drawings
FIG. 1 is a diagram illustrating a hardware architecture of a data statistics apparatus according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of a data statistics method according to the present invention;
FIG. 3 is a flow chart illustrating a data statistics method according to a second embodiment of the present invention;
FIG. 4 is a flow chart illustrating a data statistics method according to a third embodiment of the present invention;
fig. 5 is a block diagram of a data statistics apparatus according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: determining data dimensions of source data, and establishing data indexes corresponding to the data dimensions; when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index; and determining a data statistical result according to the target source data, and storing the data statistical result in a database. According to the scheme, the data dimension division is carried out on the source data, the data indexes are established, and data statistics is carried out according to the established data indexes when the data statistics task is not executed, so that single data statistics is effectively reduced, the pressure of a database and a server is reduced, the data statistics time is shortened, and the data statistics speed is increased.
As an implementation solution, referring to fig. 1, fig. 1 is a schematic diagram of a hardware architecture of a data statistics apparatus according to an embodiment of the present invention, as shown in fig. 1, the data statistics apparatus may include a processor 101, for example, a CPU, a memory 102, and a communication bus 103. Wherein a communication bus 103 is used for enabling the connection communication between these components.
The memory 102 may be a high-speed RAM memory or a non-volatile memory (e.g., a disk memory). As shown in fig. 1, a data statistics program may be included in a memory 102 as a computer-readable storage medium; and the processor 101 may be configured to invoke the data statistics program stored in the memory 102 and perform the following operations:
determining data dimensions of source data, and establishing data indexes corresponding to the data dimensions;
when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and determining a data statistical result according to the target source data, and storing the data statistical result in a database.
In one embodiment, the processor 101 may be configured to invoke a data statistics program stored in the memory 102 and perform the following operations:
judging whether the database is in an idle state or not;
and when the database is in an idle state, acquiring the data statistics task.
In one embodiment, the processor 101 may be configured to invoke a data statistics program stored in the memory 102 and perform the following operations:
acquiring a data request of the current connection of the database;
when the number of the data requests connected currently is smaller than or equal to a preset number, determining that the database is in an idle state;
and when the number of the data requests which are currently connected is larger than a preset number, determining that the database is not in an idle state.
In one embodiment, the processor 101 may be configured to invoke a data statistics program stored in the memory 102 and perform the following operations:
judging whether a data statistics result of the data statistics task exists in the database or not;
and when the data statistics result of the data statistics task does not exist in the database, determining that the data statistics task is not executed, and executing the step of searching the target source data corresponding to the data statistics task in the source data according to the data index.
In one embodiment, the processor 101 may be configured to invoke a data statistics program stored in the memory 102 and perform the following operations:
and when the data statistics result of the data statistics task exists in the database, determining that the data statistics task is executed, and ending the data statistics task.
In one embodiment, the processor 101 may be configured to invoke a data statistics program stored in the memory 102 and perform the following operations:
acquiring the data statistics task at regular time;
when the data statistics task is detected not to be executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and determining a data statistical result according to the target source data, and storing the data statistical result in a database.
In one embodiment, the processor 101 may be configured to invoke a data statistics program stored in the memory 102 and perform the following operations:
acquiring a data statistics interface corresponding to the data statistics task;
and calling the data statistics interface, and executing the step of searching the target source data corresponding to the data statistics task in the source data according to the data index.
Based on the hardware architecture of the data statistical device, the embodiment of the data statistical method is provided.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data statistical method of the present invention, where the data statistical method includes:
step S10, determining data dimensions of source data, and establishing data indexes corresponding to the data dimensions;
in this embodiment, the existing data statistics method that receives statistical parameters for a business project statistical report through a preset statistical page and submits the statistical parameters to a designated computing cluster after a statistical task is generated to perform a data statistical task depends heavily on the generation time of the statistical task, and is prone to occupy database and server resources for a long time under the condition that the generation of the statistical task is delayed, and in data statistics, the data volume of single traversal is large, so that the time of data statistics is long, and the statistical speed is slow.
Based on the defects of the existing data statistical method, the invention provides a data statistical method, which is characterized in that data dimension is divided for data, a proper data index is established for the divided data, data statistics is introduced through the data index, the data volume of single traversal during data statistics is reduced, and meanwhile, the pressure of a database and a server during data statistics is reduced, so that the data statistical speed is increased.
In this embodiment, an execution subject of the data statistics method of the present invention is a data statistics apparatus, where the data statistics apparatus refers to a device or apparatus that can execute a data statistics task, that is, a device or apparatus with a data statistics function, the data statistics apparatus may be a terminal device, for example, the data statistics apparatus may be a PC or a portable computer, etc., and the data statistics apparatus may also be a server, and of course, in other embodiments, the data statistics apparatus may be determined according to actual needs, which is not limited in this embodiment.
In this embodiment, the data statistics apparatus obtains source data, determines data dimensions of the source data, and establishes data indexes corresponding to the data dimensions, where the source data refers to original data generated by the data statistics apparatus itself or other devices connected to the data statistics apparatus, for example, the source data may be data generated when a device video is loaded or data loaded when a device alarm is pushed, and the source data may also be dotting data generated when the device performs other related operations, which is not limited in this embodiment.
In this embodiment, the data dimension refers to a data type or attribute of the source data, for example, the data dimension may be an event, a date, a time period, a province, a city, a device type, and the like, and the source data includes data of one or more data dimensions. The data index refers to a single and physical storage structure for sorting data, and represents a collection of a certain type of data, for example, for a data dimension date, a first-level index date, a second-level index year, a third-level index month and a fourth-level index day can be established under the date. It should be noted that each data dimension may establish a corresponding data index, which is only some optional embodiments listed in this embodiment, and the specific way of establishing the data index may be determined according to actual needs, which is not limited in this embodiment.
Specifically, the data statistics device acquires source data, analyzes the acquired source data to obtain a field of the source data, determines a data dimension of the source data according to the type of the field, and then establishes a data index of each data dimension.
Step S20, when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
in this embodiment, when detecting that the data statistics task is not executed, the data statistics apparatus searches target source data corresponding to the data statistics task in the source data according to the data index, where the data statistics task refers to a statistics operation or an instruction executed by a user, the statistics task may be data for counting dimensions of each data, for example, the statistics task may be data for counting a certain year/month/day or data for counting a certain event, and the target source data refers to data corresponding to the data statistics task, for example, when the data statistics task is data for counting a certain year/month/day, the target source data is data for a certain year/month/day in the source data, and when the data statistics task is data for counting a certain event, the target source data is data for a certain event in the source data.
In this embodiment, the data statistics apparatus performs data dimension division on the source data, establishes a data index for each data dimension, obtains a data statistics task, and detects whether the data statistics task has been executed.
Specifically, the data statistics device judges whether a data statistics result of the data statistics task exists in the database after acquiring the data statistics task, determines that the data statistics task is not executed when the data statistics result of the data statistics task does not exist in the database, acquires a data statistics interface corresponding to the data statistics task when the data statistics task is determined not to be executed, and calls the data statistics interface to search target source data corresponding to the data statistics task in source data according to a data index; and when the data statistics result of the data statistics task exists in the database, determining that the data statistics task is executed, and ending the data statistics task when the data statistics task is determined to be executed.
In this embodiment, after the data statistics device obtains the data statistics task, by detecting whether the data statistics task has been executed or not, and when the data statistics task has been executed, the data statistics device searches for target source data corresponding to the data statistics task in the source data according to the data index, so that repeated statistics under the condition that the data statistics task has been executed is avoided, and the efficiency of data statistics is improved.
And S30, determining a data statistical result according to the target source data, and storing the data statistical result into a database.
In this embodiment, after the data statistics apparatus finds the target source data according to the data index, the data statistics apparatus generates a data statistics result according to the target source data, and stores the data statistics result in the data. The data statistical result is stored in the form of the data statistical result table in the database, and the data statistical result is stored in the form of the data table, so that the screening of multi-dimensional data can be met, and the terminal equipment can conveniently call statistics and results of specified statistical indexes. Of course, in other embodiments, the form of storing the data statistics result in the database may be determined according to actual situations, and this embodiment does not limit this.
In the technical scheme provided by this embodiment, by determining the data dimension of the source data and establishing the data index corresponding to each data dimension, when it is detected that the data statistics task is not executed, target source data corresponding to the data statistics task is searched in the source data according to the data index, a data statistics result is determined according to the target source data, and the data statistics result is stored in the database. According to the scheme, the data dimension division is carried out on the source data, the data index is established, and data statistics is carried out according to the established data index when the data statistics task is not executed, so that single data statistics is effectively reduced, the pressure of a database and a server is reduced, the data statistics time is shortened, and the data statistics speed is increased.
Referring to fig. 3, fig. 3 is a schematic flowchart of a second embodiment of the data statistics method of the present invention, based on the first embodiment, before the step S20, the method further includes:
step S40, judging whether the database is in an idle state;
in this embodiment, the data statistics apparatus determines whether the database is in an idle state, where the idle state of the database refers to a state where the database or the server resource is relatively sufficient, and generally. When the database receives fewer data requests, the resources of the database are sufficient, and the database can be considered to be in an idle state.
Specifically, the data statistics device obtains the data requests of the current connection of the database after performing data dimension division on the source data and establishing a data index, calculates the number of the data requests of the current connection of the database, and determines that the database is in an idle state when the number of the data requests of the current connection of the database is less than or equal to a preset number; when the number of the data requests currently connected to the database is greater than the preset number, it is determined that the database is not in an idle state, where the preset number may be set according to an actual situation, and this embodiment does not limit this, and the preset number may be 100.
And S50, acquiring the data statistics task when the database is in an idle state.
In this embodiment, when the data statistics device determines that the database is in an idle state, the data statistics task is obtained, so that when the data statistics task is not executed, data corresponding to the data statistics task is searched in the source data according to the data index to obtain target source data, and then a data statistics result is generated according to the target source data and stored in a data statistics result table in the database.
In the technical scheme provided by this embodiment, the data statistics apparatus obtains the data statistics task by determining whether the database is in an idle state or not, when the database is in the idle state. According to the scheme, whether the database is in the idle state or not is judged, the data statistics task is obtained when the database is in the idle state, whether the data statistics task is executed or not is judged, and then the data statistics task is executed when the data statistics task is not executed, so that the statistics task is executed only when the database is in the idle state, the pressure of the database and a server is effectively reduced, and meanwhile the data statistics speed is improved.
Referring to fig. 4, fig. 4 is a schematic flowchart of a third embodiment of the data statistics method of the present invention, and based on the first embodiment, after the step S10, the method further includes:
step S60, acquiring the data statistics task regularly;
step S70, when detecting that the data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and S80, determining a data statistical result according to the target source data, and storing the data statistical result in a database.
In this embodiment, the data statistics device acquires the data statistics task at regular time, and the data statistics task may be triggered at regular time, for example, the data statistics task is triggered at every preset time length, where the preset time length may be set according to an actual situation, and the preset time length may be selected as 2 hours, which is not limited in this embodiment.
The data statistics device regularly acquires the data statistics task, detects whether the data statistics task is executed or not, searches target source data corresponding to the data statistics task in the source data according to the data index when the data statistics task is not executed, determines a data statistics result according to the target source data, and stores the data statistics result in the database. The contents of the embodiments may be referred to specifically, and the details of the embodiments are not repeated herein.
In an application scene, if the database is not in an idle state all the time, so that the data statistics task cannot be executed, the data statistics task can be effectively prevented from being executed by acquiring the data statistics task regularly and executing the data statistics task. The priority of the regularly executed data statistics task is higher than that of the database in an idle state, in other words, when the regularly executed data statistics task is triggered, the database still needs to execute the data statistics task preferentially even if the database is still not in the idle state.
For example, if the data statistics task is set to be executed once every two hours, assuming that the first data statistics task is executed as 00. It can be understood that if the database has an idle state in the time range of 00-02, the data statistics task is automatically executed when the database is in the idle state, and when reaching 02.
In the technical scheme provided by this embodiment, the data statistics task is obtained at regular time, when it is detected that the data statistics task is not executed, the target source data corresponding to the data statistics task is searched in the source data according to the data index, the data statistics result is determined according to the target source data, and the data statistics result is stored in the database. The scheme combines timing statistics with dynamic idle-time statistics, further reduces the data statistics pressure of a database, improves the data statistics speed, can perform data statistics on the timing statistics task when the dynamic idle-time statistics is abnormal, and ensures the stability of the data statistics task.
Referring to fig. 5, the present invention also provides a data statistics apparatus, including:
a determining module 100, configured to determine data dimensions of source data, and establish a data index corresponding to each data dimension;
the searching module 200 is configured to search, when it is detected that a data statistics task is not executed, target source data corresponding to the data statistics task in the source data according to the data index;
the storage module 300 is configured to determine a data statistics result according to the target source data, and store the data statistics result in a database.
In one embodiment, the data statistics apparatus further comprises:
a judging module 400, configured to judge whether the database is in an idle state;
an obtaining module 500, configured to obtain the data statistics task when the database is in an idle state.
In an embodiment, in terms of determining whether the database is in an idle state, the determining module 400 is specifically applied to:
acquiring a data request of the current connection of the database;
when the number of the data requests connected currently is smaller than or equal to a preset number, determining that the database is in an idle state;
and when the number of the data requests which are currently connected is larger than a preset number, determining that the database is not in an idle state.
In an embodiment, the determining module 400 is specifically applied to:
judging whether a data statistical result of the data statistical task exists in the database or not;
and when the data statistics result of the data statistics task does not exist in the database, determining that the data statistics task is not executed.
In an embodiment, the determining module 400 is specifically applied to:
and when the data statistics result of the data statistics task exists in the database, determining that the data statistics task is executed, and ending the data statistics task.
In an embodiment, the obtaining module 500 is specifically applied to:
and acquiring the data statistics task at fixed time.
In an embodiment, the obtaining module 500 is specifically applied to:
and acquiring a data statistics interface corresponding to the data statistics task, and calling the data statistics interface.
Based on the foregoing embodiments, the present invention further provides a data statistics apparatus, where the data statistics apparatus may include a memory, a processor, and a data statistics program stored in the memory and executable on the processor, and when the processor executes the data statistics program, the steps of the data statistics method according to any of the foregoing embodiments are implemented.
Based on the foregoing embodiments, the present invention further provides a computer-readable storage medium, on which a data statistics program is stored, and the data statistics program, when executed by a processor, implements the steps of the data statistics method according to any one of the foregoing embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or system comprising the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a smart tv, a mobile phone, a computer, etc.) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (8)
1. A method of data statistics, the method comprising:
determining data dimensions of source data, and establishing data indexes corresponding to the data dimensions;
judging whether the database is in an idle state or not;
when the database is in an idle state, acquiring a data statistics task, or when the database is not in the idle state, acquiring the data statistics task at regular time;
judging whether a data statistical result of the data statistical task exists in the database or not;
determining that the data statistics task is not executed when the data statistics result of the data statistics task does not exist in the database;
when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and determining a data statistical result according to the target source data, and storing the data statistical result into a data statistical result table in a database, wherein the data statistical result is stored through the data statistical result table so as to meet the screening of multi-dimensional data.
2. The data statistics method of claim 1, wherein the step of determining whether the database is in an idle state comprises:
acquiring a data request of the current connection of the database;
when the number of the data requests connected currently is smaller than or equal to a preset number, determining that the database is in an idle state;
and when the number of the data requests which are currently connected is larger than a preset number, determining that the database is not in an idle state.
3. The data statistics method of claim 1, wherein the step of determining whether the data statistics result of the data statistics task exists in the database further comprises:
and when the data statistics result of the data statistics task exists in the database, determining that the data statistics task is executed, and ending the data statistics task.
4. The data statistics method of claim 1, wherein the step of determining data dimensions of the source data and establishing a data index corresponding to each of the data dimensions further comprises:
acquiring the data statistics task at regular time;
when the data statistics task is detected not to be executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and determining a data statistical result according to the target source data, and storing the data statistical result in a database.
5. The data statistical method of claim 1, wherein the step of searching the source data for the target source data corresponding to the data statistical task according to the data index further comprises:
acquiring a data statistics interface corresponding to the data statistics task;
and calling the data statistics interface, and executing the step of searching the target source data corresponding to the data statistics task in the source data according to the data index.
6. A data statistics apparatus, characterized in that the data statistics apparatus comprises:
the determining module is used for determining data dimensions of source data, establishing data indexes corresponding to the data dimensions, judging whether a database is in an idle state, and acquiring a data statistics task when the database is in the idle state or acquiring the data statistics task at regular time when the database is not in the idle state;
the searching module is used for judging whether a data statistical result of the data statistical task exists in the database or not; determining that the data statistics task is not executed when the data statistics result of the data statistics task does not exist in the database; when detecting that a data statistics task is not executed, searching target source data corresponding to the data statistics task in the source data according to the data index;
and the storage module is used for determining a data statistical result according to the target source data and storing the data statistical result to a database.
7. A statistics device comprising a memory, a processor and a statistics program stored on said memory and executable on said processor, said statistics program when executed by said processor implementing the steps of the statistics method of any of claims 1-5.
8. A computer-readable storage medium, in which a data statistics program is stored, which when executed by a processor implements the steps of the data statistics method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110669906.2A CN113268487B (en) | 2021-06-16 | 2021-06-16 | Data statistical method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110669906.2A CN113268487B (en) | 2021-06-16 | 2021-06-16 | Data statistical method, device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113268487A CN113268487A (en) | 2021-08-17 |
| CN113268487B true CN113268487B (en) | 2023-04-11 |
Family
ID=77235359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110669906.2A Active CN113268487B (en) | 2021-06-16 | 2021-06-16 | Data statistical method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113268487B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9576004B1 (en) * | 2016-01-25 | 2017-02-21 | International Business Machines Corporation | Free space management in databases |
| CN108268546A (en) * | 2016-12-30 | 2018-07-10 | 中国移动通信集团黑龙江有限公司 | A kind of method and device for optimizing database |
| CN108733839A (en) * | 2018-05-25 | 2018-11-02 | 北京神州绿盟信息安全科技股份有限公司 | A kind of statistical method and device of mass data |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209686A (en) * | 2018-02-22 | 2019-09-06 | 北京嘀嘀无限科技发展有限公司 | Storage, querying method and the device of data |
| CN110928851B (en) * | 2019-10-12 | 2023-04-07 | 中国平安财产保险股份有限公司 | Method, device and equipment for processing log information and storage medium |
| CN110909010B (en) * | 2019-11-25 | 2022-09-06 | 杭州晨鹰军泰科技有限公司 | Data intelligent analysis configuration management method, device, equipment and storage medium |
| CN112052259A (en) * | 2020-09-28 | 2020-12-08 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and computer storage medium |
| CN112181993A (en) * | 2020-10-27 | 2021-01-05 | 广州市网星信息技术有限公司 | Service data query method, device, server and storage medium |
| CN112434027A (en) * | 2020-10-30 | 2021-03-02 | 金蝶软件（中国）有限公司 | Indexing method and device for multi-dimensional data, computer equipment and storage medium |
| CN112463800A (en) * | 2020-12-11 | 2021-03-09 | 微医云(杭州)控股有限公司 | Data reading method and device, server and storage medium |
-
2021
- 2021-06-16 CN CN202110669906.2A patent/CN113268487B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9576004B1 (en) * | 2016-01-25 | 2017-02-21 | International Business Machines Corporation | Free space management in databases |
| CN108268546A (en) * | 2016-12-30 | 2018-07-10 | 中国移动通信集团黑龙江有限公司 | A kind of method and device for optimizing database |
| CN108733839A (en) * | 2018-05-25 | 2018-11-02 | 北京神州绿盟信息安全科技股份有限公司 | A kind of statistical method and device of mass data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113268487A (en) | 2021-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104035938A (en) | Performance continuous integration data processing method and device | |
| CN111367925A (en) | Data dynamic real-time updating method, device and storage medium | |
| CN108584598B (en) | Elevator fault automatic analysis and early warning method, storage medium and intelligent terminal | |
| CN109299101B (en) | Data retrieval method, device, server and storage medium | |
| CN108415998B (en) | Application dependency relationship updating method, terminal, device and storage medium | |
| CN110807145A (en) | Query engine acquisition method, device and computer-readable storage medium | |
| CN114900546B (en) | Data processing method, device and equipment and readable storage medium | |
| CN111488736B (en) | Self-learning word segmentation method, device, computer equipment and storage medium | |
| CN107688626B (en) | Slow query log processing method and device and electronic equipment | |
| CN110659294B (en) | Space-time data ad hoc query method, system, electronic device and storage medium | |
| WO2020211363A1 (en) | Method and apparatus for improving efficiency of program loading, computer device and storage medium | |
| CN110704472A (en) | Data query statistical method and device | |
| CN104503983A (en) | Method and device for providing website certification data for search engine | |
| CN108959294B (en) | Method and device for accessing search engine | |
| CN107784043B (en) | Monitoring method, device and system for data table of data warehouse | |
| CN113268487B (en) | Data statistical method, device and computer readable storage medium | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN111221742A (en) | Test case updating method and device, storage medium and server | |
| CN109298928B (en) | Service processing method and device | |
| CN108255704B (en) | Abnormal response method of script calling event and terminal thereof | |
| WO2022253131A1 (en) | Data parsing method and apparatus, computer device, and storage medium | |
| CN110929131A (en) | Data export method, device, equipment and storage medium | |
| CN111159106B (en) | Data query method and device | |
| CN114528313A (en) | Data processing method and device and electronic equipment | |
| CN110851452B (en) | Data table connection processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |