CN115129724A - Statistical report paging method, system, equipment and medium - Google Patents
Statistical report paging method, system, equipment and medium Download PDFInfo
- Publication number
- CN115129724A CN115129724A CN202211037366.7A CN202211037366A CN115129724A CN 115129724 A CN115129724 A CN 115129724A CN 202211037366 A CN202211037366 A CN 202211037366A CN 115129724 A CN115129724 A CN 115129724A
- Authority
- CN
- China
- Prior art keywords
- data
- dimension
- query
- report
- inverted index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000013507 mapping Methods 0.000 claims abstract description 65
- 238000004364 calculation method Methods 0.000 claims abstract description 21
- 238000012163 sequencing technique Methods 0.000 claims abstract description 14
- 230000002776 aggregation Effects 0.000 claims abstract description 13
- 238000004220 aggregation Methods 0.000 claims abstract description 13
- 238000012216 screening Methods 0.000 claims abstract description 4
- 238000004590 computer program Methods 0.000 claims description 15
- 239000002131 composite material Substances 0.000 claims description 7
- 230000005055 memory storage Effects 0.000 claims description 7
- 150000001875 compounds Chemical class 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a statistical report paging method, a statistical report paging system, a statistical report paging device and a statistical report paging medium, wherein the statistical report paging method comprises the following steps: s1, establishing a corresponding data inverted index table according to the dimensionality of each data; s2, acquiring a first query dimension according to dimension information in the query condition acquired by the user side, and querying a first inverted index table corresponding to the first query dimension from the data inverted index table; s3, establishing a nested mapping report nested according to the dimension sequence according to the first inverted index table, and grouping and sequencing the query data; s4, according to the grouping information and the paging information acquired by the user terminal, screening partial access position information from the data position information of the nested mapping report and acquiring partial access data from the nested mapping report; s5 performs aggregation calculation and subtotal on the partial data. By the technical scheme, the problems that memory occupation is suddenly increased and service is unstable when the current report paging method is used for carrying out full statistics can be solved.
Description
Technical Field
The present invention relates to the field of statistical report paging, and more particularly, to a statistical report paging method, system, device, and medium.
Background
The most basic requirement of a report is to find out eligible data and analyze them as a whole.
When the amount of data is large, this process is likely to be very time consuming and even result in call timeouts. When processing large amount of data, especially when the data to be output is also large, paging is a common effective method; for statistical reports, paging has two requirements at the same time:
1) paging requires information on the total and total number of rows to display the total result and to calculate the total number of pages.
2) The grouping of the statistical reports is based on the subtotal of the dimension or the subtotal of the dimension with the hierarchical relationship on different levels.
Whether subtotal, total row count, or aggregate calculations in dimension, are based on all data. If a portion of the data is not captured, the aggregate calculations for the dimensions associated with it may not be correct.
At present, most report paging methods carry out total statistics on data meeting conditions, store results and page; however, this method is very likely to cause the memory usage to be greatly increased, resulting in unstable service.
Disclosure of Invention
In order to solve the technical problems, the invention provides a statistical report paging method, a statistical report paging system, a statistical report paging device and a statistical report paging medium, wherein the statistical report paging method is used for solving the problems that the memory occupation is suddenly increased and the service is unstable when the current report paging method is used for carrying out full-scale statistics.
In order to achieve the above object, the present invention provides a statistical type report paging method, which comprises the following steps:
s1 the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
s2, the database acquires a first query dimension according to the dimension information in the query condition acquired by the user side, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
s3, the database establishes a nested mapping report form nested according to the dimension sequence according to the first inverted index table, and the nested mapping report form is used for grouping and sequencing query data;
s4, the database screens partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user terminal, and acquires partial access data from the nested mapping report;
s5, the database carries out aggregation calculation and subtotal on the partial data and returns the data result to the data service layer.
Further, S1 specifically includes:
s10, the database establishes a corresponding data inverted index table according to the dimensionality of each data and an inverted index mode based on memory storage, and the record value in the data inverted index table is the memory storage address of the data.
Further, S3 specifically includes:
s31, the database acquires all data of the first query dimension and sorts the data according to a preset sequence;
s32, the database generates a composite query condition according to the query condition and the nth data of the first query dimension, and acquires all data corresponding to the composite query condition; wherein n is a positive integer;
s33, constructing all data corresponding to the compound query condition to generate the nested mapping report.
Further, after S5, the method further includes:
s51, the database respectively obtains the resultSize values of the 1 st to nth data of the first query dimension, and carries out summation calculation; the resultSize value is the sum of the grouping numbers of all descendants contained in the current node;
s52, the database respectively obtains the initial position offset value of the returned data result and the page number limit value in the page information, and determines whether the sum of all resultSize values reaches the sum of the initial position offset value and the page number limit value;
and if the data is not obtained in the step S53, repeating the steps S2 to S4 by the database, and constructing a new nested mapping report for the (n + 1) th data for acquiring new partial access data.
Further, after S53, the method further includes:
s54 the database obtains the location information of the last return record when the sum of all resultSize values is equal to the sum of the start location offset value and the fractional page number limit value.
Further, before S1, the method further includes:
and the user side acquires the query conditions and the grouping information set by the user according to the report query requirements.
Further, after S5, the method further includes:
s6, the user terminal displays the data result according to the dimension sequence.
The invention also provides a statistical report paging system for realizing the statistical report paging method, which comprises a database, wherein the database is used for:
establishing a corresponding data inverted index table according to the dimensionality of each data, wherein the recorded value in the data inverted index table is the address of the data;
acquiring a first query dimension according to dimension information in the query condition acquired by the user side, and querying a first inverted index table corresponding to the first query dimension from the data inverted index table;
establishing a nested mapping report nested according to the dimension sequence according to the first inverted index table, and grouping and sequencing query data;
screening partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user side, and acquiring partial access data from the nested mapping report;
and performing aggregation calculation and subtotal on the partial data and returning the data result to the data service layer.
The present invention further provides a computer device comprising a memory, a processor and a computer program, the computer program being stored on the memory and being executable on the processor, the processor implementing the following steps when executing the computer program:
s1, the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
s2, the database acquires a first query dimension according to the dimension information in the query condition acquired by the user side, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
s3, the database establishes a nested mapping report form nested according to the dimension sequence according to the first inverted index table, and the nested mapping report form is used for grouping and sequencing query data;
s4, the database screens partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user terminal, and acquires partial access data from the nested mapping report;
s5, the database carries out aggregation calculation and subtotal on the partial data and returns the data result to the data service layer.
The present invention further provides a computer-readable storage medium storing a computer program which, when executed by a processor, performs the steps of:
s1 the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
s2, the database acquires a first query dimension according to the dimension information in the query condition acquired by the user side, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
s3, the database establishes a nested mapping report form nested according to the dimension sequence according to the first inverted index table, and the nested mapping report form is used for grouping and sequencing query data;
s4, the database screens partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user terminal, and acquires partial access data from the nested mapping report;
s5, the database carries out aggregation calculation and subtotal on the partial data and returns the data result to the data service layer.
Compared with the prior art, the technical scheme of the invention has the following technical effects:
the globality of the data mainly exists in the same dimension, and the data can be divided into different blocks and respectively subjected to aggregation calculation according to different values in one dimension;
based on the method, in the paging method based on partial data, the database firstly establishes the inverted index according to the dimensionality of the data, and the stored value is the address of the data; then, according to the dimension information in the query condition, finding the inverted index corresponding to the first dimension; then based on the inverted index of the first dimension, establishing a mapping table nested according to the dimension order, and grouping and sequencing data; then, according to the grouping information and the paging information, calculating the position of the access in the mapping table for accessing partial data; then, performing aggregation calculation and subtotal on the extracted partial data, and returning the data to the data service layer;
therefore, the database can be paged based on partial data, and the data service layer can process data results so as to display the paged data at the client side;
the paging method is a real paging method based on partial data, has high data processing speed and low memory occupation, and can quickly and efficiently provide accurate and reliable statistical report paging data for a client.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a diagram illustrating a conventional report paging method in the prior art;
FIG. 2 is a flowchart illustrating a statistical report paging method according to an embodiment of the present invention;
FIG. 3 is a diagram of an inverted index table of data in an actual embodiment of the present invention;
FIG. 4 is a diagram illustrating a structure of a nested mapping report in an embodiment of the present invention;
FIG. 5 is a diagram illustrating the flow of mapping constructs in an actual embodiment of the present invention;
fig. 6 is a schematic flow chart of searching the nth row of data in an actual embodiment of the present invention.
Detailed Description
In the prior art, most report paging methods carry out total statistics on data meeting conditions, store results and page; however, this method is very likely to cause the memory usage to be greatly increased, resulting in unstable service.
As shown in fig. 1, a typical paging process of the conventional report paging method is as follows:
1. the database 300 loads all data meeting the filtering condition;
2. the database 300 groups and sorts the data according to the dimensional order;
3. the database 300 performs aggregate calculation on a group basis and returns all results to the data service layer 200;
4. the data service layer 200 calculates the subtotal of the specified dimension according to the returned result;
5. the data service layer 200 returns the result to the client 100 according to the paging information, the start position and the end position of the positioning result.
How to complete the aggregation and subtotal of data under the condition of not using all data is a difficult problem in report using technology.
Therefore, the invention provides a statistical type report paging method, a statistical type report paging system, statistical type report paging equipment and a statistical type report paging medium, which are used for solving the problems.
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
as shown in fig. 2, an embodiment of the present invention provides a statistical report paging method, including:
s1 the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
the S2 database acquires a first query dimension according to the dimension information in the query condition acquired by the user side, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
the S3 database establishes a nested mapping report form nested according to the dimension order according to the first inverted index table, and is used for grouping and sequencing the query data;
s4 database according to the group information and the paging information obtained from the user end, sifts part of the access position information from the data position information of the nested mapping report and obtains part of the access data from the nested mapping report;
and the S5 database performs aggregate calculation and subtotal on the partial data and returns the data result to the data service layer.
The global property of the data mainly exists in the same dimension, and the data can be divided into different blocks and respectively subjected to aggregation calculation according to different values in one dimension;
based on this, in the paging method based on partial data according to the specific embodiment of the present invention, the database first establishes the inverted index according to the dimensionality of the data, and the stored value is the address of the data; then according to the dimension information in the query condition, finding the inverted index corresponding to the first dimension; then based on the inverted index of the first dimension, establishing a mapping table nested according to the dimension sequence, and grouping and sequencing data; then according to the grouping information and the paging information, calculating the position of the access in the mapping table for accessing partial data; then, performing aggregation calculation and subtotal on the extracted partial data, and returning the data to the data service layer;
therefore, the database can be paged based on partial data, and the data service layer can process data results so as to display the paged data at the client side;
the paging method is a real paging method based on partial data, has high data processing speed and low memory occupation, and can quickly and efficiently provide accurate and reliable statistical report paging data for a client.
In a preferred embodiment, after S5, the method further comprises:
and the S6 user side displays the data results according to the dimension sequence.
In particular embodiments, after the database returns data to the data services layer, the client may organize the data for presentation according to the dimensions, i.e., present paginated data.
In a preferred embodiment, prior to S1, the method further comprises:
and the user side acquires the query conditions and the grouping information set by the user according to the report query requirements.
In a preferred embodiment, S1 specifically includes:
and S10, the database establishes a corresponding data inverted index table according to the dimensionality of each data and the inverted index mode based on the memory storage, and the record value in the data inverted index table is the memory storage address of the data.
In particular embodiments, dimensions are varied and the order of dimensions is varied, and by taking advantage of the inverted index based on memory storage, data can be grouped without incurring excessive additional space overhead, i.e., paging data structures are efficiently organized.
The data inverted index table is a typical mapping table structure, and its value is a list of addresses of records. Regardless of the storage, each row of records requires a storage address.
As shown in FIG. 3, in a practical embodiment, the record for a dimension of data having a value equal to "Key 1" is: 1032, 1226, this number is the address of the record.
In a preferred embodiment, S3 specifically includes:
the S31 database acquires all data of the first query dimension and sorts the data according to a preset sequence;
the S32 database generates a composite query condition according to the query condition and the nth data of the first query dimension, and acquires all data corresponding to the composite query condition; wherein n is a positive integer;
s33 generates a nested mapping report form by using all data structures corresponding to the compound query condition.
In a specific embodiment, a grouping and sorting structure of query data may be established in the memory through the above steps.
In a preferred embodiment, after S5, the method further comprises:
the S51 database respectively obtains the resultSize values of the 1 st to nth data of the first query dimension, and carries out summation calculation; the resultSize value is the sum of the grouping numbers of all descendants contained in the current node;
the S52 database respectively obtains the initial position offset value of the returned data result and the paging number limit value in the paging information, and judges whether the sum of all the resultSize values reaches the sum of the initial position offset value and the paging number limit value;
and if the data is not obtained in the step S53, repeating the steps S2 to S4 by the database, and constructing a new nested mapping report for the n +1 th data for acquiring new partial access data.
In a particular embodiment, the resultSize value is the sum of the number of groupings of all descendants that the current node contains; the offset value is the starting position of the returned result; the limit value is the maximum row data returned, namely the page number set by the user;
in a preferred embodiment, after S53, the method further comprises:
s54 the database obtains the location information of the last return record when the sum of all resultSize values is equal to the sum of the start location offset value and the page number limit value.
In practical embodiments, the structure of a joint index may be derived based on the inverted index table in FIG. 3.
FIG. 4 is a structure of a nested mapping report; after the joint indexing, the dimension 1, the dimension 2 and the dimension 3 are included; wherein,
1. the dimensional order of the report forms, which is hierarchical in the structure, constitutes the hierarchical order of the structure.
The first dimension of the report corresponds to the first level of the nested mapping (i.e., dimension 1), the second dimension of the report corresponds to the second level (i.e., dimension 2), and so on.
2. This nested mapping table constitutes a tree.
At a certain level, a data item (represented by Entry in the class) is not a leaf node if it is not the last dimension, and its children are data of the subsequent dimension of the current dimension, and the parent-child relationship is represented by a TreeMap structure.
In the last dimension, a data list is stored in the data item; for simplicity, objects are represented by an Identification (ID); as in fig. 4, the value (represented by value in the class) of the data item of "dimension 1= key 1, dimension 2= key 1, and dimension 3= key 1" is: object (1032), object (1226).
3. Each data item, if it is not a leaf node (i.e. the last dimension), needs to record the number of its own children (represented by size in the class), and the number of all its descendants (i.e. the sum of the grouping numbers of all descendants that the current node contains resultSize).
Its resultSize = the sum of the resultSize of all its children + 1.
The leaf node's resultSize is equal to 1 because when exposed to the user, it counts as a line of data regardless of how many pieces of data it aggregates.
As in fig. 4, the data item of "dimension 1= key 1, dimension 2= key 1", the number of descendants thereof is 3.
4. Each time a piece of data is inserted into this nested mapping, an update message is passed up the links of the parent-child relationship when a new node is created, which is done by a Listener (represented by Listener in the class) so that the ancestor node can compute the updated resultSize.
In practice, the nested mapping structure described above can be expressed using the following classes:
public interface TreeMap{
List< Entry > nodes;
List<Number> measures;
int size;
int resultSize;
List<Listener> listeners;
}
public class Entry {
String key;
Object value;
}
public class Listener {
public void onEvent(EventType type, TreeMap child);
}
wherein,
1. when the current node is not a leaf node, value is of TreeMap type, thus forming a nested structure.
2. When the current node is a leaf node, value is a list of data objects.
Nodes are children of the current node; if the current node is a leaf node, the value is null.
Size is the number of children, equal to nodes.
ResultSize is the sum of the number of groups of all descendants that the current node contains.
And 6. the listeners are the listeners of the current object, specifically the father of the current object, and the ancestor is informed to update the data when the current object adds/deletes one child node.
Measures is the aggregate computation of the current level, which is the subtotal of the current dimension.
In practice, the paging query interface of the report is:
public class ReportQuerySpec {
List<Dimension> dimensions;
List<Measure> measures;
String criteria;
int offset
Int limit;
}
wherein,
dimensions are the dimensions of a report, which is an ordered list.
Criterion is the query condition,
offset is the starting position of the returned result,
limit is how many line data are returned at most, i.e. number of pages.
Based on the above, the structure of fig. 4 can be derived from the structure of fig. 3, so as to establish a grouping and ordering structure of query data in the memory. The method comprises the following specific steps:
1. and acquiring all data of the first dimension of the report, and sequencing according to a required or default sequence.
2. And acquiring all data of the composite condition of the dimension according to the condition of the query condition + the nth data (n is initially 1) of the first dimension.
3. After the data is acquired, the nested mapping is constructed according to the mapping construction process, and the specific mapping construction process is shown in fig. 5.
4. The sum of reustsize of the 1 st to nth data of the first dimension is calculated;
if resultSize > = offset + limit, then the data for the subsequent dimension is not looked up.
5. If the resultSize < offset + limit, setting n to n +1, and acquiring and constructing a mapping table according to the method from the step 2 to the step 4.
6. Setting the target position as offset + limit, and obtaining the position of the last returned record according to the search flow, wherein the specific search flow is shown in fig. 6;
and obtaining the position of the first offset strip data, namely the position of the first return record according to the flow.
Therefore, the nested mapping report form in fig. 4 can be obtained based on the data inverted index table in fig. 3, and paging of the statistical type report form is realized.
As shown in fig. 5, the mapping construction process specifically includes the following steps:
1) traversing each record;
2) judging whether more records exist;
3) if yes, traversing each dimension of the current node; if not, ending the process;
4) judging whether the current dimension is the last dimension;
5) if the dimension is the last dimension, judging whether the data of the current dimension exists in the context;
511) if yes, obtaining a list corresponding to the current dimension from the context;
512) if the dimension does not exist, a new list storage data is created for the current dimension;
52) adding the current record into the list and returning to the step 1) for circulating the flow;
6) if the dimension is not the last dimension, setting the context as nodes of the current node; judging whether the data of the current dimension exists in the context;
611) if yes, obtaining a mapping table corresponding to the current dimension from the context;
612) if not, a new mapping table is created for the current dimension;
62) setting the current node as the mapping table above, and returning to the step 4) to circulate the process.
As shown in fig. 6, the process of searching the nth data specifically includes the following steps:
1) setting the number of lines to be searched to be N;
2) calculating the sum of resultSize of 1 st to n-1 st elements in the first layer of the nesting mapping table, and calculating totalResultsize;
3) entering the nth element of the first layer and traversing its child nodes;
4) judging whether more child nodes exist;
5) if yes, calculating the sum of totalResultSize + ResultSize of the current node; if not, the process is ended;
6) judging whether the calculated result (namely the sum of totalResultSize + ResultSize of the current node) is larger than or equal to N;
7) if the number of the N is larger than or equal to the number of the N, judging whether the calculated result is equal to N or not;
71) if yes, ending the process;
if not, entering the next layer of dimensionality and returning to the step 4) for circulating flow;
8) if the sum is less than the preset value, setting totalResultSize = totalResultSize + resultSize;
setting the current node as the next child node, and returning to the step 4) for the circulation flow.
It should be noted that, although the steps in the flowchart are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
Example two:
the embodiment of the invention also provides a statistical report paging system, which is used for realizing the statistical report paging method, and the system comprises a database, wherein the database is used for:
establishing a corresponding data inverted index table according to the dimensionality of each data, wherein the recorded value in the data inverted index table is the address of the data;
acquiring a first query dimension according to dimension information in the query condition acquired by the user side, and querying a first inverted index table corresponding to the first query dimension from the data inverted index table;
establishing a nested mapping report nested according to the dimension order according to the first inverted index table, and grouping and sequencing the query data;
screening partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user side, and acquiring partial access data from the nested mapping report;
and performing aggregation calculation and subtotal on the partial data and returning the data result to the data service layer.
For the specific limitations of the above apparatus, reference may be made to the limitations of the above method, which are not described herein again.
Example three:
an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program, where the computer program is stored in the memory and can be run on the processor, and the processor implements the following steps when executing the computer program:
s1 the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
the S2 database acquires a first query dimension according to the dimension information in the query condition acquired by the user side, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
the S3 database establishes a nested mapping report form nested according to the dimension sequence according to the first inverted index table, and is used for grouping and sequencing the query data;
s4, the database screens part of the access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user terminal, and acquires part of the access data from the nested mapping report;
and the S5 database performs aggregate calculation and subtotal on the partial data and returns the data result to the data service layer.
Example four:
an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the following steps:
s1 the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
s2, the database acquires a first query dimension according to the dimension information in the query condition acquired by the user terminal, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
the S3 database establishes a nested mapping report form nested according to the dimension sequence according to the first inverted index table, and is used for grouping and sequencing the query data;
s4, the database screens part of the access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user terminal, and acquires part of the access data from the nested mapping report;
and the S5 database performs aggregate calculation and subtotal on the partial data and returns the data result to the data service layer.
It should be noted that the above-mentioned embodiments are only preferred embodiments of the present invention and the technical principles applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A statistical report paging method is characterized by comprising the following steps:
s1 the database establishes a corresponding data inverted index table according to the dimensionality of each data, and the record value in the data inverted index table is the address of the data;
s2, the database acquires a first query dimension according to the dimension information in the query condition acquired by the user terminal, and queries a first inverted index table corresponding to the first query dimension from the data inverted index table;
s3, the database establishes a nested mapping report form nested according to the dimension sequence according to the first inverted index table, and the nested mapping report form is used for grouping and sequencing query data;
s4, the database screens partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user terminal, and acquires partial access data from the nested mapping report;
s5, the database carries out aggregation calculation and subtotal on the partial data and returns the data result to the data service layer.
2. The method for paging statistical report forms as claimed in claim 1, wherein S1 specifically comprises:
s10, the database establishes a corresponding data inverted index table according to the dimensionality of each data and an inverted index mode based on memory storage, and the record value in the data inverted index table is the memory storage address of the data.
3. The method for paging statistical report forms as claimed in claim 1, wherein S3 specifically comprises:
s31, the database acquires all data of the first query dimension and sorts the data according to a preset sequence;
s32, the database generates a composite query condition according to the query condition and the nth data of the first query dimension, and acquires all data corresponding to the composite query condition; wherein n is a positive integer;
s33, constructing all data corresponding to the compound query condition to generate the nested mapping report.
4. The method for paging statistical reports as claimed in claim 3, wherein after S5, the method further comprises:
s51, the database respectively obtains the resultSize values of the 1 st to nth data of the first query dimension, and carries out summation calculation; the resultSize value is the sum of the grouping numbers of all descendants contained in the current node;
s52, the database respectively obtains the initial position offset value of the returned data result and the page number limit value in the page information, and determines whether the sum of all resultSize values reaches the sum of the initial position offset value and the page number limit value;
and if the data is not obtained in the step S53, repeating the steps S2 to S4 by the database, and constructing a new nested mapping report for the (n + 1) th data for acquiring new partial access data.
5. The method for paginating statistical reports of claim 4, wherein after S53, the method further comprises:
s54 the database obtains the location information of the last return record when the sum of all resultSize values is equal to the sum of the start location offset value and the fractional page number limit value.
6. The method for paging statistical reports as claimed in claim 1, wherein before S1, the method further comprises:
and the user side acquires the query conditions and the grouping information set by the user according to the report query requirements.
7. The method for paging statistical reports as claimed in claim 1, wherein after S5, the method further comprises:
and S6, the user side displays the data results according to the dimension sequence.
8. A statistical report paging system for implementing the statistical report paging method according to any one of claims 1 to 7, the system comprising a database for:
establishing a corresponding data inverted index table according to the dimensionality of each data, wherein the recorded value in the data inverted index table is the address of the data;
acquiring a first query dimension according to dimension information in query conditions acquired by a user side, and querying a first inverted index table corresponding to the first query dimension from the data inverted index table;
establishing a nested mapping report nested according to the dimension sequence according to the first inverted index table, and grouping and sequencing query data;
screening partial access position information from the data position information of the nested mapping report according to the grouping information and the paging information acquired by the user side, and acquiring partial access data from the nested mapping report;
and performing aggregation calculation and subtotal on the partial data and returning the data result to the data service layer.
9. A computer device comprising a memory, a processor and a computer program, the computer program being stored on the memory and being executable on the processor, wherein the processor when executing the computer program implements the steps of the statistical report paging method according to any of the claims 1-7.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the statistical report paging method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211037366.7A CN115129724A (en) | 2022-08-29 | 2022-08-29 | Statistical report paging method, system, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211037366.7A CN115129724A (en) | 2022-08-29 | 2022-08-29 | Statistical report paging method, system, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115129724A true CN115129724A (en) | 2022-09-30 |
Family
ID=83387469
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211037366.7A Pending CN115129724A (en) | 2022-08-29 | 2022-08-29 | Statistical report paging method, system, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115129724A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149777A (en) * | 2023-10-27 | 2023-12-01 | 腾讯科技（深圳）有限公司 | Data query method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107291947A (en) * | 2016-09-21 | 2017-10-24 | 广州特道信息科技有限公司 | A kind of method of Query semi-structured for data and distribution NewSQL Database Systems |
| CN107918618A (en) * | 2016-10-10 | 2018-04-17 | 腾讯科技（北京）有限公司 | Data processing method and device |
| US20180302393A1 (en) * | 2017-04-18 | 2018-10-18 | HFA Group | Parameterized data delivery system for a spreadsheet application |
| CN113986931A (en) * | 2021-12-27 | 2022-01-28 | 畅捷通信息技术股份有限公司 | Report paging method and device, storage medium and computing equipment |
-
2022
- 2022-08-29 CN CN202211037366.7A patent/CN115129724A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107291947A (en) * | 2016-09-21 | 2017-10-24 | 广州特道信息科技有限公司 | A kind of method of Query semi-structured for data and distribution NewSQL Database Systems |
| CN107918618A (en) * | 2016-10-10 | 2018-04-17 | 腾讯科技（北京）有限公司 | Data processing method and device |
| US20180302393A1 (en) * | 2017-04-18 | 2018-10-18 | HFA Group | Parameterized data delivery system for a spreadsheet application |
| CN113986931A (en) * | 2021-12-27 | 2022-01-28 | 畅捷通信息技术股份有限公司 | Report paging method and device, storage medium and computing equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117149777A (en) * | 2023-10-27 | 2023-12-01 | 腾讯科技（深圳）有限公司 | Data query method, device, equipment and storage medium |
| CN117149777B (en) * | 2023-10-27 | 2024-02-06 | 腾讯科技（深圳）有限公司 | Data query method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109165215B (en) | Method and device for constructing space-time index in cloud environment and electronic equipment | |
| CN106462583B (en) | System and method for rapid data analysis | |
| US6732085B1 (en) | Method and system for sample size determination for database optimizers | |
| US8352476B2 (en) | Frequent itemset counting using clustered prefixes and index support | |
| CN104408180A (en) | Stored data inquiring method and device | |
| CN115129724A (en) | Statistical report paging method, system, equipment and medium | |
| CN113987002A (en) | Data exchange method based on mass data analysis platform | |
| CN116126864A (en) | Index construction method, data query method and related equipment | |
| CN114722036B (en) | Data processing method, device, electronic equipment and readable storage medium | |
| CN113722415B (en) | Point cloud data processing method and device, electronic equipment and storage medium | |
| CN111813773B (en) | Power grid meter reading data storage method, uploading method, device and storage device | |
| CN112836124A (en) | Image data acquisition method and device, electronic equipment and storage medium | |
| CN111125158B (en) | Data table processing method, device, medium and electronic equipment | |
| CN112241410A (en) | Data storage method, data index construction method and device and computer equipment | |
| CN109710633B (en) | Method and device for determining man-in-middle information and intelligent terminal | |
| US8655911B2 (en) | Expressing frequent itemset counting operations | |
| CN114564501B (en) | Database data storage and query method, device, equipment and medium | |
| CN111723122A (en) | Method, device and equipment for determining association rule between data and readable storage medium | |
| CN111339245B (en) | Data storage method, device, storage medium and equipment | |
| CN113742344A (en) | Method and device for indexing power system data | |
| CN112148941A (en) | Information prompting method and device and terminal equipment | |
| CN114943004B (en) | Attribute graph query method, attribute graph query device, and storage medium | |
| US11061876B2 (en) | Fast aggregation on compressed data | |
| CN117370619B (en) | Method and device for storing and sub-sampling images in fragments | |
| CN118277456B (en) | Initiating node output method in MPP distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220930 |