CN108268523B - Database aggregation processing method and device - Google Patents
Database aggregation processing method and device Download PDFInfo
- Publication number
- CN108268523B CN108268523B CN201611264048.9A CN201611264048A CN108268523B CN 108268523 B CN108268523 B CN 108268523B CN 201611264048 A CN201611264048 A CN 201611264048A CN 108268523 B CN108268523 B CN 108268523B
- Authority
- CN
- China
- Prior art keywords
- equivalent
- index
- query
- aggregation table
- indexes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/244—Grouping and aggregation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24537—Query rewriting; Transformation of operators
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a database aggregation processing method and device. The method comprises the following steps: obtaining dimensionality and indexes in query conditions for querying a database, wherein the indexes are queried contents, and the dimensionality is a limiting condition for limiting the queried contents; searching equivalent indexes corresponding to each index in the query condition; judging whether the aggregation table of the equivalent indexes includes dimensions or not for the searched equivalent indexes; if the equivalent index does not include the dimension in the aggregation table, judging whether the dimension is included in the target aggregation table or not; and if so, inquiring in the target aggregation table by using the equivalent index and the dimensionality, and returning a result obtained after inquiry. By the method and the device, the problem of low query efficiency when data query is carried out in the database in the related technology is solved.
Description
Technical Field
The application relates to the technical field of database query, in particular to a database aggregation processing method and device.
Background
In the existing products, much data is recorded in the original data table of the front end, thus causing the original data table to be too large. When data is queried in the original data table, the performance overhead is large, the query process is slow, and the query efficiency is low. To speed up the data query of the front end, it is usually accelerated by pre-aggregating the tables. However, when using the indexes on the pre-polymerization table, it is necessary to present all internal indexes on the pre-polymerization table to the front end, and the front end caller needs to select indexes from all internal indexes on the pre-polymerization table according to indexes in the original database, and if the pre-polymerization table includes the dimension in the query condition, the pre-polymerization table is subjected to dimension parsing according to the indexes selected from all internal indexes on the pre-polymerization table, and if the pre-polymerization table does not include the dimension in the query condition, the pre-polymerization table is considered that the pre-polymerization table cannot be parsed.
However, in some cases, even if the pre-polymerization table does not include the dimension in the query condition, the pre-polymerization table can be parsed, so in the related art, it is not accurate to determine whether to parse the pre-polymerization table according to whether the dimension in the query condition is included in the pre-polymerization table, and the value of the pre-polymerization table is not fully utilized when data query is performed in the database, which affects query efficiency.
Aiming at the problem of low query efficiency in data query in a database in the related art, no effective solution is provided at present.
Disclosure of Invention
The present application mainly aims to provide a database aggregation processing method and apparatus, so as to solve the problem of low query efficiency when data query is performed in a database in the related art.
In order to achieve the above object, according to one aspect of the present application, there is provided a database aggregation processing method. The method comprises the following steps: obtaining dimensionality and indexes in query conditions for querying a database, wherein the indexes are queried contents, and the dimensionality is a limiting condition for limiting the queried contents; searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table; judging whether the aggregation table of the equivalent indexes includes dimensions or not for the searched equivalent indexes; if the aggregation table of the equivalent index does not include the dimension, judging whether the target aggregation table includes the dimension, wherein the target aggregation table is the table formed by connecting the aggregation table of the equivalent index and the bridging table thereof; and under the condition that the judgment result is yes, inquiring in the target aggregation table by using the equivalent index and the dimensionality, and returning the result obtained after inquiry.
Further, in a case that one index in the query condition corresponds to a plurality of equivalent indexes, performing query in the target aggregation table using the equivalent indexes and the dimensions includes: selecting the equivalent index with the minimum data volume overhead of a target aggregation table where the equivalent index is located from the multiple equivalent indexes; and querying in a target aggregation table with minimum data volume overhead by using the equivalent indexes and the dimensions.
Further, in a case that one index in the query condition corresponds to a plurality of equivalent indexes, performing query in the aggregation table using the equivalent indexes and the dimensions includes: sequencing a target aggregation table where a plurality of equivalent indexes are located according to the data volume overhead from small to large; judging whether the dimensionality of the query condition is included or not from the target aggregation table with the minimum data volume overhead, and querying the target aggregation table with the minimum data volume overhead under the condition that the judgment result is yes; and under the condition that the judgment result is negative, sequentially judging whether the next target aggregation table comprises dimensions.
Furthermore, the equivalent index corresponding to the index is configured in advance, and the equivalent index is equal to the corresponding index.
Further, in the case that there are a plurality of target aggregation tables, querying in the target aggregation table using the equivalence index and the dimension includes: selecting an aggregation table with the minimum data volume overhead from a plurality of target aggregation tables; and querying in an aggregation table with minimum data volume overhead by using all equivalent indexes and dimensions.
Further, a summary column of the lower-granularity data of the predetermined granularity is added to the data record row of the predetermined granularity of the target aggregation table.
In order to achieve the above object, according to another aspect of the present application, there is provided a database aggregation processing apparatus. The device includes: the system comprises an acquisition unit, a query unit and a processing unit, wherein the acquisition unit is used for acquiring dimension and index in query conditions for querying a database, the index is queried content, and the dimension is a limiting condition for limiting the queried content; the searching unit is used for searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in the aggregation table, and the aggregation table is a table obtained by aggregating the original data table; the first judgment unit is used for judging whether the aggregation table where the equivalent indexes are located comprises the dimensionality or not for the searched equivalent indexes; the second judgment unit is used for judging whether the target aggregation table comprises dimensions or not under the condition that the aggregation table of the equivalent index does not comprise the dimensions, wherein the target aggregation table is a table formed by connecting the aggregation table of the equivalent index with the bridging table of the target aggregation table; and the query unit is used for querying in the target aggregation table by using the equivalent index and the dimension under the condition that the judgment result is yes, and returning the result obtained after query.
Further, the query unit includes: the selection module is used for selecting the equivalent index with the minimum data volume overhead of the target aggregation table where the equivalent index is located from the multiple equivalent indexes under the condition that one index in the query condition corresponds to the multiple equivalent indexes; and the first query module is used for querying the target aggregation table with the minimum data volume overhead by using the equivalent indexes and the dimensions.
Further, the query unit includes: the sorting module is used for sorting the target aggregation tables where the equivalent indexes are located according to the data volume overhead from small to large under the condition that one index in the query condition corresponds to the equivalent indexes; the second query module is used for judging whether the dimensionality of the query condition is included or not from the target aggregation table with the minimum data volume overhead, and querying the target aggregation table with the minimum data volume overhead under the condition that the judgment result is yes; and the judging module is used for sequentially judging whether the next target aggregation table comprises dimensions or not under the condition that the judging result is negative.
Furthermore, the equivalent index corresponding to the index is configured in advance, and the equivalent index is equal to the corresponding index.
Through the application, the following steps are adopted: obtaining dimensionality and indexes in query conditions for querying a database, wherein the indexes are queried contents, and the dimensionality is a limiting condition for limiting the queried contents; searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table; judging whether the aggregation table of the equivalent indexes includes dimensions or not for the searched equivalent indexes; if the aggregation table of the equivalent index does not include the dimension, judging whether the target aggregation table includes the dimension, wherein the target aggregation table is the table formed by connecting the aggregation table of the equivalent index and the bridging table thereof; and under the condition that the judgment result is yes, the equivalent index and the dimensionality are used for inquiring in the target aggregation table, and the result obtained after inquiry is returned, so that the problem of low inquiry efficiency when data inquiry is carried out in a database in the related technology is solved. By using the equivalent indexes and the dimensionalities to query the aggregation table and the bridging table thereof, the value of the aggregation table is fully used, and the effect of improving the query efficiency when data query is performed in the database is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
fig. 1 is a flowchart of a database aggregation processing method provided according to an embodiment of the present application; and
fig. 2 is a schematic diagram of a database aggregation processing apparatus according to an embodiment of the present application.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an embodiment of the application, a database aggregation processing method is provided.
Fig. 1 is a flowchart of a database aggregation processing method according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S101, obtaining the dimension and the index in the query condition for querying the database, wherein the index is the content of query, and the dimension is the limiting condition for limiting the query content.
For example, the query conditions obtained for querying the database are the order amount and the access amount of the product a in the first quarter, the first quarter and the product a serve as the limiting conditions for limiting the query content, and the order amount and the access amount serve as the query content. That is, the dimensions are quarter number and product A, and the metrics are order volume and visit volume.
And S102, searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating the original data table.
Each index in the query condition is an index in an original data table, and the equivalent index is an index in the aggregation table. And searching the equivalent index corresponding to each index in the query condition, for example, searching the equivalent index corresponding to the order quantity in the query condition.
Optionally, in the database aggregation processing method provided in the embodiment of the present application, the equivalent index corresponding to the index is pre-configured, and the equivalent index is equal to the index corresponding to the equivalent index. That is, the index in the query condition is used for querying in the original data table, and the equivalent index corresponding to the index is used for querying in the aggregation table, and the result obtained after query is returned is the same. And searching the equivalent indexes corresponding to each index in the query condition by a dynamic replacement method of the equivalent indexes.
The aggregation table in the present application is a table obtained by aggregating the original data table according to a preset condition. For example, the aggregate table contains only limited dimension columns and indices, thus making data queries in the aggregate table more efficient.
Step S103, judging whether the aggregation table of the equivalent indexes includes the dimension or not for the searched equivalent indexes.
Because it cannot be ensured that the aggregation table where the equivalent index is located includes the dimension in the query condition, after the equivalent index is found, whether the aggregation table where the equivalent index is located includes the dimension in the query condition is judged. For example, the dimension is the first quarter and the product a, and after the equivalent index corresponding to the order quantity is found, whether the aggregation table in which the equivalent index corresponding to the order quantity is located includes the dimension first quarter and the product a in the query condition is judged.
It should be noted that the dimensions mentioned in this application indicate all dimensions of the query condition, that is, whether all dimensions in the query condition are included in the aggregation table where the equivalent index is located is determined.
And step S104, if the aggregation table of the equivalent index does not comprise the dimension, judging whether the target aggregation table comprises the dimension, wherein the target aggregation table is the table formed by connecting the aggregation table of the equivalent index with the bridging table thereof.
And if the aggregation table of the equivalent index does not comprise all dimensions in the query condition, judging whether the target aggregation table comprises all dimensions in the query condition, wherein the target aggregation table is the table formed by connecting the aggregation table of the equivalent index with the bridging table thereof.
It should be noted that, because the index table may appear in any snowflake configuration table, the index table may be converted into the bridging table through snowflake configuration table connection in the query component, in this application, the bridging table may be connected with a bridging table of a predetermined granularity through a certain data column in the aggregation table, and during query, table connection is performed through the bridging table and the pre-aggregation table, and meanwhile, query or filtering is performed on the bridging table by using the dimension. The table after the aggregation table is connected with the bridge table is referred to as a target aggregation table in the present application, the aggregation table is not necessarily one table after being connected with the bridge table, but the aggregation table is connected with the bridge table through a connection relationship.
Since the bridging table of the equivalent index may include the dimension when the aggregation table of the equivalent index does not include the dimension, it is determined whether the aggregation table of the equivalent index and the bridging table of the equivalent index include the dimension.
And step S105, under the condition that the judgment result is yes, inquiring in the target aggregation table by using the equivalent index and the dimension, and returning the result obtained after inquiry.
And if the judgment result is yes, namely the aggregation table where the equivalent index is located and the bridging table of the aggregation table include the dimension, querying the target aggregation table by using the equivalent index and the dimension, and returning a result obtained after query.
Through the steps, the bridging table is connected with the aggregation table, the bridging table with the connection relation and the aggregation table are inquired and filtered by using the equivalent indexes and the dimensions, the consistent data effect is achieved, meanwhile, the acceleration of the aggregation table can be applied, and the inquiry efficiency when data inquiry is carried out in a database is improved.
Optionally, in the database aggregation processing method provided in the embodiment of the present application, when one index in the query condition corresponds to multiple equivalent indexes, performing query in the target aggregation table using the equivalent indexes and the dimensions includes: selecting the equivalent index with the minimum data volume overhead of a target aggregation table where the equivalent index is located from the multiple equivalent indexes; and querying in a target aggregation table with minimum data volume overhead by using the equivalent indexes and the dimensions.
In the present application, one index in the query condition corresponds to a plurality of equivalent indexes, and this means that a certain index in the query condition corresponds to a plurality of equivalent indexes.
For example, in the case of multiple equivalent indexes corresponding to the order quantity (a certain index in the query condition), an equivalent index with the minimum data volume overhead of the target aggregation table where the equivalent index is located is selected from the multiple equivalent indexes corresponding to the order quantity, and query is performed on the target aggregation table with the minimum data volume overhead by using the equivalent index and the dimension corresponding to the order quantity.
Optionally, in the database aggregation processing method provided in the embodiment of the present application, when one index in the query condition corresponds to multiple equivalent indexes, performing query in the aggregation table using the equivalent indexes and the dimensions includes: sequencing a target aggregation table where a plurality of equivalent indexes are located according to the data volume overhead from small to large; judging whether the dimensionality of the query condition is included or not from the target aggregation table with the minimum data volume overhead, and querying the target aggregation table with the minimum data volume overhead under the condition that the judgment result is yes; and under the condition that the judgment result is negative, sequentially judging whether the next target aggregation table comprises dimensions.
For example, under the condition of multiple equivalent indexes corresponding to an order quantity (a certain index in a query condition), a target aggregation table with the minimum data quantity overhead is selected according to the data quantity overhead of a target aggregation table where the multiple equivalent indexes are located, whether the dimension (namely, a first quarter and a product a) of the query condition is included is judged from the target aggregation table with the minimum data quantity overhead, and under the condition that the target aggregation table with the minimum data quantity overhead includes the first quarter and the product a of the query condition, query is performed in the target aggregation table with the minimum data quantity overhead. And if the target aggregation table with the minimum data volume overhead does not comprise the first quarter and the product A of the query condition, sequentially judging whether the next target aggregation table comprises the first quarter and the product A of the query condition or not, and inquiring in the target aggregation table under the condition that a certain target aggregation table comprises the first quarter and the product A of the query condition.
By carrying out data query on the pre-target aggregation table with the minimum overhead, the effect of accelerating the data query is achieved while the query effect is ensured.
Optionally, in the database aggregation processing method provided in the embodiment of the present application, when there are a plurality of target aggregation tables, querying in the target aggregation table using the equivalent index and the dimension includes: selecting an aggregation table with the minimum data volume overhead from a plurality of target aggregation tables; and querying in an aggregation table with minimum data volume overhead by using all equivalent indexes and dimensions.
Selecting an aggregation table with the minimum data volume overhead from the plurality of target aggregation tables when the plurality of target aggregation tables are available; and querying in the target aggregation table with the minimum data volume overhead by using all equivalent indexes and dimensions, so that the effect of accelerating data query is achieved while the query effect is ensured.
Optionally, in the database aggregation processing method provided in the embodiment of the present application, a summary column of lower-layer granularity data of a predetermined granularity is added to a data record row of the predetermined granularity in the target aggregation table.
The target aggregation table in the present application may have several data granularity levels, similar to a visitor, a session, and a page view (a visitor may have multiple sessions, and a session may also have multiple page views, so that there are different granularities, and different granularities may be table-connected, i.e. a snowflake type), where a statistical field is added on a predetermined granularity, for example, in a data record row of the visitor, a corresponding session number summary column and a page view number summary column are added, which may be used to calculate two indexes, i.e. a session number and a page view number, respectively.
And aiming at the data columns obtained by adding all the accumulated index columns with the lower layer finer granularity on the target aggregation table with the preset granularity, carrying out configuration of equivalent indexes on the newly added data columns in the metadata configuration of the query layer.
The method comprises the steps of adding all possible or frequently-inquired index aggregation columns with finer granularity on a granularity table with preset granularity to achieve the purpose of connecting an aggregation table with a bridging table, prior to replacing equivalent indexes on an inquiry layer, preferentially judging whether all indexes can be inquired from a table with the minimum overhead or not, wherein the table has related dimension columns, so that the aggregation table is fully utilized, and meanwhile, the inquiry efficiency of data inquiry in a database is guaranteed.
According to the database aggregation processing method provided by the embodiment of the application, the dimensionality and the index in the query condition for querying the database are obtained, wherein the index is the content of query, and the dimensionality is the limiting condition for limiting the query content; searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table; judging whether the aggregation table of the equivalent indexes includes dimensions or not for the searched equivalent indexes; if the aggregation table of the equivalent index does not include the dimension, judging whether the target aggregation table includes the dimension, wherein the target aggregation table is the table formed by connecting the aggregation table of the equivalent index and the bridging table thereof; and under the condition that the judgment result is yes, the equivalent index and the dimensionality are used for inquiring in the target aggregation table, and the result obtained after inquiry is returned, so that the problem of low inquiry efficiency when data inquiry is carried out in a database in the related technology is solved. By using the equivalent indexes and the dimensionalities to query the aggregation table and the bridging table thereof, the value of the aggregation table is fully used, and the effect of improving the query efficiency when data query is performed in the database is achieved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
The embodiment of the present application further provides a database aggregation processing apparatus, and it should be noted that the database aggregation processing apparatus according to the embodiment of the present application may be used to execute the method for database aggregation processing provided in the embodiment of the present application. The following describes a database aggregation processing apparatus provided in an embodiment of the present application.
Fig. 2 is a schematic diagram of a database aggregation processing apparatus according to an embodiment of the present application. As shown in fig. 2, the apparatus includes: the device comprises an acquisition unit 10, a search unit 20, a first judgment unit 30, a second judgment unit 40 and an inquiry unit 50.
Specifically, the obtaining unit 10 is configured to obtain a dimension and an index in a query condition for querying the database, where the index is content of a query, and the dimension is a constraint condition that limits the content of the query.
The searching unit 20 is configured to search an equivalent index corresponding to each index in the query condition, where the equivalent index is an index in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table.
The first determining unit 30 is configured to determine, for the found equivalent indicator, whether the aggregation table where the equivalent indicator is located includes a dimension.
The second determining unit 40 is configured to determine whether the target aggregation table includes a dimension when the aggregation table where the equivalent index is located does not include the dimension, where the target aggregation table is a table in which the aggregation table where the equivalent index is located is connected to the bridge table.
And the query unit 50 is configured to query the target aggregation table by using the equivalent index and the dimension if the determination result is yes, and return a result obtained after the query.
The database aggregation processing apparatus provided in the embodiment of the application acquires, by the acquiring unit 10, a dimension and an index in a query condition for querying a database, where the index is a query content and the dimension is a constraint condition that limits the query content; the searching unit 20 searches for an equivalent index corresponding to each index in the query condition, where the equivalent index is an index in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table; the first judging unit 30 judges whether the aggregation table where the equivalent index is located includes a dimension for the found equivalent index; the second judging unit 40 judges whether the target aggregation table includes a dimension or not under the condition that the aggregation table where the equivalent index is located does not include the dimension, wherein the target aggregation table is a table in which the aggregation table where the equivalent index is located is connected with the bridge table; when the determination result is yes, the query unit 50 uses the equivalent index and the dimension to perform query in the target aggregation table, and returns the result obtained after query, thereby solving the problem of low query efficiency when performing data query in the database in the related art. By using the equivalent indexes and the dimensionalities to query the aggregation table and the bridging table thereof, the value of the aggregation table is fully used, and the effect of improving the query efficiency when data query is performed in the database is achieved.
Optionally, in the database aggregation processing apparatus provided in the embodiment of the present application, the query unit 50 includes: the selection module is used for selecting the equivalent index with the minimum data volume overhead of the target aggregation table where the equivalent index is located from the multiple equivalent indexes under the condition that one index in the query condition corresponds to the multiple equivalent indexes; and the first query module is used for querying the target aggregation table with the minimum data volume overhead by using the equivalent indexes and the dimensions.
Optionally, in the database aggregation processing apparatus provided in the embodiment of the present application, the query unit 50 includes: the sorting module is used for sorting the target aggregation tables where the equivalent indexes are located according to the data volume overhead from small to large under the condition that one index in the query condition corresponds to the equivalent indexes; the second query module is used for judging whether the dimensionality of the query condition is included or not from the target aggregation table with the minimum data volume overhead, and querying the target aggregation table with the minimum data volume overhead under the condition that the judgment result is yes; and the judging module is used for sequentially judging whether the next target aggregation table comprises dimensions or not under the condition that the judging result is negative.
Optionally, in the database aggregation processing apparatus provided in the embodiment of the present application, the equivalent index corresponding to the index is pre-configured, and the equivalent index is equal to the index corresponding to the equivalent index.
The database aggregation processing apparatus includes a processor and a memory, the acquiring unit 10, the searching unit 20, the first judging unit 30, the second judging unit 40, the querying unit 50, and the like are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to one or more, and the database aggregation is processed by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The present application further provides an embodiment of a computer program product, which, when being executed on a data processing device, is adapted to carry out program code for initializing the following method steps: obtaining dimensionality and indexes in query conditions for querying a database, wherein the indexes are queried contents, and the dimensionality is a limiting condition for limiting the queried contents; searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table; judging whether the aggregation table of the equivalent indexes includes dimensions or not for the searched equivalent indexes; if the aggregation table of the equivalent index does not include the dimension, judging whether the target aggregation table includes the dimension, wherein the target aggregation table is the table formed by connecting the aggregation table of the equivalent index and the bridging table thereof; and under the condition that the judgment result is yes, inquiring in the target aggregation table by using the equivalent index and the dimensionality, and returning the result obtained after inquiry.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (5)
1. A database aggregation processing method is characterized by comprising the following steps:
obtaining a dimension and an index in a query condition for querying a database, wherein the index is content of query, and the dimension is a limiting condition for limiting the content of query;
searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table;
judging whether the aggregation table of the equivalent indexes comprises the dimensionality or not for the searched equivalent indexes;
if the aggregation table of the equivalent index does not include the dimension, judging whether the target aggregation table includes the dimension, wherein the target aggregation table is a table formed by connecting the aggregation table of the equivalent index with a bridging table of the equivalent index;
if the judgment result is yes, using the equivalent index and the dimensionality to query in the target aggregation table, and returning a result obtained after query;
wherein, when one index in the query condition corresponds to a plurality of equivalent indexes, using the equivalent indexes and the dimensions to perform query in the target aggregation table includes:
selecting the equivalent index with the minimum data volume overhead of the target aggregation table where the equivalent index is located from the multiple equivalent indexes;
querying in the target aggregation table with the minimum data volume overhead by using the equivalence index and the dimension;
wherein, when one index in the query condition corresponds to a plurality of equivalent indexes, using the equivalent indexes and the dimensions to query in an aggregation table comprises:
sequencing the target aggregation tables where the equivalent indexes are located according to the data volume overhead from small to large;
judging whether the dimensionality of the query condition is included or not from the target aggregation table with the minimum data volume overhead, and querying the target aggregation table with the minimum data volume overhead under the condition that the judgment result is yes;
under the condition that the judgment result is negative, sequentially judging whether the dimension is included in the next target aggregation table or not;
and a summary column of the lower-layer granularity data with the preset granularity is added in a data record row with the preset granularity of the target aggregation table.
2. The method of claim 1, wherein the equivalent index corresponding to the index is pre-configured, and the equivalent index is equal to the corresponding index.
3. The method of claim 1, wherein, in the case that the target aggregation table is plural, using the equivalence index and the dimension to perform the query in the target aggregation table comprises:
selecting an aggregation table with the minimum data volume overhead from a plurality of target aggregation tables;
and querying in the aggregation table with the minimum data volume overhead by using all the equivalence indexes and the dimensions.
4. A database aggregation processing apparatus, comprising:
the system comprises an acquisition unit, a query unit and a processing unit, wherein the acquisition unit is used for acquiring dimension and index in query conditions for querying a database, the index is content of query, and the dimension is a limiting condition for limiting the content of query;
the searching unit is used for searching equivalent indexes corresponding to each index in the query condition, wherein the equivalent indexes are indexes in an aggregation table, and the aggregation table is a table obtained by aggregating an original data table;
the first judgment unit is used for judging whether the aggregation table where the equivalent indexes are located comprises the dimensionality or not for the searched equivalent indexes;
a second judging unit, configured to judge whether a target aggregation table includes the dimension when the aggregation table of the equivalent index does not include the dimension, where the target aggregation table is a table in which the equivalent index is connected to a bridge table thereof;
the query unit is used for querying in the target aggregation table by using the equivalent index and the dimension under the condition that the judgment result is yes, and returning a result obtained after query;
wherein the query unit comprises:
the selection module is used for selecting the equivalent index with the minimum data volume overhead of the target aggregation table where the equivalent index is located from the multiple equivalent indexes under the condition that one index in the query condition corresponds to the multiple equivalent indexes;
the first query module is used for querying a target aggregation table with the minimum data volume overhead by using the equivalent index and the dimension;
the sorting module is used for sorting the target aggregation tables where the equivalent indexes are located according to the data volume overhead from small to large under the condition that one index in the query condition corresponds to the equivalent indexes;
the second query module is used for judging whether the dimensionality of the query condition is included or not from the target aggregation table with the minimum data volume overhead, and querying the target aggregation table with the minimum data volume overhead under the condition that the judgment result is yes;
the judging module is used for sequentially judging whether the dimension is included in the next target aggregation table or not under the condition that the judging result is negative;
and a summary column of the lower-layer granularity data with the preset granularity is added in a data record row with the preset granularity of the target aggregation table.
5. The apparatus of claim 4, wherein the equivalent index corresponding to the index is pre-configured, and the equivalent index is equal to the corresponding index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611264048.9A CN108268523B (en) | 2016-12-30 | 2016-12-30 | Database aggregation processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611264048.9A CN108268523B (en) | 2016-12-30 | 2016-12-30 | Database aggregation processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108268523A CN108268523A (en) | 2018-07-10 |
| CN108268523B true CN108268523B (en) | 2021-06-22 |
Family
ID=62755266
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611264048.9A Active CN108268523B (en) | 2016-12-30 | 2016-12-30 | Database aggregation processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108268523B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110060118A (en) * | 2019-02-27 | 2019-07-26 | 浙江执御信息技术有限公司 | A kind of order is honoured an agreement full link method for real-time monitoring, device and computer equipment |
| CN112035173B (en) * | 2020-11-04 | 2021-03-05 | 北京爱奇艺智能科技有限公司 | Method and system for supporting multi-dimensional content aggregation display of VR (virtual reality) all-in-one machine |
| CN112445814A (en) * | 2020-12-15 | 2021-03-05 | 北京乐学帮网络技术有限公司 | Data acquisition method and device, computer equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103853818A (en) * | 2014-02-12 | 2014-06-11 | 博易智软（北京）技术股份有限公司 | Multidimensional data processing method and device |
| CN103975556A (en) * | 2011-11-22 | 2014-08-06 | 瑞典爱立信有限公司 | Improved replication management for remote multicast replication network |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020029207A1 (en) * | 2000-02-28 | 2002-03-07 | Hyperroll, Inc. | Data aggregation server for managing a multi-dimensional database and database management system having data aggregation server integrated therein |
| US6963826B2 (en) * | 2003-09-22 | 2005-11-08 | C3I, Inc. | Performance optimizer system and method |
| DE112006002942B4 (en) * | 2005-11-04 | 2014-03-13 | GM Global Technology Operations LLC (n. d. Ges. d. Staates Delaware) | Active material actuated head restraint assemblies |
| CN101196890B (en) * | 2006-12-08 | 2010-06-16 | 国际商业机器公司 | Method and device for analyzing information and application performance during polymerized data base operation |
| CN101876984A (en) * | 2009-12-07 | 2010-11-03 | 金蝶软件（中国）有限公司 | Data management system and data relation query method and device thereof |
| CN102012936B (en) * | 2010-12-07 | 2013-04-17 | 中国电信股份有限公司 | Massive data aggregation method and system based on cloud computing platform |
| CN102937980B (en) * | 2012-10-18 | 2016-04-13 | 亿赞普(北京)科技有限公司 | A kind of Cluster Database data enquire method |
| CN104317958B (en) * | 2014-11-12 | 2018-01-16 | 北京国双科技有限公司 | A kind of real-time data processing method and system |
| CN104484398B (en) * | 2014-12-12 | 2018-02-02 | 北京国双科技有限公司 | Data aggregation method and device in tables of data |
| CN105183769B (en) * | 2015-07-31 | 2018-08-24 | 浙江工商大学 | Based on the cubical track data visualized in situ method of flow data |
-
2016
- 2016-12-30 CN CN201611264048.9A patent/CN108268523B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103975556A (en) * | 2011-11-22 | 2014-08-06 | 瑞典爱立信有限公司 | Improved replication management for remote multicast replication network |
| CN103853818A (en) * | 2014-02-12 | 2014-06-11 | 博易智软（北京）技术股份有限公司 | Multidimensional data processing method and device |
Non-Patent Citations (2)
| Title |
|---|
| Mondrian中聚合表的应用;大圆那些事;《https://www.cnblogs.com/panfeng412/archive/2012/03/25/mondrian-aggregate-table.html》;20120325;全文 * |
| 数据仓库技术在银行信贷管理信息系统中的应用;贾东;《信息技术》;20030430;第27卷(第4期);38-40 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108268523A (en) | 2018-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9323809B2 (en) | System and methods for rapid data analysis | |
| CN108197296B (en) | Data storage method based on Elasticissearch index | |
| CN105653537B (en) | Paging query method and device for database application system | |
| CN105468651B (en) | Relational database data query method and system | |
| US9262454B2 (en) | Web page retrieval method and device | |
| CN108268523B (en) | Database aggregation processing method and device | |
| CN108241692B (en) | Data query method and device | |
| CN106933897B (en) | Data query method and device | |
| CN104484392B (en) | Query sentence of database generation method and device | |
| CN108255886B (en) | Evaluation method and device of recommendation system | |
| CN109388614A (en) | A kind of method, system and the equipment of catalogue file number quota | |
| CN106933907B (en) | Processing method and device for data table expansion indexes | |
| CN106649385B (en) | Data reordering method and device based on HBase database | |
| CN106933902B (en) | Data multidimensional free analysis query method and device | |
| CN112527824B (en) | Paging query method, paging query device, electronic equipment and computer-readable storage medium | |
| CN106708880B (en) | Topic associated word acquisition method and device | |
| CN109033248B (en) | Method and device for storing data record and method and device for inquiring data record | |
| CN104794237B (en) | web information processing method and device | |
| CN111125157B (en) | Query data processing method and device, storage medium and processor | |
| CN109992708B (en) | Method, device, equipment and storage medium for metadata query | |
| CN110968555B (en) | Dimension data processing method and device | |
| CN108170693B (en) | Hot word pushing method and device | |
| CN106933909B (en) | Multi-dimensional data query method and device | |
| CN106933933B (en) | Data table information processing method and device | |
| CN111400253B (en) | Statistical data query method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Cuigong Hotel, 76 Zhichun Road, Shuangyushu District, Haidian District, Beijing Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |