CN108932257B - Multi-dimensional data query method and device - Google Patents
Multi-dimensional data query method and device Download PDFInfo
- Publication number
- CN108932257B CN108932257B CN201710379694.8A CN201710379694A CN108932257B CN 108932257 B CN108932257 B CN 108932257B CN 201710379694 A CN201710379694 A CN 201710379694A CN 108932257 B CN108932257 B CN 108932257B
- Authority
- CN
- China
- Prior art keywords
- dimension
- combination
- index
- data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a device for inquiring multidimensional data, which are characterized in that a plurality of dimension index combinations are obtained by grouping the dimensions and indexes of the data, each dimension index combination corresponds to a dimension index data table, the dimension index data table is used for storing the data of the dimensions and the indexes contained in the corresponding dimension index combination, after a data inquiry request for inquiring the data corresponding to at least one target index in at least one target dimension is obtained, at least one target dimension index combination where the at least one target dimension is located is quickly determined by inquiring a prestored dimension information table and a combination definition table, at the moment, the data corresponding to each target index in the target dimension can be inquired only from the dimension index data table corresponding to the target dimension index combination, the method and the device are simple and convenient, the inquiry data amount is only related to the dimension contained in the target dimension index combination, and the method is irrelevant to other dimensions, so that the query data volume is greatly reduced, and the data query efficiency is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a device for querying multidimensional data.
Background
With the advent of the big data era, people often need to query data from multiple dimensions, and the query and analysis efficiency of the data is often influenced by the difference of data storage structures.
In the prior art, a data table is usually preset for data to be stored, and the data table is divided into a plurality of columns according to the dimension and index of the data to be stored, so that the obtained data to be stored is directly stored in the corresponding position in the data table.
However, as the number of dimensions for storing data increases, the data query amount of the data table of the existing storage structure increases exponentially, and the query efficiency is greatly reduced.
Disclosure of Invention
In view of the above problems, the present invention is proposed to provide a method and an apparatus for querying multidimensional data, which overcome the above problems or at least partially solve the above problems.
A method of querying multidimensional data, the method comprising:
acquiring a data query request for querying data corresponding to at least one target index under at least one target dimension;
inquiring a prestored dimension information table and a prestored combination definition table to determine at least one target dimension index combination where the at least one target dimension is located;
obtaining a prestored dimension index data table which corresponds to the at least one target dimension index combination one by one;
and querying data corresponding to the at least one target index under the corresponding target dimension from the acquired dimension index data table.
Preferably, the method further comprises:
generating a combination identifier corresponding to the dimension index combination one by one;
and generating the dimension information table by utilizing the corresponding relation between each dimension contained in each dimension index combination and the combination identification of the dimension index combination.
Preferably, the querying a pre-stored dimension information table and a combination definition table to determine at least one target dimension index combination where the at least one target dimension is located includes:
inquiring a prestored dimension information table to determine a target combination identifier corresponding to the at least one target dimension;
inquiring a prestored combination definition table to obtain target dimension index combinations in one-to-one correspondence with the target combination identifications, wherein the combination definition table comprises combination parameters for distinguishing all the dimension index combinations, and the combination parameters comprise combination identifications, combination names and storage unit identifications in one-to-one correspondence with the corresponding dimension index combinations;
the obtaining of the prestored dimension index data table corresponding to the at least one target dimension index combination one to one includes:
and performing data query on the storage unit corresponding to the obtained storage unit identifier to obtain a dimension index data table stored by the storage unit, wherein the dimension index data table comprises data of each index in at least one dimension contained in the corresponding dimension index combination.
Preferably, the method further comprises:
generating an index information table by utilizing the corresponding relation between each index contained in each dimension index combination and the combination identification of the dimension index combination;
creating a storage unit corresponding to each storage unit identifier according to the dimension information table, the index information table and a prestored combination definition table;
generating a dimension index data table of the dimension index combination by using at least one dimension and at least one index contained in the dimension index combination corresponding to the storage unit;
and storing the dimension index data table to the storage unit.
Preferably, the method further comprises:
obtaining a data storage request, wherein the data storage request carries indexes and dimensionalities of data to be stored;
inquiring the dimension information table and the index information table, and judging whether indexes and dimensions of the data to be stored belong to exist or not;
if yes, determining the index to which the data to be stored belongs and a dimension index combination corresponding to a dimension, and storing the data to be stored to a dimension index data table corresponding to the determined dimension index combination;
and if the dimension and the index of the data to be stored do not exist, taking the dimension and the index of the data to be stored as a new dimension index combination, sequentially adding a combination identifier, a combination name and a storage unit identifier of the new dimension index combination in the combination definition table, and updating the dimension information table and the index information table.
An apparatus for querying multidimensional data, the apparatus comprising:
the query request obtaining module is used for obtaining a data query request for querying data corresponding to at least one target index under at least one target dimension;
the combined query module is used for querying a prestored dimension information table and a prestored combined definition table so as to determine at least one target dimension index combination where the at least one target dimension is located;
the data table acquisition module is used for acquiring a prestored dimension index data table corresponding to the at least one target dimension index combination one by one;
and the data query module is used for querying data corresponding to the at least one target index under the corresponding target dimension from the acquired dimension index data table.
Preferably, the apparatus further comprises:
the first generation module is used for generating a combination identifier which corresponds to the dimension index combination one by one;
and the second generation module is used for generating the dimension information table by utilizing the corresponding relation between each dimension contained in each dimension index combination and the combination identifier of the dimension index combination.
Preferably, the combination query module includes:
the first query unit is used for querying a prestored dimension information table to determine a target combination identifier corresponding to the at least one target dimension;
the second query unit is used for querying a prestored combination definition table to obtain target dimension index combinations in one-to-one correspondence with the target combination identifications, wherein the combination definition table comprises combination parameters used for distinguishing all the dimension index combinations, and the combination parameters comprise combination identifications, combination names and storage unit identifications in one-to-one correspondence with the corresponding dimension index combinations;
the data table obtaining module is specifically configured to perform data query on the storage unit corresponding to the obtained storage unit identifier, and obtain a dimension index data table stored in the storage unit, where the dimension index data table includes data of each index in at least one dimension included in the corresponding dimension index combination.
Preferably, the apparatus further comprises:
the third generation module is used for generating an index information table by utilizing the corresponding relation between each index contained in each dimension index combination and the combination identifier of the dimension index combination;
the creating module is used for creating a storage unit corresponding to each storage unit identifier according to the dimension information table, the index information table and a prestored combination definition table;
a fourth generation module, configured to generate a dimension index data table of the dimension index combination by using data of each index in at least one dimension included in the dimension index combination corresponding to the storage unit;
and the storage module is used for storing the dimension index data table to the storage unit.
Preferably, the apparatus further comprises:
the storage request obtaining module is used for obtaining a data storage request, and the data storage request carries indexes and dimensionalities of data to be stored;
the query judging module is used for querying the dimension information table and the index information table and judging whether indexes and dimensions of the data to be stored belong to exist or not;
the first updating module is used for determining the index to which the data to be stored belongs and the dimension index combination corresponding to the dimension when the judgment result of the query judging module exists, and updating the dimension index data table corresponding to the dimension index combination by using the data to be stored;
and the second updating module is used for taking the dimension and the index of the data to be stored as a new dimension index combination when the judgment result of the query judging module is not existed, sequentially adding a combination identifier, a combination name and a storage unit identifier of the new dimension index combination in the combination definition table, and updating the dimension information table and the index information table.
By means of the technical scheme, the invention provides a method and a device for inquiring multi-dimensional data, the method and the device can acquire a plurality of dimensional index combinations by grouping the dimensions and indexes of the data, each dimensional index combination corresponds to one dimensional index data table, and the dimensional index data table only stores the data of the dimensions and indexes included in the corresponding dimensional index combination, so that after a data inquiry request for inquiring the data corresponding to at least one target index in at least one target dimension is acquired, the method and the device can quickly determine at least one target dimensional index combination in which the at least one target dimension is located by inquiring the pre-stored dimensional information table and combination definition table, at the moment, the data corresponding to each target index in the target dimension can be inquired only from the dimensional index data table corresponding to the target dimensional index combination, and the method and the device are simple and convenient, because the query data volume is only related to the dimension contained in the target dimension index combination and is not related to other dimensions, the query data volume is greatly reduced, and the data query efficiency is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart of a method for querying multidimensional data according to an embodiment of the present invention;
FIG. 2 is a flow chart of another multi-dimensional data query method provided by the embodiment of the invention;
FIG. 3 is a flow chart of another multi-dimensional data query method provided by the embodiment of the invention;
FIG. 4 is a block diagram illustrating a multi-dimensional data query apparatus according to an embodiment of the present invention;
FIG. 5 is a block diagram of another multi-dimensional data query device according to an embodiment of the present invention;
FIG. 6 is a block diagram illustrating an apparatus for querying multidimensional data according to an embodiment of the present invention;
FIG. 7 is a block diagram illustrating an apparatus for querying multidimensional data according to an embodiment of the present invention;
fig. 8 is a schematic diagram illustrating a hardware structure of a multi-dimensional data query apparatus according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Based on the analysis of the background art, the invention is described by taking an advertisement monitoring system as an example, in the application scenario, different dimensions such as originality, media, channels, operating systems and device types, and different indexes such as exposure, click, independent exposure and independent click can be set for data to be stored, and in practical application, the advertisement effect can be evaluated by checking the data of each index in different dimensions.
It should be noted that, in other application scenarios, corresponding other dimensions and indexes may be set, and are not limited to the dimensions and indexes listed above, and the scheme is described here only by taking the application scenario of advertisement monitoring as an example, and the application processes of the data processing schemes of other application scenarios are similar, and the present invention is not described any more.
In the prior art, all the dimensions and all the indexes are generally combined together to generate a data table, as shown in the following table 1:
TABLE 1
Based on the data storage structure shown in the data table, since it stores all the dimensions determined in one data table, this will result in the data volume per day being the product of the number of all the dimensions. Taking the dimensions in table 1 as an example, assuming that there are 100 creatives, 20 media, 10 channels, 20 operating systems, and 8 device types, the data amount per day is 100 × 20 × 10 × 20 × 8 — 3200000, and the data amount in less than one month has a problem of query performance, which affects the efficiency of each index in the query dimension.
In order to reduce the data volume and improve the data query efficiency, the invention provides a new data storage structure, which adopts the method of data storage of metadata management and dynamic data generation, meets the requirement of multi-dimensional data query, solves the query performance problem, provides the experience of user data query, and can refer to the following description of each embodiment.
As shown in fig. 1, which is a flowchart of an embodiment of a multi-dimensional data query method provided in the present invention, this embodiment mainly describes how to quickly and accurately query data, and specifically may include the following steps:
step S11, obtaining a data query request for querying data corresponding to at least one target index under at least one target dimension;
in practical applications, in order to know the usage of the product, various data in the usage of the product are usually analyzed, so as to determine the usage of the product according to the analysis result, and provide an improvement scheme for the product. For example, in order to understand the advertising effect of various creative advertisements, the advertisement click condition, the advertisement feedback condition of the user, the advertisement receiving condition of the electronic device, and the like are analyzed, so that the designer can adjust the advertisement content, the format, and the like according to the analysis result.
Based on this, in order to obtain the data under the above various conditions, a corresponding data query request may be initiated to the electronic device, and in practical applications, in order to quickly and accurately obtain the required data, the required data is generally queried according to the dimensions and/or indexes of the data, and therefore, the data query request may include at least one target index in at least one target dimension, that is, at this time, the user wishes to query the data corresponding to the at least one target index in the at least one target dimension.
Taking an advertisement monitoring system as an example, in order to know the advertisement effect of each creative advertisement, the dimension of the creative can be added to the data query request for transmission; if the effect of the advertisement in each channel needs to be compared, the dimension of the channel can be added into the data query request for transmission, and the like, even multiple dimensions such as originality, channel, media and the like can be added into the data query request, and specific indexes such as exposure, click and the like which need to be queried are added according to the characteristics of the dimension.
Therefore, the data query request can carry at least one target dimension for querying the data to be queried and at least one target index below the target dimension, and the content of the data query request can be determined according to the data analysis requirement. In addition, the present application does not limit the generation and output of the data query request.
Step S12, inquiring a prestored dimension information table and a prestored combination definition table, and determining at least one target dimension index combination where the at least one target dimension is located;
in the application, in order to improve the data query efficiency, a plurality of dimensions and indexes involved in data are divided into a plurality of dimension index combinations, and each dimension index combination comprises at least one dimension and at least one index. It should be noted that, the specific division manner of the dimension index combination is not limited in the present application, and may be implemented according to factors such as the dimension characteristic of the present application and the actual query requirement, and the present application is not described in detail herein.
Optionally, in the present application, one dimension index combination may include only one dimension and multiple indexes, so as to improve the data query efficiency to the maximum extent, but the number of dimensions included in the dimension index combination is not limited to this.
In addition, in practical applications, in order to identify each dimension index combination, a combination parameter, such as a combination identifier (e.g., a combination ID), a combination name, a storage unit identifier, etc., corresponding to each dimension index combination one to one may be generated.
Optionally, in this embodiment, a combination definition table may also be generated and stored according to the combination parameters of different dimension index combinations, as shown in table 2 below, but is not limited to the output manner of the combination definition table shown in table 2. The combination identifier may be a continuous number or letter, the storage unit identifier may be a continuous storage address, and the like, and it should be noted that there is no repeated combination parameter in the combination definition table, that is, each combination ID is unique, each combination name is also unique, and each storage unit identifier is also unique.
TABLE 2
| Combination ID | Name of combination | Memory cell identification |
| 1 | Creation of | C0001 |
| …… | …… | …… |
In this embodiment, the dimensions included in different dimension index combinations are usually different, and in order to facilitate searching for a corresponding dimension index combination according to the dimensions, the present application may associate a plurality of dimensions with any combination parameter of the dimension index combination where the dimension index combination is located, thereby generating a dimension information table. As shown in table 3 below, a corresponding relationship between each dimension and the combination ID of the dimension index combination in which the dimension is located is established.
TABLE 3
| Combination ID | Dimension (d) of |
| 1 | Creation of |
| 2 | Channel with a plurality of channels |
| …… | …… |
As can be seen from table 3 above, in practical application, the content and number of dimensions included in different dimension index combinations can be determined by the combination ID, and meanwhile, the combination ID of the dimension index combination where the target dimension is located can also be determined according to the target dimension to be queried, and then information such as the combination name, the storage unit identifier, and the like of the dimension index combination where the target dimension is located can be known by querying the combination definition table, which is very convenient.
Step S13, obtaining a prestored dimension index data table which is in one-to-one correspondence with the at least one target dimension index combination;
step S14, querying data corresponding to the at least one target indicator in the corresponding target dimension from the obtained dimension indicator data table.
In this embodiment, in order to improve the data query efficiency, for data corresponding to each dimension and index included in each dimension index data combination, one dimension index data table corresponding to the dimension index combination is generated, that is, each dimension index data table is only used to store data corresponding to the corresponding dimension index combination, as shown in table 4 below, but is not limited to the output manner of the dimension index data table shown in table 4.
TABLE 4
| Date | Creation of | Exposure method | Click on | Independent exposure | Independent clicking |
| 2016-9-23 | Large screen | 2 | 1 | 1 | 1 |
| …… | …… | …… | …… | …… | …… |
As shown in table 4, when storing the collected data, the obtained acquisition times of a group of data to be stored are usually stored correspondingly, for example, the acquisition times 2016-9-23 of the group of data to be stored, whose originality is large screen, exposure is 2, click is 1, independent exposure is 1, and independent click is 1, as shown in table 4, the group of data may be stored in a row, so that the user or the device can find the data to be queried by row, but not limited to the above listed storage manner.
Based on the analysis, when data of at least one index under one or more dimensions needs to be inquired, the required data can be quickly searched and found only by determining the corresponding dimension index data table. Therefore, the data size of the data query is only related to the dimension contained in the dimension index combination and is not related to other dimensions, the data size is greatly reduced, and the data query efficiency is improved.
In order to implement the data query scheme, in the application, a plurality of dimension index combinations are determined, one-to-one corresponding dimension index data tables are generated, and corresponding data can be filled in the dimension index data tables according to dimensions and indexes included in the dimension index data tables.
Before determining the dimension index combination, the data to be stored is usually subjected to dimension and index division, and then a dimension index data table and the like are generated according to the manner described in the above embodiment. Optionally, the present invention may set a plurality of corresponding dimensions for different application scenarios, where, for example, an advertisement monitoring application scenario, the set plurality of dimensions may include a specific creative of the advertisement, a media to which the advertisement belongs, a channel to be played, an operating system of an applicable device, a device type, and the like. The set contents of the multiple dimensions can be different for different application scenarios, and the invention is not described in detail herein.
Moreover, the obtained specific index data is usually different according to different application requirements in an application scenario, for example, in order to evaluate advertisement efficiency, data of indexes such as exposure, click, independent exposure, independent click, and the like may be obtained, but is not limited thereto.
It should be noted that in other application scenarios, the dimension and the index data to be collected will change accordingly, and the detailed description of the present application is omitted here.
Based on the above description, as shown in fig. 2, the present application further provides a flowchart of another embodiment of a method for querying multidimensional data, where the method may include:
step S21, determining multiple dimensions and multiple indexes of the data to be collected;
in practical application of this embodiment, which index data under which dimensions need to be acquired, that is, a data acquisition rule, may be determined according to characteristics and purposes of data to be acquired, so that the device can acquire data according to the data acquisition rule.
The data type, the content and the like of the data to be acquired, the content of the determined data acquisition rule, the output mode and the like are not limited.
Step S22, combining and dividing the plurality of dimensions and the plurality of indexes to generate a plurality of dimension index combinations;
in the method, the multiple dimensions can be divided according to each actual data analysis purpose, and the indexes needed by each dimension group are determined by combining the characteristics of the dimensions and the corresponding data analysis purposes, so that multiple dimension index combinations are generated.
Each dimension index combination comprises at least one dimension and at least one index. Optionally, in order to improve the efficiency of subsequently querying the target data, each dimension index combination may include, but is not limited to, one dimension and multiple indexes.
Step S23, determining a combination parameter corresponding to each dimension index combination, and generating a combination definition table indicating the combination parameter of each dimension index combination, wherein the combination parameter comprises a combination ID, a combination name and a storage unit identifier;
in practical applications, a combination name of a dimension index combination may be determined according to the dimensions included in the dimension index combination, and a plurality of consecutive numbers may be sequentially generated as a combination ID of each dimension index combination, as shown in table 1 above, but is not limited thereto.
Moreover, after the storage space of the data to be acquired is determined, the storage space can be divided into a plurality of storage units, and a corresponding number is written for each storage unit to serve as the storage unit identifier for identifying the storage position of the data corresponding to each dimension index combination. In this embodiment, the dimension index data table generated for the dimension index combination may be stored in the storage unit corresponding to the storage unit identifier, so that the corresponding dimension index data table is accurately obtained based on the storage unit identifier of the dimension index combination in the following, and then the required data is queried from the dimension index data table.
Step S24, generating a dimension information table indicating the dimensions contained in each dimension index combination and an index information table indicating the indexes contained in each dimension index combination;
optionally, because any combination parameter and the dimension index combination are in a one-to-one correspondence relationship, in this embodiment, any combination parameter may be used to represent the corresponding dimension index combination, so that the inclusion relationship between the dimension index combination and the dimension and index included in the combination parameter is used to establish the correspondence relationship between the combination parameter and the dimension and index, and thus a corresponding dimension information table and an index information table are generated, as shown in table 3 and table 5 below, the selected combination parameter may be a combination ID, but is not limited thereto.
TABLE 5
| Combination ID | Index (I) |
| 1 | Exposure method |
| 1 | Click on |
| …… | …… |
Therefore, after the dimensionality of the inquired data is determined, the combination ID corresponding to the dimensionality can be determined through the query table 3, and then the combination name corresponding to the combination ID is known through the query table 2, namely the dimension index combination to which the inquired dimensionality belongs can be known. Similarly, after the index to be queried is known, the corresponding combination ID can be known through the lookup table 5, and then the dimension index combination to which the index belongs can be known through the lookup table 2.
In this case, since the index types of different dimensions generally overlap or even are the same, in practical applications, the required data is not generally obtained only through the target index. Of course, if the data of the target index in each dimension needs to be queried, the above-mentioned method may be implemented.
Step S25, generating a dimension index data table corresponding to the dimension index combination by using the dimension and the index included in the same dimension index combination;
as shown in table 4 above, the dimension of the corresponding dimension index combination and each index type may be filled in the first row of the dimension index data table, and the collected specific values of each index under the corresponding dimension are filled in the corresponding cells according to the dividing manner, so as to implement data storage.
Step S26, after data to be stored are collected, a dimension information table, an index information table and a combination definition table are inquired, and the dimension and the storage unit corresponding to the dimension index combination where the index is located are determined;
step S27, storing the data to be stored to the corresponding position in the dimension index data table in the storage unit;
by combining the analysis, after data of indexes such as exposure, click, independent exposure, independent click and the like of the large-screen creative advertisement are obtained, the dimension is determined to be the dimension index combination of the large screen through inquiring the table, and then the storage unit of the dimension index data table is determined, so that the data are accurately stored to the corresponding cell of the dimension index data table.
Step S28, obtaining a data query request for querying data corresponding to at least one target index under at least one target dimension;
step S29, querying the dimension information table, and determining a target combination ID corresponding to the at least one target dimension;
step S210, inquiring the combination definition table to obtain a combination name and a storage unit identifier of the target dimension index combination corresponding to the target combination identifier one to one;
step S211, according to the storage unit identification, obtaining a dimension index data table of the target dimension index combination from the corresponding storage unit;
in practical application of this embodiment, the storage unit having the storage unit identifier may be determined first, and then the dimension index data table stored in the storage unit, that is, the dimension index data table corresponding to the target dimension index combination one to one, is obtained, where the dimension index data table includes data of each index in at least one dimension included in the target dimension index combination.
Step S212, inquiring data corresponding to the at least one target index under the corresponding target dimension from the acquired dimension index data table;
step S213, generating and outputting an analysis report based on the queried data.
According to the embodiment, a corresponding analysis algorithm can be determined to analyze the queried data according to the actual requirement of the queried data, so that a corresponding analysis report is generated according to the analysis result. The content of the analysis algorithm, the content of the analysis report, the output mode, and the like are not limited in the present application.
In summary, the present application adopts a new data storage scheme, that is, a plurality of dimensions and indexes of data are divided into a plurality of dimension index combinations, and a dimension index data table is set for each dimension index combination and is separately stored in a storage unit, and for collected data, which dimension index combination the collected data belongs to is determined, and then the collected data is updated to a corresponding dimension index data table.
Referring to fig. 3, a flowchart of another multi-dimensional data query method provided in the embodiment of the present invention is provided, and this embodiment may be an optional implementation manner for implementing data storage, but is not limited to the implementation manner described herein, and may be adaptively adjusted according to actual needs. The method provided by this embodiment may include:
step S31, obtaining a data storage request;
the data storage request carries indexes and dimensionalities of the data to be stored. In practical application, after the dimension index data table is generated, a data storage request can be initiated at any time, and the acquired data is stored in the dimension index data table of the corresponding storage unit.
Step S32, inquiring the dimension information table and the index information table, judging whether the index and the dimension of the data to be stored exist, if so, executing step S33; if not, go to step S35;
in practical application, because the dimension and the index to which the newly acquired data belongs may not exist before, the newly acquired data cannot be directly stored in the dimension index data table at this time, a new dimension index data table may need to be established, or the dimension and the index type in the existing dimension index data table may be updated, which is not specifically limited in the present application.
Step S33, determining the index of the data to be stored and the dimension index combination corresponding to the dimension;
the process of determining the corresponding dimension index combination by using the index and the dimension to which the data to be stored belongs may refer to the description of the corresponding parts of the above embodiments, and the present embodiment is not described in detail herein.
Step S34, updating the dimension index data table which is correspondingly stored by the dimension index combination and is determined by the data to be stored;
in combination with the above analysis, if the dimension and index to which the acquired data to be stored belongs already exist, it indicates that corresponding data already exists in the original dimension index data table, and at this time, the newly acquired data to be stored can be updated with the data corresponding to the corresponding dimension and index, that is, the dimension index data table is updated.
Step S35, taking the dimension and index of the data to be stored as a new dimension index combination;
step S36, sequentially adding the combination identifier, the combination name and the storage unit identifier of the new dimension index combination in the combination definition table, and correspondingly updating the dimension information table and the index information table;
the contents and the generation manner of the combination definition table, the dimension information table, and the index information table may be described in the corresponding parts of the above embodiments, and the present embodiment is not described in detail herein.
And step S37, aiming at the new dimension index combination, generating a corresponding dimension index information table in the storage unit corresponding to the newly added storage unit identifier, and storing the data to be stored to the corresponding position of the dimension index information table.
In this embodiment, for the new dimension index information table and the process of storing data in the dimension index information table, reference may be made to the descriptions of the corresponding parts of the above embodiments, and this embodiment is not described in detail here.
In summary, in this embodiment, when new data needs to be stored, based on the data storage structure provided in the present application, the new data can be directly updated into an existing corresponding dimension index data table, or a new dimension index data table is constructed in a new storage unit, which is very convenient and is beneficial to fast query of the data in a later period.
The structure of the multi-dimensional data query device provided by the invention will be described in terms of functional module composition.
Referring to fig. 4, a block diagram of a multi-dimensional data query apparatus according to an embodiment of the present invention is provided, where the apparatus may include:
a query request obtaining module 401, configured to obtain a data query request for querying data corresponding to at least one target indicator in at least one target dimension;
a combined query module 402, configured to query a pre-stored dimension information table and a combined definition table, and determine at least one target dimension index combination where at least one target dimension is located;
the generation manner of the dimension information table and the combination definition table may refer to the description of the corresponding part of the above method embodiment, and this embodiment is not described in detail here.
In this embodiment, multiple dimensions and multiple indexes of data that may be queried may be determined according to actual needs of data query and characteristics of each data dimension, and the multiple dimensions and the multiple indexes are divided according to different data query needs, so as to generate multiple dimension index combinations. It should be noted that, the present application does not limit the dimensions included in each dimension index combination, and the types and the numbers of the indexes.
A data table obtaining module 403, configured to obtain a dimension index data table in which at least one target dimension index combination is pre-stored in a one-to-one correspondence manner;
in this embodiment, a corresponding dimension index data table is generated for each dimension index combination, and is used to store data corresponding to the dimensions and indexes included in the dimension index combination, and store the dimension index data table in a separate storage unit, so that each storage unit only stores one dimension index data table.
The data query module 404 is configured to query, from the obtained dimension index data table, data corresponding to the at least one target index in a corresponding target dimension.
Optionally, as shown in fig. 5, the apparatus provided in the present application may further include:
a first generating module 405, configured to generate a combination identifier corresponding to the dimension index combination one to one;
in practical applications, for each dimension index combination, a corresponding combination parameter, such as a combination identifier, a combination name, a storage unit identifier, and the like, may be determined, and each of the combination parameters is unique, so that different dimension index combinations can be distinguished according to each combination parameter.
A second generating module 406, configured to generate a dimension information table by using a correspondence between each dimension included in each dimension index combination and a combination identifier of the dimension index combination;
optionally, the dimension information table including the other combination parameters and the dimensions may also be generated by using the corresponding relationship between the dimensions and the other combination parameters of the dimension index combination where the dimensions are located. It can be seen that the content included in the dimension information table is not limited in the present application.
A third generating module 407, configured to generate an index information table by using a correspondence between each index included in each dimension index combination and a combination identifier of the dimension index combination;
referring to table 5 above, the index information table may use a combination identifier to represent a relationship between each index and a corresponding dimension index combination, and certainly, other combination parameters may be used to represent the corresponding dimension index combination, and in general, the combination parameters in the dimension information table and the combination parameters in the index information table may be the same, but are not limited.
A creating module 408, configured to create a storage unit corresponding to each storage unit identifier according to the dimension information table, the index information table, and a pre-stored combination definition table;
the combination definition table may be generated by using the combination parameters of each dimension index combination, as shown in table 2 above, but is not limited thereto.
A fourth generating module 409, configured to generate a dimension index data table of a dimension index combination by using data of each index in at least one dimension included in the dimension index combination corresponding to the storage unit;
the storage module 410 is configured to store the dimension index data table in a storage unit.
Each dimension index combination only corresponds to one dimension index data table, and each dimension index data table is stored in a single storage unit.
Based on this, referring to FIG. 6, the above-mentioned combined query module 402 may include
A first querying unit 4021, configured to query a pre-stored dimension information table, and determine a target combination identifier corresponding to the at least one target dimension;
a second query unit 4022, configured to query a pre-stored combination definition table to obtain target dimension index combinations corresponding to the target combination identifiers one to one;
the combination definition table comprises combination parameters for distinguishing all dimension index combinations, and the combination parameters comprise combination identifications, combination names and storage unit identifications which are in one-to-one correspondence with the corresponding dimension index combinations;
the data table obtaining module 403 may specifically be configured to perform data query on the storage unit corresponding to the obtained storage unit identifier, and obtain a dimension index data table stored in the storage unit, where the dimension index data table includes data of each index in at least one dimension included in the corresponding dimension index combination.
Optionally, as shown in fig. 7, the apparatus provided in the present application may further include:
a storage request obtaining module 411, configured to obtain a data storage request, where the data storage request carries an index and a dimension to which data to be stored belongs;
the query judging module 412 is configured to query the dimension information table and the index information table, and judge whether indexes and dimensions of the data to be stored belong to exist;
a first updating module 413, configured to determine, when the determination result of the query determining module is present, a dimension index combination corresponding to the index and the dimension to which the data to be stored belongs, and update a dimension index data table corresponding to the dimension index combination by using the data to be stored;
a second updating module 414, configured to, when the determination result of the query determining module is that the data to be stored does not exist, use the dimension and the index to which the data to be stored belongs as a new dimension index combination, sequentially add a combination identifier, a combination name, and a storage unit identifier of the new dimension index combination to the combination definition table, and update the dimension information table and the index information table.
In summary, in this embodiment, a plurality of dimension index combinations are obtained by grouping the dimensions and indexes of the data, and a dimension index data table is generated for each dimension index combination and is separately stored, and is only used for storing the data corresponding to the dimensions and indexes included in the corresponding dimension index combination. Therefore, when data of at least one index under one or more dimensions needs to be inquired, the needed data can be quickly found only by finding the corresponding dimension index data table. Therefore, the data size of the data query is only related to the dimension contained in the dimension index combination and is not related to other dimensions, so that the data size is greatly reduced, and the data query efficiency is improved.
Referring to fig. 8, an embodiment of the present invention further provides a hardware structure diagram of a query apparatus for multidimensional data, where the hardware structure diagram may include: a processor 81, a memory 82, a communication interface 83, and a communication bus 84, etc. The processor 81, the memory 82 and the communication interface 83 may be connected by a communication bus 84.
The memory 82 can store the program of each functional module in the above-mentioned embodiment of the query device for multidimensional data described in the functional architecture, and the processor 81 reads and executes the corresponding program from the memory 82, thereby implementing the functions of data storage and query.
In this embodiment, the memory 82 may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The processor 81 may be a central processing unit CPU or the like, which may include a kernel to retrieve a corresponding program from the memory 82. The kernel may be set to be one or more than one, data storage, query analysis, and the like are realized by adjusting kernel parameters, and the specific implementation process may refer to the description of the corresponding part of the above method embodiment, which is not described herein again.
The invention also provides a computer program product, comprising program code means for performing the steps of the method of:
acquiring a data query request for querying data corresponding to at least one target index under at least one target dimension;
inquiring a prestored dimension information table and a prestored combination definition table, and determining at least one target dimension index combination where the at least one target dimension is located;
obtaining a prestored dimension index data table which corresponds to the at least one target dimension index combination one by one;
and querying data corresponding to the at least one target index under the corresponding target dimension from the acquired dimension index data table.
In summary, the data processing scheme provided by the invention provides a new data storage structure, and based on the data storage structure, in the proposed data query method, the queried data volume is only related to the dimension number contained in the dimension index combination where the dimension of the queried data is located, and is not related to other dimensions, so that the data query efficiency is greatly improved, and adverse effects on the data query performance after the dimension number is increased are avoided.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
The above are merely examples of the present invention, and are not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (6)
1. A method for querying multidimensional data, the method comprising:
acquiring a data query request for querying data corresponding to at least one target index under at least one target dimension; generating a combined identifier corresponding to the dimension index combination one by one;
generating a dimension information table by utilizing the corresponding relation between each dimension contained in each dimension index combination and the combination identification of the dimension index combination;
inquiring a prestored dimension information table and a prestored combination definition table to determine at least one target dimension index combination where the at least one target dimension is located; the inquiring the prestored dimension information table and the combination definition table to determine at least one target dimension index combination where the at least one target dimension is located includes:
inquiring a prestored dimension information table to determine a target combination identifier corresponding to the at least one target dimension;
inquiring a prestored combination definition table to obtain target dimension index combinations in one-to-one correspondence with the target combination identifications, wherein the combination definition table comprises combination parameters for distinguishing all the dimension index combinations, and the combination parameters comprise combination identifications, combination names and storage unit identifications in one-to-one correspondence with the corresponding dimension index combinations;
obtaining a prestored dimension index data table which corresponds to the at least one target dimension index combination one by one; the obtaining of the prestored dimension index data table corresponding to the at least one target dimension index combination one to one includes:
performing data query on a storage unit corresponding to the obtained storage unit identifier to obtain a dimension index data table stored by the storage unit, wherein the dimension index data table comprises data of each index in at least one dimension contained in a corresponding dimension index combination;
and querying data corresponding to the at least one target index under the corresponding target dimension from the acquired dimension index data table.
2. The method of claim 1, further comprising:
generating an index information table by utilizing the corresponding relation between each index contained in each dimension index combination and the combination identification of the dimension index combination;
creating a storage unit corresponding to each storage unit identifier according to the dimension information table, the index information table and a prestored combination definition table;
generating a dimension index data table of the dimension index combination by using at least one dimension and at least one index contained in the dimension index combination corresponding to the storage unit;
and storing the dimension index data table to the storage unit.
3. The method of claim 2, further comprising:
obtaining a data storage request, wherein the data storage request carries indexes and dimensionalities of data to be stored;
inquiring the dimension information table and the index information table, and judging whether indexes and dimensions of the data to be stored belong to exist or not;
if yes, determining the index to which the data to be stored belongs and a dimension index combination corresponding to a dimension, and storing the data to be stored to a dimension index data table corresponding to the determined dimension index combination;
and if the dimension and the index of the data to be stored do not exist, taking the dimension and the index of the data to be stored as a new dimension index combination, sequentially adding a combination identifier, a combination name and a storage unit identifier of the new dimension index combination in the combination definition table, and updating the dimension information table and the index information table.
4. An apparatus for querying multidimensional data, the apparatus comprising:
the query request obtaining module is used for obtaining a data query request for querying data corresponding to at least one target index under at least one target dimension;
the first generation module is used for generating combination identifiers which correspond to the dimension index combinations one by one;
the second generation module is used for generating a dimension information table by utilizing the corresponding relation between each dimension contained in each dimension index combination and the combination identification of the dimension index combination;
the combined query module is used for querying a prestored dimension information table and a prestored combined definition table so as to determine at least one target dimension index combination where the at least one target dimension is located; the combined query module comprises: the first query unit is used for querying a prestored dimension information table to determine a target combination identifier corresponding to the at least one target dimension;
the second query unit is used for querying a prestored combination definition table to obtain target dimension index combinations in one-to-one correspondence with the target combination identifications, wherein the combination definition table comprises combination parameters used for distinguishing all the dimension index combinations, and the combination parameters comprise combination identifications, combination names and storage unit identifications in one-to-one correspondence with the corresponding dimension index combinations;
the data table acquisition module is used for acquiring a prestored dimension index data table corresponding to the at least one target dimension index combination one by one; the data table acquisition module is specifically configured to perform data query on a storage unit corresponding to the obtained storage unit identifier to obtain a dimension index data table stored in the storage unit, where the dimension index data table includes data of each index in at least one dimension included in a corresponding dimension index combination;
and the data query module is used for querying data corresponding to the at least one target index under the corresponding target dimension from the acquired dimension index data table.
5. The apparatus of claim 4, further comprising:
the third generation module is used for generating an index information table by utilizing the corresponding relation between each index contained in each dimension index combination and the combination identifier of the dimension index combination;
the creating module is used for creating a storage unit corresponding to each storage unit identifier according to the dimension information table, the index information table and a prestored combination definition table;
a fourth generation module, configured to generate a dimension index data table of the dimension index combination by using data of each index in at least one dimension included in the dimension index combination corresponding to the storage unit;
and the storage module is used for storing the dimension index data table to the storage unit.
6. The apparatus of claim 5, further comprising:
the storage request obtaining module is used for obtaining a data storage request, and the data storage request carries indexes and dimensionalities of data to be stored;
the query judging module is used for querying the dimension information table and the index information table and judging whether indexes and dimensions of the data to be stored belong to exist or not;
the first updating module is used for determining the index to which the data to be stored belongs and the dimension index combination corresponding to the dimension when the judgment result of the query judging module exists, and updating the dimension index data table corresponding to the dimension index combination by using the data to be stored;
and the second updating module is used for taking the dimension and the index of the data to be stored as a new dimension index combination when the judgment result of the query judging module is not existed, sequentially adding a combination identifier, a combination name and a storage unit identifier of the new dimension index combination in the combination definition table, and updating the dimension information table and the index information table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710379694.8A CN108932257B (en) | 2017-05-25 | 2017-05-25 | Multi-dimensional data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710379694.8A CN108932257B (en) | 2017-05-25 | 2017-05-25 | Multi-dimensional data query method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108932257A CN108932257A (en) | 2018-12-04 |
| CN108932257B true CN108932257B (en) | 2021-10-08 |
Family
ID=64450587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710379694.8A Active CN108932257B (en) | 2017-05-25 | 2017-05-25 | Multi-dimensional data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108932257B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109614402B (en) * | 2018-12-11 | 2020-09-29 | 京东数字科技控股有限公司 | Multidimensional data query method and device |
| CN109635075B (en) * | 2018-12-11 | 2023-02-07 | 广州市西美信息科技有限公司 | Method and device for marking word-dividing marks on text contents |
| CN109783585A (en) * | 2018-12-27 | 2019-05-21 | 益萃网络科技（中国）有限公司 | Multidimensional data rendering method, device, computer equipment and readable storage medium storing program for executing |
| CN109902100A (en) * | 2019-01-31 | 2019-06-18 | 平安科技（深圳）有限公司 | Report form inquiring method, device and storage medium |
| CN109947838A (en) * | 2019-03-26 | 2019-06-28 | 中国联合网络通信集团有限公司 | A kind of method and device of storing data |
| CN110413634B (en) * | 2019-06-27 | 2022-03-29 | 北京奇艺世纪科技有限公司 | Data query method, system, device and computer readable storage medium |
| CN112632061A (en) * | 2020-12-03 | 2021-04-09 | 海腾保险代理有限公司 | Multidimensional data storage method and device |
| CN112559914B (en) * | 2020-12-21 | 2024-05-14 | 北京搜房科技发展有限公司 | Index data display method and device |
| CN113782162A (en) * | 2021-03-26 | 2021-12-10 | 北京京东拓先科技有限公司 | Method, device, equipment and storage medium for allocating inquiry fee |
| CN113393190B (en) * | 2021-06-10 | 2023-12-05 | 北京京东振世信息技术有限公司 | Warehouse information processing method and device, electronic equipment and readable medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010087782A2 (en) * | 2009-01-31 | 2010-08-05 | Eng Lee Soh | Dynamic multidimensional knowledge clustering, management and representation system |
| CN106528787A (en) * | 2016-11-09 | 2017-03-22 | 合网络技术（北京）有限公司 | Mass data multi-dimensional analysis-based query method and device |
| CN106557498A (en) * | 2015-09-25 | 2017-04-05 | 北京国双科技有限公司 | Date storage method and device and data query method and apparatus |
| CN106682180A (en) * | 2016-12-29 | 2017-05-17 | 广州华多网络科技有限公司 | Data inquiry method and device |
-
2017
- 2017-05-25 CN CN201710379694.8A patent/CN108932257B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010087782A2 (en) * | 2009-01-31 | 2010-08-05 | Eng Lee Soh | Dynamic multidimensional knowledge clustering, management and representation system |
| CN106557498A (en) * | 2015-09-25 | 2017-04-05 | 北京国双科技有限公司 | Date storage method and device and data query method and apparatus |
| CN106528787A (en) * | 2016-11-09 | 2017-03-22 | 合网络技术（北京）有限公司 | Mass data multi-dimensional analysis-based query method and device |
| CN106682180A (en) * | 2016-12-29 | 2017-05-17 | 广州华多网络科技有限公司 | Data inquiry method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108932257A (en) | 2018-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108932257B (en) | Multi-dimensional data query method and device | |
| CN109561326B (en) | Data query method and device | |
| US10534753B2 (en) | Caseless file lookup in a distributed file system | |
| US9305176B2 (en) | Database generation from a spreadsheet | |
| CN110413634B (en) | Data query method, system, device and computer readable storage medium | |
| CN108205577B (en) | Array construction method, array query method, device and electronic equipment | |
| CN108399088A (en) | Page display method, user terminal, page server and style configuration server | |
| US9235613B2 (en) | Flexible partitioning of data | |
| CN110019111B (en) | Data processing method, data processing device, storage medium and processor | |
| CN109299352B (en) | Method and device for updating website data in search engine and search engine | |
| CN106933907B (en) | Processing method and device for data table expansion indexes | |
| CN114328632A (en) | User data analysis method and device based on bitmap and computer equipment | |
| CN109388644B (en) | Data updating method and device | |
| CN109697234B (en) | Multi-attribute information query method, device, server and medium for entity | |
| CN110019357B (en) | Database query script generation method and device | |
| CN113849524B (en) | Data processing method and device | |
| US9230011B1 (en) | Index-based querying of archived data sets | |
| CN110968679A (en) | Data query method and device | |
| CN114741392A (en) | Data query method and device, electronic equipment and storage medium | |
| CN110598072B (en) | Feature data aggregation method and device | |
| CN110597849B (en) | Data query method and device | |
| CN110019507B (en) | Data synchronization method and device | |
| CN112162951A (en) | Information retrieval method, server and storage medium | |
| CN111858609A (en) | Fuzzy query method and device for block chain | |
| CN106776652B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100080 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Cuigong Hotel, 76 Zhichun Road, Shuangyushu District, Haidian District, Beijing Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |