CN113553341A - Multidimensional data analysis method, multidimensional data analysis device, multidimensional data analysis equipment and computer readable storage medium - Google Patents
Multidimensional data analysis method, multidimensional data analysis device, multidimensional data analysis equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113553341A CN113553341A CN202110849478.1A CN202110849478A CN113553341A CN 113553341 A CN113553341 A CN 113553341A CN 202110849478 A CN202110849478 A CN 202110849478A CN 113553341 A CN113553341 A CN 113553341A
- Authority
- CN
- China
- Prior art keywords
- dimension
- target
- data
- data cube
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Abstract
The embodiment of the invention relates to the technical field of data processing, and discloses a multidimensional data analysis method, which comprises the following steps: receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried; determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes; inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result; and returning the query result to the user. Through the mode, the embodiment of the invention realizes the flexibility of query and improves the query efficiency.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a multidimensional data analysis method, a multidimensional data analysis device, multidimensional data analysis equipment and a computer readable storage medium.
Background
Currently, as digitization advances, the amount of data accumulated continues to grow. The need for multidimensional analysis of mass data is becoming increasingly valuable. However, the conventional OLAP (online analytical processing) technology has difficulty in meeting the demand for high efficiency and easy use. In the data analysis system in the prior art, generally, on-line online analysis and processing by OLAP can perform multidimensional data analysis and query, and the result is mainly calculated by MapReduce and imported to, for example, MySQL, Oracle, SQL Server, so as to realize interactive query of data indexes. The other is to realize user interactive data analysis through an SQL interface by relying on a multidimensional analysis and calculation tool, such as Kylin, Durid, and the like, and SQL needs to be written for multidimensional analysis.
The inventor finds that the existing multidimensional data analysis method has low execution efficiency and is difficult to realize self-service multidimensional analysis.
Disclosure of Invention
In view of the foregoing problems, embodiments of the present invention provide a multidimensional data analysis method, apparatus, device, and computer-readable storage medium, which are used to solve the technical problems in the prior art that most of self-help analysis of users cannot be realized, and the execution efficiency is low.
According to an aspect of the embodiments of the present invention, there is provided a multidimensional data analysis method applied to a multidimensional data analysis apparatus, the method including:
receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried;
determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes;
inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
In an optional manner, the determining a target data cube and a structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analyzing in the target data cube according to the structured query statement to obtain a query result includes:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative approach, the query result includes a result list; after the query result is returned to the user, the method includes:
receiving paging and sequencing requests of users;
paging and sorting the data results in the result list according to the paging and sorting request to obtain a processed result list;
and returning the processed result list to the user.
In an optional manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing the source data to obtain a fact table and a plurality of dimension tables; the fact table comprises a foreign key of the dimension table and fact data; the dimension table comprises at least one dimension table dimension;
acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table and the dimension table to construct a data model corresponding to each business theme;
configuring a dimension column and a measurement column on the data model according to the dimension and index demand information;
determining index values of the measurement columns under the combination of all the dimension columns according to the dimension table corresponding to the business theme and the associated fact table to obtain a data cube;
and respectively mapping the data cube, the dimension column and the measurement column into a theme, a dimension and an index defined by the non-relational database.
According to another aspect of the embodiments of the present invention, there is provided a multidimensional data analysis apparatus including:
the receiving module is used for receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried;
the determining module is used for determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes;
the analysis module is used for inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and the return module is used for returning the query result to the user.
In an optional manner, the query analyzing in the target data cube according to the structured query statement to obtain a query result includes:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an optional manner, the determining a target data cube and a structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
According to another aspect of the embodiments of the present invention, there is provided a multidimensional data analysis apparatus including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation of the multidimensional data analysis method.
According to another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which at least one executable instruction is stored, and when the executable instruction is executed on a multidimensional data analysis device, the multidimensional data analysis device executes the operations of the multidimensional data analysis method.
According to the embodiment of the invention, the business data is used for constructing the data cube according to the business theme, and the business dimension column and the measurement column in the data cube are determined according to the dimension and the index corresponding to the business theme, so that the dimension and the index of the business layer can be combined in a self-service manner according to the query request input by the user to query and analyze in the data cube, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart illustrating a multidimensional data analysis method provided by an embodiment of the invention;
FIG. 2 is a schematic structural diagram of a multidimensional data analysis device provided by an embodiment of the invention;
fig. 3 is a schematic structural diagram of a multidimensional data analysis device provided by the embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein.
First, technical terms appearing in the embodiments of the present invention are explained.
Data cube (cube), a multidimensional space constructed by dimensions, contains all the basic data (source data) to be analyzed, and all the aggregate data operations are performed on the data cube. The data cube is only an image of the multidimensional model, and the multidimensional model is not limited to a three-dimensional model and can be combined with more dimensions.
Dimension: is an angle for observing data, one dimension corresponds to multiple dimension tables.
Measuring value: the data presented, i.e. the index, is analyzed. A multidimensional analysis of the metric columns may be performed.
Fact table: the central table of the dimension database is called the fact table. Its rows represent facts, the central content of which is a measure of different instances of an activity or event; it stores both the fact value and the foreign key of the dimension table, and all the data for analysis are finally from the fact table.
Dimension table: a dimension table puts facts into the context, which represents things such as time, product, customer, and location, as can be a time dimension table, product dimension table, customer dimension table, location dimension table, and the like.
SQL (structured Query language), abbreviated SQL, refers to a structured Query language, and is a database Query and programming language for accessing data and querying, updating, and managing a relational database system.
Fig. 1 is a flowchart illustrating a multidimensional data analysis method provided by an embodiment of the present invention, which is executed by a multidimensional data analysis apparatus. The multidimensional data analysis device can be a computer device, a terminal, a distributed device and the like. As shown in fig. 1, the method comprises the steps of:
step 110: a query request of a user is received.
In the embodiment of the present invention, the query request includes a subject, a dimension, and an index of the service data. The query request may be input by a user in a preset query interface. The query request also includes a business dimension filter term, which is used to determine a business logic label.
Step 120: determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, and the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes.
And determining a corresponding target data cube according to the service theme in the query request. And the target data cube is obtained by processing the source data in advance according to the requirements of the business theme on the dimensionality and the index and then constructing.
In the embodiment of the invention, the multidimensional data analysis device comprises a plurality of data cubes, and each data cube corresponds to one business theme. And all the data cubes are obtained by processing and constructing source data in advance according to the requirements of the business theme on the dimensionality and the index. The target data cube is one of a plurality of data cubes.
The structured query statement may be an SQL statement, and the SQL statement includes target dimension column information and target measure column information.
In the embodiment of the present invention, determining a target data cube and a structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube; the data cube with the service theme matched with the service theme of the query request indicates that the same service theme exists or a certain mapping relation exists, for example, a service theme field in the query request is the same as a service theme field corresponding to the data cube in the multidimensional data analysis device, or an ID corresponding to the theme in the query request is determined, and the ID corresponding to the data cube in the multidimensional data analysis device is determined according to the ID, so that a target data cube is determined;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In the embodiment of the invention, the process of constructing the cube of data is as follows:
s1: source data is collected. Acquiring source data according to a preset data standard; the preset data specification may be a corresponding setting performed by a person skilled in the art according to a service scenario.
S2: and processing the source data to obtain a fact table and a dimension table. The fact table comprises the external keys of the dimension table and fact data; the dimension table includes at least one dimension table dimension. The process of processing the source data comprises the processes of extracting, verifying and converting the data. Specifically, when data in a certain fact table is acquired, a numerical value in the fact table is extracted, the value is checked, for example, whether obvious error data exists or not is checked, and then the source data is converted into the format of the required fact table and dimension table according to a preset conversion rule. The dimension table is multiple, and may be a date dimension table, a user type dimension table, or the like.
S3: obtaining each service theme required by a user and dimension and index requirement information corresponding to the service theme. The business themes required by the user and the English dimension and index requirements of the business themes are set according to actual requirements, and embodiments of the present invention are not particularly limited. It will be appreciated that the business topic corresponds to a plurality of dimensions, which correspond to dimensions in a respective dimension table.
S4: and associating the fact table and the dimension table to construct a data model corresponding to the business theme. Specifically, the fact table and the dimension table may be associated according to a business theme to obtain a data model. For example, for the communication field, the service theme may include a user theme, a charging theme, a video polyphonic ringtone theme, and the like; for a user theme, a fact table relevant to the user theme and at least one dimension table relevant to the user theme in a plurality of dimension tables can be obtained, and the relevant fact table and the at least one dimension table are associated to form a data model corresponding to the user theme; for the charging subject, a fact table related to the charging subject and at least one dimension table related to the charging subject in the dimension tables can be obtained, the fact table and the dimension table are associated to form a data model corresponding to the charging subject, and the like, so that a plurality of data models facing different service subjects are constructed.
S4: and configuring a dimension column and a measurement column on the data model according to the dimension and index demand information. Taking a user theme as an example, the dimensions to be configured include dimensions such as date, company, client, login type, user type, channel, province and the like, the measurement includes the number of users, browsing times and the like, index values of the measurements such as the number of users, browsing times and the like are counted under each dimension according to a dimension table corresponding to the user theme and an associated fact table, and the index values can be obtained by inquiring corresponding fact values from the fact table and calculating. The method comprises the steps that the dimension and the measurement configured on a data model are determined according to the requirements of a business theme on the dimension and the measurement, the requirements of the business theme on the dimension and the measurement are set by a user according to a specific business theme, and a target dimension column and a target measurement column are configured on the data model according to the dimension and the measurement requirement information, a dimension table and a fact table corresponding to the business theme by obtaining the dimension and the measurement requirement information corresponding to the business theme. Specifically, a Kylin multidimensional analysis engine can be relied on, and a dimension field and a measurement field of the data model are set according to the dimension and index requirements of a user theme. Apache Kylin is an open-source, distributed analytical data warehouse, providing SQL query interface and multidimensional analysis (OLAP) capability over Hadoop/Spark to support very large scale data.
S5: and determining index values of the measurement columns under the combination of all the dimension columns according to the dimension table corresponding to the service theme and the associated fact table to obtain the data cube.
When a data Cube is constructed by relying on a Kylin multidimensional analysis engine, taking the requirement of a user theme on dimension and measurement (index) as an example, data (date), company (company), client (client), login _ type (login type), user _ type (user type), channel (channel) and province are correspondingly set as a dimension column of the data Cube, and fields such as user _ id (number of users) and pv (browsing times) are used as a measurement column of the data Cube. The combination of the various dimension columns constitutes a dimension of the data cube. In the process of constructing the Cube, index values of the measurement columns under the combination of all the dimension columns are calculated and stored, namely, several dimension columns of 'date, company, client, logic _ type, user _ type, channel and provision' are respectively combined to obtain corresponding measurement columns and values corresponding to the measurement columns, and the values corresponding to the measurement columns are stored in corresponding fact tables. And at this moment, the establishment of the Cube configuration of the multidimensional data is completed, and the second-level multidimensional analysis can be performed on the data by compiling SQL query. The target data cube is any one of the multidimensional data cubes.
S6: and respectively mapping the data cube, the dimension column and the measurement column into a theme, a dimension and an index defined by the non-relational database. Further, after the data cube is constructed, mapping the target data cube, the target dimension column and the target metric column to a service theme, a service dimension and a service index respectively:
in the embodiment of the invention, after the data Cube is constructed, the Cube is abstractly defined in the non-relational database as a business theme: because a business theme corresponds to a data Cube, the data Cube is defined in an abstract way as a theme object, and the theme object comprises the following attributes: the theme ID, the theme name, the Chinese name, the theme SQL, the theme sequence Index, the theme description and the like enable the business theme objects inquired by the user through the interface to be in one-to-one correspondence with the Cube. Wherein, the non-relational database is a NoSQL database.
In the embodiment of the invention, after the data Cube is constructed, the dimension of the Cube of the abstract definition is the dimension of the business theme: defining each dimension of the Cube as a dimension object of the business theme, wherein the dimension object comprises the following attributes: dimension ID, dimension name, dimension Chinese, dimension column, dimension value, dimension order Index, dimension description, and the like. The business topic dimensions of the user interface query correspond to the dimension columns of the data cube. For example, if the user selects the query of "province" under the user theme on the interface, the "province" is the user theme dimension corresponding to the user interface, and the dimension column "provision" column corresponding to the data cube.
In the embodiment of the invention, after the data Cube is constructed, the measurement column of the Cube is abstractly defined as an index of a service theme: the measure column of the definition data Cube is an index of a business theme, and an index object comprises the following attributes: index ID, Index name, Index Chinese, measure SQL, filter SQL, Index order Index, Index caliber description, SQL pseudo code, Index description, etc. With this arrangement, the index input by the interface is made to correspond to the calculation result of the measure column in the data cube. For example, the user selects an "active user number" index of the user topic on the interface, and the corresponding measurement column SQL of the data cube is Count (discontinuity user _ id). The filtering SQL is used for distinguishing the service logic label of the Cube, for example, identifying the user type as active or newly added.
Through the arrangement, the data cube, the dimension column of the data cube and the measurement column of the data cube are mapped to the service theme, the service dimension and the service index defined by the configurable NoSQL (non-relational database). That is, the user inputs a query request including a business theme, a business dimension and a business index on the interface, so that the corresponding data cube, the dimension column of the data cube and the measurement column of the data cube can be determined, and the corresponding structured query statement is generated according to the target dimension column information and the target measurement column information. The structured query statement is a structured request statement which is statistically analyzed according to the target dimension column and the target measure column in the target data cube.
The generation process of the structured query statement comprises the following steps: and generating a From section of the SQL by using the target data cube corresponding to the business theme. For example, the Cube of the query object is user _ subject, and the From segment corresponding to the generated SQL statement is From user _ subject; and generating a Select section and a Group by section of the SQL by using the target dimension column of the target data cube corresponding to the service dimension of the query. For example, the service dimension of the query is Company name and Province, the corresponding dimension in the target data cube is Company and Provision, the Select section corresponding to generating the SQL statement is Select Company and Provision, and the Group by section generating the SQL statement is Group by Company and Provision; and generating SQL statistical values by using the target measurement columns of the target data cubes corresponding to the queried business indexes. For example, the service index is the number of users, the metric column of the corresponding target data cube is user _ id, the statistic value of the corresponding generated SQL statement is count (discontinuity user _ id), and the generated SQL segments are assembled into a complete SQL statement, so as to obtain the structured query statement.
Step 130: and inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result.
After the structured query statement is determined, the structured query statement is executed in a data analysis engine, and a target dimension column and a target measurement column in a target data cube can be determined according to the target dimension column information and the target measurement column information, so that a dimension value of the target dimension column and an index value of the target dimension column under the target dimension column are determined. Specifically, the index value of the target measure column under the target dimension column is obtained according to the fact table query of the structured query statement in the target data cube.
And generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In the embodiment of the invention, the data structure of the query result is in a semi-structured JSON data format and comprises dimension column header information, index column header information and a data list. The dimension column header information comprises dimension information of the query, including a dimension name, a Chinese name, a dimension ID, a dimension Index and the like; the Index header information comprises Index information of the query, including a brand name, a Chinese name, an Index ID, an Index and the like; the result list comprises data results of the query, and each result object comprises all dimension values and index values of the query.
Step 140: and returning the query result to the user.
When a user inquires a plurality of service indexes, the inquiry result comprises inquiry results corresponding to the service indexes.
In the embodiment of the invention, paging and sorting requests of a user for query results are received, and sorting and paging are carried out on a result list in the returned query results according to the paging and sorting requests. Specifically, paging and sorting requests of a user are received; paging and sorting the data results in the result list according to the paging and sorting request to obtain a processed result list; and returning the processed result list to the user.
In the embodiment of the present invention, the implementation of steps 110 to 130 is encapsulated by an SDK (generally referred to as a software development kit, which is a collection of development tools used by some software engineers to establish application software for a specific software package, a software framework, a hardware platform, an operating system, and the like), and a unique data query interface is exposed to the outside, so that a data query manner is unified and standardized, and all systems called for data query are kept consistent.
According to the embodiment of the invention, the business data is constructed into the data Cube according to the business theme, the business dimension column and the measurement column in the data Cube are determined according to the dimension and the index corresponding to the business theme, and the multidimensional analysis data model Cube is abstractly defined into the theme, the dimension and the index, so that the dimension and the index of the business layer can be combined in a self-service manner according to the query request input by the user to query and analyze in the data Cube, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
Fig. 2 is a schematic structural diagram of a multidimensional data analysis device provided in an embodiment of the present invention. As shown in fig. 2, the apparatus 200 includes:
a receiving module 210, configured to receive a query request of a user;
a determining module 220, configured to determine a target data cube and a structured query statement according to the query request;
the analysis module 230 is configured to perform query analysis in the target data cube according to the structured query statement to obtain a query result;
a returning module 240, configured to return the query result to the user.
In an optional manner, the determining a target data cube and a structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analyzing in the target data cube according to the structured query statement to obtain a query result includes:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative approach, the query result includes a result list; after the query result is returned to the user, the method includes:
receiving paging and sequencing requests of users;
paging and sorting the data results in the result list according to the paging and sorting request to obtain a processed result list;
and returning the processed result list to the user.
In an optional manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing the source data to obtain a fact table and a plurality of dimension tables; the fact table comprises a foreign key of the dimension table and fact data; the dimension table comprises at least one dimension table dimension;
acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table and the dimension table to construct a data model corresponding to each business theme;
configuring a dimension column and a measurement column on the data model according to the dimension and index demand information;
determining index values of the measurement columns under the combination of all the dimension columns according to the dimension table corresponding to the business theme and the associated fact table to obtain a data cube;
and respectively mapping the data cube, the dimension column and the measurement column into a theme, a dimension and an index defined by the non-relational database.
The working process of the multidimensional data analysis device in the embodiment of the invention is consistent with the specific steps of the multidimensional data analysis method, and details are not repeated here.
According to the embodiment of the invention, the business data is constructed into the data Cube according to the business theme, the business dimension column and the measurement column in the data Cube are determined according to the dimension and the index corresponding to the business theme, and the multidimensional analysis data model Cube is abstractly defined into the theme, the dimension and the index, so that the dimension and the index of the business layer can be combined in a self-service manner according to the query request input by the user to query and analyze in the data Cube, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
Fig. 3 is a schematic structural diagram of a multidimensional data analysis device according to an embodiment of the present invention, and a specific implementation of the multidimensional data analysis device is not limited in the specific embodiment of the present invention.
As shown in fig. 3, the multidimensional data analysis apparatus may include: a processor (processor)302, a communication Interface 304, a memory 306, and a communication bus 308.
Wherein: the processor 302, communication interface 304, and memory 306 communicate with each other via a communication bus 308. A communication interface 304 for communicating with network elements of other devices, such as clients or other servers. The processor 302 is configured to execute the program 310, and may specifically perform the relevant steps in the embodiment of the multidimensional data analysis method described above.
In particular, program 310 may include program code comprising computer-executable instructions.
The processor 302 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the present invention. The multidimensional data analysis device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 306 for storing a program 310. Memory 306 may comprise high-speed RAM memory and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
Specifically, the program 310 may be invoked by the processor 302 to cause the multidimensional data analysis device to perform the following operations:
receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried;
determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes;
inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
In an optional manner, the determining a target data cube and a structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analyzing in the target data cube according to the structured query statement to obtain a query result includes:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative approach, the query result includes a result list; after the query result is returned to the user, the method includes:
receiving paging and sequencing requests of users;
paging and sorting the data results in the result list according to the paging and sorting request to obtain a processed result list;
and returning the processed result list to the user.
In an optional manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing the source data to obtain a fact table and a plurality of dimension tables; the fact table comprises a foreign key of the dimension table and fact data; the dimension table comprises at least one dimension table dimension;
acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table and the dimension table to construct a data model corresponding to each business theme;
configuring a dimension column and a measurement column on the data model according to the dimension and index demand information;
determining index values of the measurement columns under the combination of all the dimension columns according to the dimension table corresponding to the business theme and the associated fact table to obtain a data cube;
and respectively mapping the data cube, the dimension column and the measurement column into a theme, a dimension and an index defined by the non-relational database.
According to the embodiment of the invention, the business data is used for constructing the data cube according to the business theme, and the business dimension column and the measurement column in the data cube are determined according to the dimension and the index corresponding to the business theme, so that the dimension and the index of the business layer can be combined in a self-service manner according to the query request input by the user to query and analyze in the data cube, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
An embodiment of the present invention provides a computer-readable storage medium, where the storage medium stores at least one executable instruction, and when the executable instruction runs on a multidimensional data analysis device, the multidimensional data analysis device is enabled to execute a multidimensional data analysis method in any method embodiment described above.
The executable instructions may be specifically configured to cause the multidimensional data analysis device to perform the following operations:
receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried;
determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes;
inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
In an optional manner, the determining a target data cube and a structured query statement according to the query request includes:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
In an optional manner, the query analyzing in the target data cube according to the structured query statement to obtain a query result includes:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
In an alternative approach, the query result includes a result list; after the query result is returned to the user, the method includes:
receiving paging and sequencing requests of users;
paging and sorting the data results in the result list according to the paging and sorting request to obtain a processed result list;
and returning the processed result list to the user.
In an optional manner, before the receiving the query request of the user, the method includes:
collecting source data;
processing the source data to obtain a fact table and a plurality of dimension tables; the fact table comprises a foreign key of the dimension table and fact data; the dimension table comprises at least one dimension table dimension;
acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table and the dimension table to construct a data model corresponding to each business theme;
configuring a dimension column and a measurement column on the data model according to the dimension and index demand information;
determining index values of the measurement columns under the combination of all the dimension columns according to the dimension table corresponding to the business theme and the associated fact table to obtain a data cube;
and respectively mapping the data cube, the dimension column and the measurement column into a theme, a dimension and an index defined by the non-relational database.
According to the embodiment of the invention, the business data is used for constructing the data cube according to the business theme, and the business dimension column and the measurement column in the data cube are determined according to the dimension and the index corresponding to the business theme, so that the dimension and the index of the business layer can be combined in a self-service manner according to the query request input by the user to query and analyze in the data cube, the query flexibility is realized, the query efficiency is improved, and the user experience is enhanced.
The embodiment of the invention provides a multidimensional data analysis device, which is used for executing the multidimensional data analysis method.
Embodiments of the present invention provide a computer program, where the computer program can be called by a processor to enable a multidimensional data analysis device to execute a multidimensional data analysis method in any of the above method embodiments.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a computer-readable storage medium, the computer program comprising program instructions that, when run on a computer, cause the computer to perform the multidimensional data analysis method of any of the above-described method embodiments.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.
Claims (10)
1. A multidimensional data analysis method is applied to a multidimensional data analysis device, and the method comprises the following steps:
receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried;
determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes;
inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and returning the query result to the user.
2. The method of claim 1, wherein determining a target data cube and a structured query statement from the query request comprises:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
3. The method of claim 2, wherein the query analysis in the target data cube according to the structured query statement to obtain a query result comprises:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
4. The method of claim 1, wherein the query result comprises a result list; after the query result is returned to the user, the method includes:
receiving paging and sequencing requests of users;
paging and sorting the data results in the result list according to the paging and sorting request to obtain a processed result list;
and returning the processed result list to the user.
5. The method according to any one of claims 1-4, wherein before receiving the user's query request, the method comprises:
collecting source data;
processing the source data to obtain a fact table and a plurality of dimension tables; the fact table comprises a foreign key of the dimension table and fact data; the dimension table comprises at least one dimension table dimension;
acquiring each service theme required by a user and dimension and index requirement information corresponding to the service theme;
associating the fact table and the dimension table to construct a data model corresponding to each business theme;
configuring a dimension column and a measurement column on the data model according to the dimension and index demand information;
determining index values of the measurement columns under the combination of all the dimension columns according to the dimension table corresponding to the business theme and the associated fact table to obtain a data cube;
and respectively mapping the data cube, the dimension column and the measurement column into a theme, a dimension and an index defined by the non-relational database.
6. A multi-dimensional data analysis apparatus, characterized in that the apparatus comprises:
the receiving module is used for receiving a query request of a user; the query request comprises the theme, the dimensionality and the index of the business data to be queried;
the determining module is used for determining a target data cube and a structured query statement according to the query request; the multidimensional data analysis device comprises at least one data cube, wherein the at least one data cube is obtained by processing the business data in advance according to the requirements of business topics on dimensionality and indexes;
the analysis module is used for inquiring and analyzing in the target data cube according to the structured inquiry statement to obtain an inquiry result;
and the return module is used for returning the query result to the user.
7. The apparatus of claim 6, wherein determining the target data cube and the structured query statement from the query request comprises:
determining a data cube with a business theme matched with the business theme of the query request from the at least one data cube as a target data cube;
determining target dimension column information in the target data cube according to the dimension of the query request;
determining target metric column information in the target data cube according to the index of the query request;
and generating the structured query statement according to the business theme, the target dimension column information and the target measurement column information of the target data cube.
8. The apparatus of claim 7, wherein the query analysis in the target data cube according to the structured query statement to obtain a query result comprises:
determining the dimension value of a target dimension column in the target data cube and the index value of the target measurement column below the target dimension column according to the target dimension column information and the target measurement column information respectively;
and generating a query result according to the target dimension column information, the target measurement column information, the dimension value of the target dimension column and the index value of the target measurement column.
9. A multidimensional data analysis device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation of the multidimensional data analysis method as claimed in any one of claims 1 to 5.
10. A computer-readable storage medium having stored therein at least one executable instruction which, when run on a multidimensional data analysis device, causes the multidimensional data analysis device to perform the operations of the multidimensional data analysis method as recited in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110849478.1A CN113553341A (en) | 2021-07-27 | 2021-07-27 | Multidimensional data analysis method, multidimensional data analysis device, multidimensional data analysis equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110849478.1A CN113553341A (en) | 2021-07-27 | 2021-07-27 | Multidimensional data analysis method, multidimensional data analysis device, multidimensional data analysis equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113553341A true CN113553341A (en) | 2021-10-26 |
Family
ID=78132890
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110849478.1A Pending CN113553341A (en) | 2021-07-27 | 2021-07-27 | Multidimensional data analysis method, multidimensional data analysis device, multidimensional data analysis equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113553341A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115392799A (en) * | 2022-10-27 | 2022-11-25 | 平安科技（深圳）有限公司 | Attribution analysis method and device, computer equipment and storage medium |
-
2021
- 2021-07-27 CN CN202110849478.1A patent/CN113553341A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115392799A (en) * | 2022-10-27 | 2022-11-25 | 平安科技（深圳）有限公司 | Attribution analysis method and device, computer equipment and storage medium |
| CN115392799B (en) * | 2022-10-27 | 2023-04-11 | 平安科技（深圳）有限公司 | Attribution analysis method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230325410A1 (en) | Data analysis engine | |
| US20220284017A1 (en) | Systems and methods for rapid data analysis | |
| EP3602351B1 (en) | Apparatus and method for distributed query processing utilizing dynamically generated in-memory term maps | |
| US8943059B2 (en) | Systems and methods for merging source records in accordance with survivorship rules | |
| KR102134494B1 (en) | Profiling data with location information | |
| US11409645B1 (en) | Intermittent failure metrics in technological processes | |
| WO2019153487A1 (en) | System performance measurement method and device, storage medium and server | |
| CN103262076A (en) | Analytical data processing | |
| EP2973046B1 (en) | System and method for compressing data in database | |
| CN106293891B (en) | Multidimensional investment index monitoring method | |
| US9727663B2 (en) | Data store query prediction | |
| CN111414410A (en) | Data processing method, device, equipment and storage medium | |
| US9727666B2 (en) | Data store query | |
| CN107871055B (en) | Data analysis method and device | |
| CN113220728B (en) | Data query method, device, equipment and storage medium | |
| CN113553341A (en) | Multidimensional data analysis method, multidimensional data analysis device, multidimensional data analysis equipment and computer readable storage medium | |
| CN113918605A (en) | Data query method, device, equipment and computer storage medium | |
| JP7213890B2 (en) | Accelerated large-scale similarity computation | |
| CN114741392A (en) | Data query method and device, electronic equipment and storage medium | |
| CN113722141A (en) | Method and device for determining delay reason of data task, electronic equipment and medium | |
| JP6201053B2 (en) | Feature data management system and feature data management method | |
| CN113778996A (en) | Large data stream data processing method and device, electronic equipment and storage medium | |
| CN113138906A (en) | Call chain data acquisition method, device, equipment and storage medium | |
| CN115017185A (en) | Data processing method, device and storage medium | |
| CN111143328A (en) | Agile business intelligent data construction method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |