CN110716989A - Dimension data processing method and device, computer equipment and storage medium - Google Patents
Dimension data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110716989A CN110716989A CN201910798079.XA CN201910798079A CN110716989A CN 110716989 A CN110716989 A CN 110716989A CN 201910798079 A CN201910798079 A CN 201910798079A CN 110716989 A CN110716989 A CN 110716989A
- Authority
- CN
- China
- Prior art keywords
- data
- dimension
- target
- dimension table
- setting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Abstract
The application relates to a dimension data processing method, a dimension data processing device, computer equipment and a storage medium. The method comprises the following steps: acquiring a target dimension table of a data warehouse; setting the dimension parameters of the target dimension table according to preset dimension attribute information to obtain first dimension data; setting data table parameters of the target dimension table according to attribute information of a preset dimension table to obtain second dimension data; and setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data. The method can be used for carrying out data management and data processing on the dimensional data of the data warehouse.
Description
Technical Field
The present application relates to the field of data warehouse technologies, and in particular, to a method and an apparatus for processing dimensional data, a computer device, and a storage medium.
Background
The data center platform is a data platform for acquiring, calculating, storing, processing and the like mass data through a data technology. With the increasing popularization of the data center, the data center is a key construction target of the data center in the construction process. The data center station is based on a data warehouse, the dimension data is the most important basic data of the data warehouse, and how to define and manage the dimension data of the data warehouse is an important part for building the data warehouse. In a data management system of a current data warehouse, effective management of dimension data of the data warehouse is lacked, so that the efficiency of a data processing process is low when the data warehouse is used for building a data center.
Disclosure of Invention
In view of the above, it is necessary to provide a dimension data processing method, an apparatus, a computer device, and a storage medium, which can perform data management and data processing on dimension data of a data warehouse.
A method of dimensional data processing of a data warehouse, the method comprising:
acquiring a target dimension table of a data warehouse;
setting the dimension parameters of the target dimension table according to preset dimension attribute information to obtain first dimension data;
setting data table parameters of the target dimension table according to attribute information of a preset dimension table to obtain second dimension data;
and setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
In one embodiment, the setting the dimension parameter of the target dimension table according to the preset dimension attribute information to obtain the first dimension data includes:
defining a dimension parameter of the target dimension table according to preset dimension attribute information, mapping the dimension parameter and field data in the target dimension table, and taking data corresponding to the mapped dimension parameter as the first dimension data.
In one embodiment, the setting of the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table includes:
and according to the first dimension data of the target dimension table, setting the first data service for inquiring field data in the target dimension table according to the dimension parameters.
In one embodiment, the setting of the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table includes:
and setting the second data service for inquiring data in the target dimension table according to the data table parameters according to the second dimension data of the target dimension table.
In one embodiment, the method further comprises:
setting field parameters of the target dimension table according to field attribute information of a preset dimension table to obtain third dimension data;
the data service corresponding to the target dimension table comprises a third data service, and the third data service is a data service which is set according to the third dimension data and inquires corresponding field data in the target dimension table according to the field parameters.
In one embodiment, the setting of the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table includes:
setting data connection service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table;
and the data connection service is used for performing data connection on the data in the target dimension table and other data except the target dimension table by calling an association method according to the first dimension data and/or the second dimension data.
In one embodiment, the method further comprises:
and calling a preset detection task to perform quality monitoring on the data in the target dimension table according to a preset monitoring rule.
A dimensional data processing apparatus of a data warehouse, the apparatus comprising:
the acquisition module is used for acquiring a target dimension table of the data warehouse;
the first setting module is used for setting the dimension parameters of the target dimension table according to preset dimension attribute information to obtain first dimension data;
the second setting module is used for setting data table parameters of the target dimension table according to attribute information of a preset dimension table to obtain second dimension data;
and the data service setting module is used for setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any of the above embodiments when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any of the above embodiments.
According to the dimension data processing method, the dimension data processing device, the computer equipment and the storage medium, the dimension parameters of the target dimension table are set according to the preset dimension attribute information, and the first dimension data are obtained, so that the data dimensions of the target dimension table are uniformly managed. In addition, data table parameters of the target dimension table are set according to the attribute information of the preset dimension table, and second dimension data are obtained, so that the table dimensions of the target dimension table are uniformly managed. Further, the data service corresponding to the target dimension table is set according to the first dimension data and the second dimension data of the target dimension table, so that the data service of the first dimension data and the second dimension data in the target dimension table is fed back according to the data service request. Therefore, data management and data processing of the dimension data of the data warehouse are achieved.
Drawings
FIG. 1 is a diagram of an application environment of a method for dimensional data processing of a data warehouse, according to an embodiment;
FIG. 2 is a flow diagram that illustrates a method for dimensional data processing for a data warehouse, according to one embodiment;
FIG. 3 is a schematic diagram of a dimension information presentation page in which dimension attribute information is preset in one embodiment;
FIG. 4 is a schematic diagram of a dimension table information presentation page in which dimension table attribute information is preset in one embodiment;
FIG. 5 is a diagram illustrating a page showing dimension value information of attribute information of fields of a preset dimension table according to an embodiment;
FIG. 6 is a block diagram of a dimension data processing apparatus of a data warehouse, in one embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The dimension data processing method of the data warehouse can be applied to the application environment shown in fig. 1. Wherein the data source terminal 20 communicates with the database device 10 through the network 30 by performing network communication. The data source terminal 20 may be plural in number, and is used to upload various data to the database device 10. The data source terminal 20 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The database device 10 is internally provided with a data warehouse for storing various data stored in the data warehouse and uploaded by the data source terminal 20. The data warehouse comprises dimension tables. Dimension tables are used to store different types of data. The database device 10 may be implemented by a separate data management server or a data management server cluster composed of a plurality of data management servers, and is used for storing and managing the data uploaded by the data source terminal 20. Further, the data management device 40 is in network communication with the database device 10, and is configured to transmit a data processing instruction to the database device 10 to instruct the database device 10 to perform corresponding data processing on data within the database.
In one embodiment, as shown in fig. 2, a method for processing dimension data of a data warehouse is provided, which is described by taking the data warehouse 10 in fig. 1 as an example, and includes the following steps:
and S100, acquiring a target dimension table of the data warehouse.
The data warehouse stores a plurality of dimension tables for storing a plurality of types of data. The dimension table is set by a technician according to the requirements of a user. However, in the plurality of dimension tables, each dimension and parameter information corresponding to each dimension are not clearly defined. The data management server also has no standardized dimension management for the dimension tables of the data warehouse. Therefore, the utilization of the data in the data warehouse cannot cover the dimensional data service requirement in each scene. In the present embodiment, the data management server acquires a target dimension table of the data warehouse. Specifically, each dimension table has identifying information such as a dimension table name, a dimension table number, and the like. And the data management server acquires the target dimension table through the identification information.
S200, setting the dimension parameters of the target dimension table according to the preset dimension attribute information to obtain first dimension data.
In this embodiment, the data management server stores a plurality of kinds of preset dimension attribute information. The preset dimension attribute information is used for carrying out dimension setting on a dimension table in the data warehouse. The data management server acquires corresponding preset dimension attribute information from multiple preset dimension attribute information according to the characteristics of the target dimension table, and sets dimension parameters of the target dimension table according to the preset dimension attribute information, so that first dimension data are obtained. As shown in fig. 3, the preset dimension attribute information may include dimension name information, dimension description information, service field information to which data described by a dimension belongs, and dimension type information. And different dimension tables have different preset dimension attribute information according to different characteristics of the dimension tables. And setting the dimension parameters of the target dimension table according to the corresponding preset dimension attribute information, and determining the numerical values of the dimension parameters of the target dimension table, so that the data dimensions of the target dimension table are uniformly managed according to the preset dimension attribute information.
In one embodiment, step S200 includes: defining a dimension parameter of the target dimension table according to the preset dimension attribute information, mapping the dimension parameter and field data in the target dimension table, and taking data corresponding to the mapped dimension parameter as first dimension data.
Specifically, the data management server defines the dimension parameters of the target dimension table according to the preset dimension attribute information. As shown in fig. 3, such as a dimension value code, a dimension value name, a dimension value sequence, a dimension value effective time, a dimension value invalid time, etc. Further, the dimension parameter is mapped with field data in the target dimension table, and corresponding field data in the target dimension table is used as first dimension data. Therefore, the management of the dimension data can be further performed by combining the dimension of the target dimension table and the data.
In an implementation manner of this embodiment, setting a data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table includes: and setting a first data service for inquiring field data in the target dimension table according to the dimension parameters according to the first dimension data of the target dimension table.
Specifically, the data management server maps the dimension parameters with field data in the target dimension table, and takes data corresponding to the mapped dimension parameters as first dimension data. At this time, the first data service is a data service for querying field data in the target dimension table according to the dimension parameter, and querying field data in the target dimension table according to the dimension parameter can be realized. See in particular fig. 3. Accordingly, the efficiency of data processing of dimensional data of the data warehouse may be improved.
In an embodiment, the method for processing dimension data of a data warehouse further includes: and setting the dimension correlation characteristics of the target dimension table according to the preset dimension attribute information to obtain dimension characteristic data, wherein the first dimension data comprises dimension characteristic data.
Specifically, the data management server not only sets dimension parameters of the target dimension table according to the preset dimension attribute information, but also sets dimension correlation characteristics of the target dimension table according to the preset dimension attribute information, and assigns the data obtained after the setting of the dimension parameters and the dimension correlation characteristics to the first dimension data, so as to establish the first data service according to the first dimension data. Therefore, the management of the dimension data of the target dimension table is more standard, and the data processing efficiency when the first data service process is carried out by using the dimension data of the target dimension table is further improved. Wherein the dimension-related properties of the target dimension table include a date dimension. Setting the dimension correlation characteristics of the target dimension table according to the preset dimension attribute information comprises setting the time format of the date dimension and the granularity of the date dimension.
S300, setting data table parameters of the target dimension table according to the attribute information of the preset dimension table to obtain second dimension data.
In this embodiment, the data management server stores a plurality of kinds of preset dimension table attribute information. The preset dimension table attribute information is used for setting data table parameters of the dimension table in the data warehouse. And the data management server acquires corresponding preset dimension table attribute information from the multiple kinds of preset dimension table attribute information according to the characteristics of the target dimension table, and sets data table parameters of the target dimension table according to the preset dimension table attribute information so as to obtain second dimension data. As shown in fig. 4, the attribute information of the preset dimension table may include a name of the data table, a configuration type of the data table, and the like. The configuration types of the data table include manual configuration and non-manual configuration. And setting data table parameters of the target dimension table according to the corresponding preset dimension table attribute information, and determining the numerical values of the data table parameters of the target dimension table, so that the data dimensions of the target dimension table are uniformly managed according to the preset dimension table attribute information.
And S400, setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
In this embodiment, after the data management server completes setting the dimension parameters and the data table parameters of the target dimension table, the data service corresponding to the target dimension table is further set. And the data service is used for calling the data in the target dimension table according to the received data service request so as to realize the feedback of the corresponding data service. Generally, data services are determined based on traffic demands. The data services include a first data service established from the first dimension data and a second data service established from the second dimension data. That is, for the dimension parameters set according to the preset dimension attribute information and the data table parameters set according to the preset dimension table attribute information, the data management server establishes different data services according to the dimension parameters and the data table parameters. Therefore, the data management server not only performs dimension management on the target dimension table, but also performs corresponding data processing on the dimension data after the dimension management, so that the management of the dimension data of the target dimension table in the data warehouse is more standard, and the efficiency of data processing for the data warehouse is improved.
In one embodiment, step S400 includes: and setting a second data service for inquiring data in the target dimension table according to the data table parameters according to the second dimension data of the target dimension table.
In this embodiment, the second data service may query the data in the target dimension table according to the data table parameters. For example, the table structure data of the target dimension table may be participated in according to the data table name in the data table parameters. Therefore, the dimension data processing efficiency for the target dimension table can be improved.
In one embodiment, the data service corresponding to the target dimension table set by the data management server includes:
data service 1: and the dimension information service is used for inquiring dimension information according to the dimension code.
Data service 2: and the dimension table information service is used for inquiring the dimension table information service according to the name of the data table of the data warehouse.
Data service 3: the dimension value encoding and dimension value name conversion service is as follows:
large domain dimension 021 → large domain dimension: 021 Shanghai city
Data service 4: the dimension value search service, as follows:
large zone dimension, search term: south → large area dimension: [025 Nanjing, 0771 Nanning ]
According to the dimension data processing method, the dimension parameters of the target dimension table are set according to the preset dimension attribute information, and the first dimension data are obtained, so that the data dimensions of the target dimension table are uniformly managed. In addition, data table parameters of the target dimension table are set according to the attribute information of the preset dimension table, and second dimension data are obtained, so that the table dimensions of the target dimension table are uniformly managed. Further, the data service corresponding to the target dimension table is set according to the first dimension data and the second dimension data of the target dimension table, so that the data service of the first dimension data and the second dimension data in the target dimension table is fed back according to the data service request. Therefore, data management and data processing of the dimension data of the data warehouse are achieved.
In an embodiment, the dimension data processing method further includes: and setting field parameters of the target dimension table according to the field attribute information of the preset dimension table to obtain third dimension data. At this time, the data service corresponding to the target dimension table includes a third data service, and the third data service is a data service that is set according to the third dimension data and that queries corresponding field data in the target dimension table according to the field parameter.
In this embodiment, the data management server not only sets the dimension parameters of the target dimension table and the data table parameters correspondingly, but also sets the field parameters of the target dimension table according to the preset dimension table field attribute information, and sets a third data service corresponding to the target dimension table and inquiring the corresponding field data in the target dimension table according to the field parameters according to the third dimension data. See in particular fig. 5. Therefore, the dimension data processing efficiency for the target dimension table can be improved.
In one embodiment, step S400 includes: and setting the data connection service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table. And the data connection service is used for performing data connection on the data in the target dimension table and other data except the target dimension table by calling the association method according to the first dimension data and/or the second dimension data.
Specifically, the data connection service is implemented by a real-time dimension table join service plug-in. The plug-in is provided in a jar packet form and supports a mainstream real-time processing framework. And according to the first dimension data and/or the second dimension data, transmitting the related data by calling an associated method. For example, the corresponding dimension table name, dimension code and dimension value code are transmitted according to the dimension table name, dimension code and dimension value code, the table data information is obtained in real time, and real-time join operation is carried out.
In an embodiment, the dimension data processing method further includes: and acquiring dimension data from the set target dimension table, and storing the dimension data into the unified database. In this case, step S400 includes: and setting data service according to the dimension data in the unified database.
Specifically, after setting the dimension parameters and the data table parameters of the target dimension table, the data management server automatically generates a spark task, collects dimension table data by using the spark task, and stores the data into an Hbase database and an Es database. And the Hbase database and the Es database are data storage databases in the unified database. And finally, setting data service according to the unified database. Therefore, the data in the data warehouse can be uniformly managed.
In an embodiment, the dimension data processing method further includes: and calling a preset detection task to perform quality monitoring on the data in the target dimension table according to a preset monitoring rule.
Specifically, the data management server monitors the data quality of the data in the target dimension table. The monitoring mode is that preset monitoring rules are configured, corresponding preset detection tasks are generated, and the quality of data in the target dimension table is monitored by scheduling the preset detection tasks. And if the data in the target dimension table has quality problems, sending alarm information to an alarm center. And when the alarm center receives the alarm information, the alarm information is sent to the responsible person. Therefore, the quality monitoring can be carried out on the data of the target dimension table in the data warehouse, and the management quality of the data in the data warehouse is improved.
It should be understood that, although the steps in the flowchart are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 6, there is provided a dimension data processing apparatus of a data warehouse, including: an acquisition module 100, a first setup module 200, a second setup module 300, and a data service setup module 400, wherein:
the obtaining module 100 is configured to obtain a target dimension table of the data warehouse.
The first setting module 200 is configured to set a dimension parameter of the target dimension table according to the preset dimension attribute information, so as to obtain first dimension data.
And the second setting module 300 is configured to set a data table parameter of the target dimension table according to the preset dimension table attribute information, so as to obtain second dimension data.
And the data service setting module 400 is configured to set data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, where the data services include a first data service established according to the first dimension data and a second data service established according to the second dimension data.
In one embodiment, the first setup module 200 may include (not shown in fig. 6):
and the first setting unit is used for defining the dimension parameters of the target dimension table according to the preset dimension attribute information, mapping the dimension parameters and field data in the target dimension table, and taking the data corresponding to the mapped dimension parameters as first dimension data.
At this time, the data service module 400 may include:
and the first data service unit is used for setting a first data service for inquiring field data in the target dimension table according to the dimension parameters according to the first dimension data of the target dimension table.
In one embodiment, the data service module 400 may include (not shown in fig. 6):
and the second data service unit is used for setting a second data service for inquiring data in the target dimension table according to the data table parameters according to the second dimension data of the target dimension table.
In one embodiment, the dimension data processing apparatus of the data warehouse further includes (not shown in fig. 6):
the third setting module is used for setting field parameters of the target dimension table according to the field attribute information of the preset dimension table to obtain third dimension data;
and the third data service unit is used for setting data services corresponding to the target dimension table according to the first dimension data, the second dimension data and the third dimension data of the target dimension table, and the data services comprise third data services which are set according to the third dimension data and used for inquiring corresponding field data in the target dimension table according to the field parameters.
In one embodiment, the data service module 400 may include (not shown in fig. 6):
and the fourth data service unit is used for setting data connection service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, and the data connection service is used for calling a correlation method according to the first dimension data and/or the second dimension data and carrying out data connection on the data of the target dimension table and other data.
In one embodiment, the dimension data processing apparatus of the data warehouse further includes (not shown in fig. 6):
and the monitoring module is used for calling a preset detection task to carry out quality monitoring on the data in the target dimension table according to a preset monitoring rule.
For specific limitations of the dimension data processing apparatus of the data warehouse, reference may be made to the above limitations of the dimension data processing method of the data warehouse, and details are not repeated here. The modules in the dimensional data processing device of the data warehouse can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a data management server, and its internal structure diagram may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer equipment is used for communicating with an external data source terminal through network connection so as to receive data uploaded by the data source terminal. The computer program is executed by a processor to implement a method of dimensional data processing for a data warehouse.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring a target dimension table of a data warehouse; setting a dimension parameter of a target dimension table according to preset dimension attribute information to obtain first dimension data; setting data table parameters of a target dimension table according to the attribute information of the preset dimension table to obtain second dimension data; and setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
In one embodiment, the processor executes a computer program to set dimension parameters of the target dimension table according to preset dimension attribute information, and when obtaining the first dimension data, further implements the following steps:
defining a dimension parameter of the target dimension table according to the preset dimension attribute information, mapping the dimension parameter and field data in the target dimension table, and taking data corresponding to the mapped dimension parameter as first dimension data.
In one embodiment, when the processor executes the computer program to realize that the data service corresponding to the target dimension table is set according to the first dimension data and the second dimension data of the target dimension table, the following steps are further realized:
and setting a first data service for inquiring field data in the target dimension table according to the dimension parameters according to the first dimension data of the target dimension table.
In one embodiment, when the processor executes the computer program to realize that the data service corresponding to the target dimension table is set according to the first dimension data and the second dimension data of the target dimension table, the following steps are further realized:
and setting the second data service for inquiring data in the target dimension table according to the data table parameters according to the second dimension data of the target dimension table.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
setting field parameters of the target dimension table according to field attribute information of the preset dimension table to obtain third dimension data;
the data service corresponding to the target dimension table comprises a third data service, and the third data service is a data service which is set according to the third dimension data and inquires corresponding field data in the target dimension table according to the field parameters.
In one embodiment, when the processor executes the computer program to realize that the data service corresponding to the target dimension table is set according to the first dimension data and the second dimension data of the target dimension table, the following steps are further realized:
setting data connection service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table;
and the data connection service is used for performing data connection on the data in the target dimension table and other data except the target dimension table by calling the association method according to the first dimension data and/or the second dimension data.
In one embodiment, the processor executes the computer program, further implementing the steps of:
and calling a preset detection task to perform quality monitoring on the data in the target dimension table according to a preset monitoring rule.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements a method for processing dimensional data of a data warehouse as described in any of the above embodiments.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a target dimension table of a data warehouse; setting a dimension parameter of a target dimension table according to preset dimension attribute information to obtain first dimension data; setting data table parameters of a target dimension table according to the attribute information of the preset dimension table to obtain second dimension data; and setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
In one embodiment, the computer program is executed by a processor, and when the dimension parameter of the target dimension table is set according to the preset dimension attribute information to obtain the first dimension data, the following steps are further implemented:
defining a dimension parameter of the target dimension table according to the preset dimension attribute information, mapping the dimension parameter and field data in the target dimension table, and taking data corresponding to the mapped dimension parameter as first dimension data.
In one embodiment, when the computer program is executed by the processor to realize the setting of the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, the following steps are further realized:
and setting a first data service for inquiring field data in the target dimension table according to the dimension parameters according to the first dimension data of the target dimension table.
In one embodiment, when the computer program is executed by the processor to realize the setting of the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, the following steps are further realized:
and setting the second data service for inquiring data in the target dimension table according to the data table parameters according to the second dimension data of the target dimension table.
In one embodiment, the computer program, when executed by the processor, further performs the steps of:
setting field parameters of the target dimension table according to field attribute information of the preset dimension table to obtain third dimension data;
the data service corresponding to the target dimension table comprises a third data service, and the third data service is a data service which is set according to the third dimension data and inquires corresponding field data in the target dimension table according to the field parameters. In one embodiment, when the computer program is executed by the processor to realize the setting of the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, the following steps are further realized:
setting data connection service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table;
and the data connection service is used for performing data connection on the data in the target dimension table and other data except the target dimension table by calling the association method according to the first dimension data and/or the second dimension data.
In one embodiment, the computer program, when executed by the processor, further performs the steps of:
and calling a preset detection task to perform quality monitoring on the data in the target dimension table according to a preset monitoring rule.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of dimensional data processing of a data warehouse, the method comprising:
acquiring a target dimension table of a data warehouse;
setting the dimension parameters of the target dimension table according to preset dimension attribute information to obtain first dimension data;
setting data table parameters of the target dimension table according to attribute information of a preset dimension table to obtain second dimension data;
and setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
2. The method according to claim 1, wherein the setting the dimension parameter of the target dimension table according to the preset dimension attribute information to obtain first dimension data comprises:
defining a dimension parameter of the target dimension table according to preset dimension attribute information, mapping the dimension parameter and field data in the target dimension table, and taking data corresponding to the mapped dimension parameter as the first dimension data.
3. The method according to claim 2, wherein the setting the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table comprises:
and according to the first dimension data of the target dimension table, setting the first data service for inquiring field data in the target dimension table according to the dimension parameters.
4. The method according to claim 1, wherein the setting the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table comprises:
and setting the second data service for inquiring data in the target dimension table according to the data table parameters according to the second dimension data of the target dimension table.
5. The method of claim 1, further comprising:
setting field parameters of the target dimension table according to field attribute information of a preset dimension table to obtain third dimension data;
the data service corresponding to the target dimension table comprises a third data service, and the third data service is a data service which is set according to the third dimension data and inquires corresponding field data in the target dimension table according to the field parameters.
6. The method according to claim 1, wherein the setting the data service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table comprises:
setting data connection service corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table;
and the data connection service is used for performing data connection on the data in the target dimension table and other data except the target dimension table by calling an association method according to the first dimension data and/or the second dimension data.
7. The method of claim 1, further comprising:
and calling a preset detection task to perform quality monitoring on the data in the target dimension table according to a preset monitoring rule.
8. A dimensional data processing apparatus of a data warehouse, the apparatus comprising:
the acquisition module is used for acquiring a target dimension table of the data warehouse;
the first setting module is used for setting the dimension parameters of the target dimension table according to preset dimension attribute information to obtain first dimension data;
the second setting module is used for setting data table parameters of the target dimension table according to attribute information of a preset dimension table to obtain second dimension data;
and the data service setting module is used for setting data services corresponding to the target dimension table according to the first dimension data and the second dimension data of the target dimension table, wherein the data services comprise a first data service established according to the first dimension data and a second data service established according to the second dimension data.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910798079.XA CN110716989A (en) | 2019-08-27 | 2019-08-27 | Dimension data processing method and device, computer equipment and storage medium |
| CA3152835A CA3152835A1 (en) | 2019-08-27 | 2020-06-24 | Dimension data processing method and apparatus, computer device, and storage medium |
| PCT/CN2020/097835 WO2021036449A1 (en) | 2019-08-27 | 2020-06-24 | Dimension data processing method and apparatus, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910798079.XA CN110716989A (en) | 2019-08-27 | 2019-08-27 | Dimension data processing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110716989A true CN110716989A (en) | 2020-01-21 |
Family
ID=69209521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910798079.XA Pending CN110716989A (en) | 2019-08-27 | 2019-08-27 | Dimension data processing method and device, computer equipment and storage medium |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN110716989A (en) |
| CA (1) | CA3152835A1 (en) |
| WO (1) | WO2021036449A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111402066A (en) * | 2020-02-22 | 2020-07-10 | 中国平安财产保险股份有限公司 | Data processing method, server and storage medium |
| WO2021036449A1 (en) * | 2019-08-27 | 2021-03-04 | 苏宁云计算有限公司 | Dimension data processing method and apparatus, computer device, and storage medium |
| CN112732712A (en) * | 2020-12-29 | 2021-04-30 | 望海康信(北京)科技股份公司 | Chart information data storage method and system, corresponding equipment and storage medium |
| CN117150348A (en) * | 2023-10-30 | 2023-12-01 | 宁德时代新能源科技股份有限公司 | Battery external damage data processing method, system, electronic equipment and storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113393190B (en) * | 2021-06-10 | 2023-12-05 | 北京京东振世信息技术有限公司 | Warehouse information processing method and device, electronic equipment and readable medium |
| CN113449024B (en) * | 2021-06-23 | 2023-02-14 | 平安普惠企业管理有限公司 | Insurance data analysis method, device, equipment and medium based on big data |
| CN114648316B (en) * | 2022-05-18 | 2022-08-23 | 国网浙江省电力有限公司 | Digital processing method and system based on inspection tag library |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104252452A (en) * | 2013-06-25 | 2014-12-31 | 腾讯科技（深圳）有限公司 | Data management method and device |
| CN109614402A (en) * | 2018-12-11 | 2019-04-12 | 北京京东金融科技控股有限公司 | Multidimensional data query method and device |
| CN110019551A (en) * | 2017-12-19 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of Building Method of Data Warehouse and device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9483537B1 (en) * | 2008-03-07 | 2016-11-01 | Birst, Inc. | Automatic data warehouse generation using automatically generated schema |
| CN107908631B (en) * | 2017-07-25 | 2021-04-20 | 平安科技（深圳）有限公司 | Data processing method, data processing device, storage medium and computer equipment |
| CN109561326B (en) * | 2017-09-26 | 2021-02-12 | 北京国双科技有限公司 | Data query method and device |
| CN110716989A (en) * | 2019-08-27 | 2020-01-21 | 苏宁云计算有限公司 | Dimension data processing method and device, computer equipment and storage medium |
-
2019
- 2019-08-27 CN CN201910798079.XA patent/CN110716989A/en active Pending
-
2020
- 2020-06-24 WO PCT/CN2020/097835 patent/WO2021036449A1/en active Application Filing
- 2020-06-24 CA CA3152835A patent/CA3152835A1/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104252452A (en) * | 2013-06-25 | 2014-12-31 | 腾讯科技（深圳）有限公司 | Data management method and device |
| CN110019551A (en) * | 2017-12-19 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of Building Method of Data Warehouse and device |
| CN109614402A (en) * | 2018-12-11 | 2019-04-12 | 北京京东金融科技控股有限公司 | Multidimensional data query method and device |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021036449A1 (en) * | 2019-08-27 | 2021-03-04 | 苏宁云计算有限公司 | Dimension data processing method and apparatus, computer device, and storage medium |
| CN111402066A (en) * | 2020-02-22 | 2020-07-10 | 中国平安财产保险股份有限公司 | Data processing method, server and storage medium |
| CN112732712A (en) * | 2020-12-29 | 2021-04-30 | 望海康信(北京)科技股份公司 | Chart information data storage method and system, corresponding equipment and storage medium |
| CN117150348A (en) * | 2023-10-30 | 2023-12-01 | 宁德时代新能源科技股份有限公司 | Battery external damage data processing method, system, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CA3152835A1 (en) | 2021-03-04 |
| WO2021036449A1 (en) | 2021-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110716989A (en) | Dimension data processing method and device, computer equipment and storage medium | |
| CN109788031B (en) | Service data acquisition method and device, computer equipment and storage medium | |
| CN108924250B (en) | Service request processing method and device based on block chain and computer equipment | |
| CN109510840B (en) | Method and device for sharing unstructured data, computer equipment and storage medium | |
| CN110266752B (en) | Block chain information pushing method and device, computer equipment and storage medium | |
| CN111563368A (en) | Report generation method and device, computer equipment and storage medium | |
| CN111310427A (en) | Service data configuration processing method and device, computer equipment and storage medium | |
| CN110213392B (en) | Data distribution method and device, computer equipment and storage medium | |
| CN110717647A (en) | Decision flow construction method and device, computer equipment and storage medium | |
| CN111177776A (en) | Multi-tenant data isolation method and system | |
| CN111061678A (en) | Service data processing method and device, computer equipment and storage medium | |
| CN110569321A (en) | grid division processing method and device based on urban map and computer equipment | |
| CN111177121A (en) | Order data feedback method and device, computer equipment and storage medium | |
| CN109218131B (en) | Network monitoring method and device, computer equipment and storage medium | |
| CN108389124B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN111897843B (en) | Configuration method and device of data flow strategy of Internet of things and computer equipment | |
| CN114201511A (en) | Project management and control method and device, computer equipment and storage medium | |
| CN110399534B (en) | Terminal performance report generation method, device, equipment and storage medium | |
| CN109474386B (en) | Signaling tracking method, system, network element equipment and storage medium | |
| CN111885184A (en) | Method and device for processing hot spot access keywords in high concurrency scene | |
| CN111130991A (en) | Instant messaging information processing method and device, computer equipment and storage medium | |
| CN110113384A (en) | Network request processing method, device, computer equipment and storage medium | |
| CN111708795B (en) | Object identification generation method, object identification updating device, computer equipment and medium | |
| CN110730106B (en) | Electronic official document exchange method and device based on tree structure and computer equipment | |
| CN113220759A (en) | Big data storage service sharing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200121 |
|
| RJ01 | Rejection of invention patent application after publication |