CN112527919A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN112527919A CN112527919A CN202011406985.XA CN202011406985A CN112527919A CN 112527919 A CN112527919 A CN 112527919A CN 202011406985 A CN202011406985 A CN 202011406985A CN 112527919 A CN112527919 A CN 112527919A
- Authority
- CN
- China
- Prior art keywords
- data
- star model
- layer
- layer star
- user entity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 6
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 34
- 230000004044 response Effects 0.000 claims abstract description 9
- 238000013507 mapping Methods 0.000 claims description 34
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 12
- 238000006243 chemical reaction Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000012800 visualization Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 208000019901 Anxiety disease Diseases 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 2
- 230000036506 anxiety Effects 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011068 loading method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000533950 Leucojum Species 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/288—Entity relationship models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Abstract
The embodiment of the invention provides a data processing method and a data processing device, wherein the method comprises the following steps: presetting a double-layer star model; the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, and the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data; in response to a user clustering request based on one or more target tag data, determining one or more target second user entity data corresponding to the one or more target tag data from the one or more second user entity data of the second layer star model. According to the embodiment of the invention, the tool requirement for constructing the user-defined client group is realized, the client grouping framework based on the double-layer star model can flexibly adapt to the requirement change, the service expansion is supported, and the technical use threshold is reduced.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and an apparatus for data processing.
Background
When user information of a specific user is obtained, searching is generally required to be performed from each fractured system, so that complete user information is supplemented, such as from a sales system, an after-sales system and the like, a large amount of cost is consumed to process data, and data of each system is difficult to integrate.
In the prior art, data of each system can be communicated and integrated into a data warehouse, so that the cost of data processing is reduced, but for acquiring user data from a plurality of bins, a manual development script needs to be adopted to execute the data in the bins, but a certain technical threshold exists in the mode, the method is not friendly to business personnel with non-technical backgrounds, the business requirements are increased and changed very quickly, new data statistical requirements are continuously provided, so that the related statistical task quantity of a customer group is accumulated, the requirement feedback efficiency is influenced, and the running cost of the data warehouse system is increased.
Disclosure of Invention
In view of the above, it is proposed to provide a method and apparatus for data processing that overcomes or at least partially solves the above mentioned problems, comprising:
a method of data processing, the method comprising:
presetting a double-layer star model; the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, and the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data;
in response to a user clustering request based on one or more target tag data, determining one or more target second user entity data corresponding to the one or more target tag data from the one or more second user entity data of the second layer star model.
Optionally, a mapping relationship is set between the first layer of star models and the second layer of star models, and data in the first layer of star models is synchronously mapped to data in the second layer of star models through the mapping relationship.
Optionally, the first-layer star model includes one or more first user entity data and one or more dimensional business data corresponding to each first user entity data, and the preset two-layer star model includes:
constructing the first-layer star model according to the acquired one or more first user entity data and the one or more dimensionality service data corresponding to each first user entity data;
mapping one or more first user entity data in the first layer star model and one or more dimensionality service data corresponding to each first user entity data according to the mapping relation to obtain one or more second user entity data and one or more label data corresponding to each second user entity data;
and constructing a second layer star model according to the one or more second user entity data and the one or more label data corresponding to each second user entity data.
Optionally, the method further comprises:
visualizing the one or more tag data.
Optionally, the method further comprises:
generating data to be shared according to the one or more target second user entity data;
and sharing the data to be shared through a preset external interface.
Optionally, each first user entity data corresponds to a main table in the first-layer star model, and the service data of each dimension corresponds to a dimension table associated with the main table in the first-layer star model;
each second user entity data corresponds to a main table in the second-layer star model, and each label data corresponds to a dimension table associated with the main table in the second-layer star model.
Optionally, the data in the first layer star model is stored in a data warehouse, and the data in the second layer star model is stored in a relational database.
An apparatus for data processing, the apparatus comprising:
the double-layer star model presetting module is used for presetting a double-layer star model; the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, and the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data;
a target second user entity data determining module, configured to determine, in response to a user clustering request based on one or more target tag data, one or more target second user entity data corresponding to the one or more target tag data from the one or more second user entity data of the second layer star model.
A server comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program, when executed by the processor, implementing a method of data processing as described above.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of data processing as described above.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, by presetting a double-layer star model, the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data, and further responding to a user grouping request based on one or more target label data, one or more target second user entity data corresponding to one or more target label data are determined from one or more second user entity data of the second-layer star model, so as to realize the tool requirement for constructing a user-defined client group, and by means of a client grouping framework based on the double-layer star model, the requirement change can be flexibly adapted, the service expansion is supported, and the technical use threshold is reduced, the method has the advantages of flexibility, expandability and usability.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic diagram of an example star model provided by an embodiment of the present invention;
FIG. 2 is a diagram illustrating an example custom customer group architecture provided by an embodiment of the present invention;
FIG. 3 is a flow chart illustrating steps of a method for data processing according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In a data warehouse, a data model can be designed around a star model and a snowflake model, and because the star model can improve the query performance by reducing the data storage amount to the maximum extent and combining smaller dimension tables in multi-dimensional data query or analysis, namely higher efficiency can be realized under the condition of limited data redundancy, and dynamic expansion of the dimension tables is supported, the invention adopts the star model for design.
The star model is a multidimensional data relationship, as shown in fig. 1, and may be composed of a fact table and multiple dimension tables, where the multiple dimension tables may be directly connected to the fact table, each dimension table may have one dimension as a primary key, for example, the dimension may be data of a type such as text, time, etc., and the primary keys of all the dimension tables may be combined into the primary key of the fact table. While the non-primary key attributes of the fact table are referred to as facts, which may be numeric values or other data capable of being calculated.
Data is organized in a star model mode, and aggregation calculation or analysis of the fact data in the fact table can be adopted according to different dimensions (partial main keys or all main keys of the fact table) to perform summation (sum), averaging (average), counting (count) and percentage (percentage), so that the condition of the business theme can be analyzed through multiple dimensions.
In practical applications, the design of the "star model" may be applied to a custom client group architecture to construct a two-layer star model, as shown in fig. 2, which may include a bin layer, a conversion layer, and a presentation layer.
The warehouse layer can be provided with an entity-relation star model which consists of a customer entity table and a plurality of dimension tables; the presentation layer may have a client-tag star model, which is a star model formed by taking "clients" as a master table and designing different dimensions around the client's tags, so that the presentation layer has the efficient querying, flexible expansion characteristics of the star model. The two star models can be distributed in the warehouse layer and the display layer respectively and belong to different physical layers.
Moreover, the two-layer star model can realize mutual data communication through a conversion layer (such as an association logic Mapping layer), and the star model of the warehouse layer can be mapped to the star model of the presentation layer, such as Mapping the data of the warehouse layer 'fact table-dimension' to the data of the presentation layer 'client-label'.
Based on the fact, the double-layer star model can have the characteristic of flexible expansion of the fact-dimension table, and with the addition and change of service requirements, the service dimension can be expanded only by expanding the dimension table without changing the overall structure on the basis of the original main fact table; meanwhile, the star model has the characteristics of less execution and high efficiency, can support more efficient meeting of customer requirements, and can support flexible extension of the label through dimension mapping.
Referring to fig. 3, a flowchart illustrating steps of a data processing method according to an embodiment of the present invention is shown, which may specifically include the following steps:
during data query of customer clustering, a double-layer star model can be preset, the double-layer star model can comprise a first-layer star model and a second-layer star model, the first-layer star model can provide data support for the second-layer star model, for example, the first-layer star model can be an entity-relation star model, and the second-layer star model can be a customer-label star model.
Specifically, the first-layer star model may include one or more first user entity data and one or more dimensional business data corresponding to each first user entity data, each first user entity data may correspond to a main table in the first-layer star model, and the business data of each dimension may correspond to a dimension table associated with the main table in the first-layer star model.
The second-layer star model may include one or more second user entity data and one or more label data corresponding to each second user entity data, each second user entity data may correspond to a main table in the second-layer star model, and each label data may correspond to a dimension table associated with the main table in the second-layer star model.
A mapping relation can be set between the first layer star model and the second layer star model, and data in the first layer star model can be synchronously mapped into data in the second layer star model through the mapping relation.
In an example, data in the first layer star model may be stored in a data warehouse, data in the second layer star model may be stored in a relational database, and the first layer star model and the second layer star model may be distributed in a warehouse layer and a display layer respectively, and belong to different physical layers.
For example, taking a charging service as an example, a two-layer star model for the charging service may be preset in the following manner:
1. the customer master table, the vehicle maintenance table, the charging maintenance table, the home charging pile maintenance table and the charging pile maintenance table can form an entity-relation star model (namely, a first-layer star model) of a plurality of warehouse layers;
2. the customer, a vehicle label, a charging label, a home charging pile label and a charging maintenance label can form a customer-label star model (namely a second layer star model) of the display layer;
3. the two layers of star models can be associated through Mapping to form a double-layer star model, and Mapping association modes can include client tag field Mapping, logical association Mapping and mutual exclusion association.
Taking the client tag field Mapping as an example, the Mapping relation granularity is field level, when the warehouse layer field Mapping is mapped to the field of the display layer, whether the conversion of the association relation is needed or not can be judged, when the association relation is set to be empty, the data can be directly associated without the conversion, otherwise, the conversion is needed according to a formula in the association relation.
In an embodiment of the present invention, step 301 may include the following sub-steps:
substep 11, constructing the first-layer star model according to the acquired one or more first user entity data and the one or more dimensional service data corresponding to each first user entity data;
in a specific implementation, a first-layer star model, such as an entity-relationship star model, may be constructed according to the obtained one or more first user entity data and the one or more dimensional business data corresponding to each first user entity data, and data in the first-layer star model may be stored in a data warehouse.
In an example, as shown in fig. 2, the first-layer star model distributed in the warehouse layer may be composed of one or more first user entity data and business data of one or more dimensions corresponding to each first user entity data, each first user entity data may correspond to a main table (e.g., a customer entity table) in the first-layer star model, and the business data of each dimension may correspond to a dimension table (e.g., a different dimension table) associated with the main table in the first-layer star model, wherein the customer entity table may be connected with one or more dimension tables, which may include a customer-type dimension table, a charging pile dimension table, a community posting dimension table, a community review dimension table, a maintenance history dimension table, a vehicle dimension table, an order dimension table, and the like.
In yet another example, data support may be provided for building a two-tier star model by obtaining source data, determining one or more first user entity data and business data for one or more dimensions corresponding to each first user entity data.
Specifically, the source Data is obtained through an ODS layer (Operational Data Store) in the Data warehouse, which may exist in the Data warehouse architecture as a source Data layer, the Data warehouse may have a first Data layer, a second Data layer, and a third Data layer, the first Data layer may be used to Store the source Data, the second Data layer may be used to Store one or more first user entity Data and one or more service Data corresponding to each first user entity Data, the third Data layer may be used to Store one or more second user entity Data and one or more tag Data corresponding to each second user entity Data, and the third Data layer may provide Data support for the relational database.
The whole data architecture of the user-defined customer group based on the double-layer star model can be composed of a source system layer, an ODPS data warehouse layer and a display layer, wherein the source system layer can comprise business systems related to customers, such as official channels, a user center, a CRM system, an after-sales OAS system, an APP, a community, a charging management platform, a financial system, a customer service system and the like, and can provide data sources.
The ODPS data warehouse layer, which may perform data collection, loading, cleaning, and processing, may have an ODS layer — a source data layer (i.e., a first data layer) for storing source data; DP layer-shared data platform layer (i.e., second data layer) for storing data of the first layer star model in the two-layer star model; the UP layer-the client label integration layer (namely, the third data layer) is used for storing data of the star model at the second layer in the double-layer star model and can provide data support for the relational database.
The presentation layer can be a visualization tool for the client label, which can be a second layer star model in the double-layer star model, and can store data of the second layer star model (such as a client-label star model), and the data source of the presentation layer can include the UP layer of the data warehouse layer and the manually labeled data.
After the source data is acquired, the source data may be preprocessed, for example, the sorting of the source data may be performed in three steps: the method comprises the steps of combing, data collecting and data loading of a source system, determining a subject domain to which the source data belongs, classifying the source data according to the subject domain, and further performing data integration on the source data in each subject domain to obtain one or more first user entity data and one or more dimensionality business data corresponding to each first user entity data.
For example, the manner of data integration includes any one or more of:
standardizing field names, standardizing data structures, filtering repeated data and screening data with conflicts.
Substep 12, mapping one or more first user entity data in the first layer star model and one or more dimensionality service data corresponding to each first user entity data according to the mapping relation to obtain one or more second user entity data and one or more label data corresponding to each second user entity data;
after the first-layer star model is constructed, one or more first user entity data in the first-layer star model and one or more dimensionality service data corresponding to each first user entity data can be mapped according to a mapping relation set between the first-layer star model and the second-layer star model, and then one or more second user entity data and one or more label data corresponding to each second user entity data can be obtained, namely, the data in the first-layer star model can be synchronously mapped into the data in the second-layer star model through the mapping relation.
In an example, as shown in fig. 2, a first layer star model distributed in the several bin layers and a second layer star model distributed in the presentation layer may be mapped through the conversion layer, data in the first layer star model distributed in the several bin layers may be synchronously mapped into data in the second layer star model distributed in the presentation layer according to a mapping relationship between an entity-relationship and a body-relationship, or data in the first layer star model distributed in the several bin layers may be called through the presentation layer by using a data processing method of natural language participle parsing.
For example, the number bin layer may store a plurality of charging detail data, such as specific fields: the charging system comprises a vehicle frame number, a charging starting time, a charging starting electric quantity, a charging time, a charging ending electric quantity and the like, but the detailed data cannot intuitively explain the charging preference of a client, so that a charging SOC (State of Charge) preference label for the client can be processed.
The conversion logic defined by referring to mapping is processed, mapping can be adopted to describe that the charging SOC conversion logic is the average value of the charging SOC in the past 30 days, then the set mapping logic can be referred to find detailed data of all charging in the past 30 days of a client, and then the charging SOC preference label of the client can be obtained by averaging the charging SOC field value.
And a substep 13, constructing a second layer star model according to the one or more second user entity data and the one or more label data corresponding to each second user entity data.
In practical applications, a second-layer star model, such as a client-tag star model, may be constructed according to one or more second user entity data and one or more tag data corresponding to each second user entity data, and data in the second-layer star model may be stored in a relational database.
In an example, as shown in fig. 2, the second-layer star model distributed in the presentation layer may be composed of one or more second user entity data and one or more tag data corresponding to each second user entity data, each second user entity data may correspond to a main table (e.g., a client entity table) in the second-layer star model, and each tag data (e.g., a tag possessed by the client) may correspond to a dimension table associated with the main table in the second-layer star model.
The customer group labels can be obtained through different customer group conditions, and can comprise clue labels, community labels, charging labels, complaint labels, sensitive customer labels, vehicle labels, order labels, basic labels and the like.
In yet another example, for each tag data distributed in the second-layer star model of the presentation layer, a dimension table associated with the main table in the second-layer star model may be corresponding to the tag data, the dimension table may have tag information corresponding to the tag data, and SQL statements that may be used for requesting parsing for user grouping may be used, and the target second user entity data corresponding to the target tag data may be determined by querying the tag information through the SQL statements.
In practical application, the client clustering framework based on the double-layer star model can comprise a double-layer star model module and a data processing module, wherein the double-layer star model can be used as a bottom layer supporting module which plays a fundamental decisive role in data conversion processing expansion and flexible customization of client label clustering; and the data processing module can have the function of associating the model layer and the display layer, and can support the data content of the display layer by processing the data based on the Mapping processing defined by the model.
The data processing module can have a client label processing function, and can process the client, the vehicle and the charging related data by acquiring the client, cleaning and summarizing the data, and then can process the client, the vehicle and the charging data into corresponding labels and store the labels according to the first-layer star models distributed on the warehouse layer, if the corresponding labels are marked on each client, then the data can be pushed by configuring a data pushing task, and the data pushing module adopts Mapping associated pushing data, so that the data can be synchronously pushed from the first-layer star models distributed on the warehouse layer to the second-layer star models distributed on the display layer.
After the two-layer star model is obtained, one or more target second user entity data corresponding to one or more target label data can be determined from one or more second user entity data of the second-layer star model in response to a user clustering request based on one or more target label data, so as to feed back data query requirements for the client clustering.
In practical application, the data processing module may further have a function of a data processing engine, and in the presentation layer, by responding to a user grouping request based on one or more target tag data, such as a tag voice formula consisting of a client tag, a vehicle tag, and a charging tag, the tag voice formula may be expressed as "acquiring a charging anxiety client group with charging soc tag greater than 50% in 9 months in 2010", and may perform parsing processing, and may further parse into a script language corresponding to the database, and may optimize the script and feed back the result to the presentation layer.
In an embodiment of the present invention, the method may further include the following steps:
visualizing the one or more tag data.
In practical application, the client clustering framework based on the double-layer star model can further comprise a visualization module, the visualization module can serve as a user interaction layer, one or more label data can be visualized according to the star model distributed on the second layer of the display layer, and therefore the client clustering tool capable of being flexibly expanded is constructed.
In one example, the visualization layer has a friendly interactive interface for business personnel without technical background, which uses a threshold bottom, is easy to enter, and can display a customer label, a vehicle label and a charging label on a page through a grouping control. The control can support a dragging mode, the time label and the charging SOC label are placed in an operation area, the condition that the time is 9 months and the charging SOC > is 50 percent is filled in, then the submitting button is clicked, the label formula of the natural language can be submitted to a data engine by a background for analysis, the feedback result can be rendered to a page, and business personnel can acquire client group data of charging anxiety in 9 months.
In an embodiment of the present invention, the method may further include the following steps:
generating data to be shared according to the one or more target second user entity data; and sharing the data to be shared through a preset external interface.
In practical application, data to be shared can be generated according to one or more target second user entity data, and data sharing can be performed on the data to be shared through a preset external interface.
Specifically, the client clustering framework based on the double-layer star model can further comprise an external interface module, and a client data interface is provided for the external interface module in a unified manner, so that the external interface module is unified and standardized, popularization and trial are facilitated, for example, in order to support business requirements, data sharing can be performed on a user-defined client group through a preset external interface, and data to be shared can comprise client basic information data, client tag data and client group data.
Aiming at a client clustering framework based on a double-layer star model, the matching correlation between a first-layer star model distributed on a plurality of warehouse layers and a double-layer star model distributed on a second-layer star model of a display layer is adopted, the quick response of client clustering marketing and clustering operation requirements is solved, a self-service tool is formed by UI (user interface) of a service tag language, after a user selects a tag, an SQL (structured query language) script corresponding to the tag can be obtained through natural language analysis, a plurality of executable SQL tasks can be generated, a data engine can be adopted to obtain target data, feedback and downloading are provided, the processing efficiency of data requirements is improved, the repeated workload of data developers can be reduced, and the custom client clustering tool can be used by service personnel without technical backgrounds.
In an example, the user-defined client clustering tool based on the double-layer star model can also manage the user rights, such as data rights, menu rights, button rights and the like, and can also manage the rights aging, the user visible data range and the user operable data range.
In the embodiment of the invention, by presetting a double-layer star model, the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data, and further responding to a user grouping request based on one or more target label data, one or more target second user entity data corresponding to one or more target label data are determined from one or more second user entity data of the second-layer star model, so as to realize the tool requirement for constructing a user-defined client group, and by means of a client grouping framework based on the double-layer star model, the requirement change can be flexibly adapted, the service expansion is supported, and the technical use threshold is reduced, the method has the advantages of flexibility, expandability and usability.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 4, a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention is shown, which may specifically include the following modules:
a double-layer star model presetting module 401, configured to preset a double-layer star model; the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, and the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data;
a target second user entity data determining module 402, configured to determine, in response to a user clustering request based on one or more target tag data, one or more target second user entity data corresponding to the one or more target tag data from the one or more second user entity data of the second layer star model.
In an embodiment of the present invention, a mapping relationship is set between the first-layer star model and the second-layer star model, and data in the first-layer star model is synchronously mapped to data in the second-layer star model through the mapping relationship.
In an embodiment of the present invention, the first-layer star model includes one or more first user entity data and one or more dimensions of service data corresponding to each first user entity data, and the two-layer star model presetting module 401 includes:
the first-layer star model building submodule is used for building a first-layer star model according to the obtained one or more first user entity data and the service data of one or more dimensions corresponding to each first user entity data;
the tag data obtaining sub-module is used for mapping one or more first user entity data in the first layer of star model and one or more dimensionality service data corresponding to each first user entity data according to the mapping relation to obtain one or more second user entity data and one or more tag data corresponding to each second user entity data;
and the second-layer star model building submodule is used for building a second-layer star model according to the one or more second user entity data and the one or more label data corresponding to each second user entity data.
In an embodiment of the present invention, the method further includes:
a visualization module to visualize the one or more tag data.
In an embodiment of the present invention, the method further includes:
the data to be shared generating module is used for generating data to be shared according to the one or more target second user entity data;
and the data sharing module is used for sharing the data to be shared through a preset external interface.
In an embodiment of the present invention, each first user entity data corresponds to a main table in the first layer star model, and the service data of each dimension corresponds to a dimension table associated with the main table in the first layer star model;
each second user entity data corresponds to a main table in the second-layer star model, and each label data corresponds to a dimension table associated with the main table in the second-layer star model.
In an embodiment of the present invention, the data in the first layer star model is stored in a data warehouse, and the data in the second layer star model is stored in a relational database.
In the embodiment of the invention, by presetting a double-layer star model, the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data, and further responding to a user grouping request based on one or more target label data, one or more target second user entity data corresponding to one or more target label data are determined from one or more second user entity data of the second-layer star model, so as to realize the tool requirement for constructing a user-defined client group, and by means of a client grouping framework based on the double-layer star model, the requirement change can be flexibly adapted, the service expansion is supported, and the technical use threshold is reduced, the method has the advantages of flexibility, expandability and usability.
An embodiment of the present invention also provides a server, which may include a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and when executed by the processor, the computer program implements the method for processing data as above.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the above data processing method.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The method and apparatus for data processing provided above are described in detail, and a specific example is applied herein to illustrate the principles and embodiments of the present invention, and the above description of the embodiment is only used to help understand the method and core ideas of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method of data processing, the method comprising:
presetting a double-layer star model; the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, and the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data;
in response to a user clustering request based on one or more target tag data, determining one or more target second user entity data corresponding to the one or more target tag data from the one or more second user entity data of the second layer star model.
2. The method according to claim 1, wherein a mapping relationship is provided between the first-layer star model and the second-layer star model, and data in the first-layer star model is synchronously mapped to data in the second-layer star model through the mapping relationship.
3. The method of claim 2, wherein the first-level star model comprises one or more first user entity data and one or more dimensions of business data corresponding to each first user entity data, and wherein the pre-setting two-level star model comprises:
constructing the first-layer star model according to the acquired one or more first user entity data and the one or more dimensionality service data corresponding to each first user entity data;
mapping one or more first user entity data in the first layer star model and one or more dimensionality service data corresponding to each first user entity data according to the mapping relation to obtain one or more second user entity data and one or more label data corresponding to each second user entity data;
and constructing a second layer star model according to the one or more second user entity data and the one or more label data corresponding to each second user entity data.
4. The method of claim 1, 2 or 3, further comprising:
visualizing the one or more tag data.
5. The method of claim 1, further comprising:
generating data to be shared according to the one or more target second user entity data;
and sharing the data to be shared through a preset external interface.
6. The method of claim 3, wherein each first user entity data corresponds to a master table in the first layer star model, and wherein the business data for each dimension corresponds to a dimension table associated with a master table in the first layer star model;
each second user entity data corresponds to a main table in the second-layer star model, and each label data corresponds to a dimension table associated with the main table in the second-layer star model.
7. The method of claim 1, wherein the data in the first tier star model is stored in a data warehouse and the data in the second tier star model is stored in a relational database.
8. An apparatus for data processing, the apparatus comprising:
the double-layer star model presetting module is used for presetting a double-layer star model; the double-layer star model comprises a first-layer star model and a second-layer star model, the first-layer star model provides data support for the second-layer star model, and the second-layer star model comprises one or more second user entity data and one or more label data corresponding to each second user entity data;
a target second user entity data determining module, configured to determine, in response to a user clustering request based on one or more target tag data, one or more target second user entity data corresponding to the one or more target tag data from the one or more second user entity data of the second layer star model.
9. A server comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program, when executed by the processor, implementing a method of data processing according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method of data processing according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011406985.XA CN112527919A (en) | 2020-12-04 | 2020-12-04 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011406985.XA CN112527919A (en) | 2020-12-04 | 2020-12-04 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112527919A true CN112527919A (en) | 2021-03-19 |
Family
ID=74997558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011406985.XA Pending CN112527919A (en) | 2020-12-04 | 2020-12-04 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112527919A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2468406A1 (en) * | 2003-05-27 | 2004-11-27 | Cognos Incorporated | System and method of modelling of a multi-dimensional data source in an entity-relationship model |
| US20120179644A1 (en) * | 2010-07-09 | 2012-07-12 | Daniel Paul Miranker | Automatic Synthesis and Presentation of OLAP Cubes from Semantically Enriched Data Sources |
| CN105787052A (en) * | 2016-02-26 | 2016-07-20 | 广州品唯软件有限公司 | Data processing model building method and data screening method based on data processing model |
| CN105913188A (en) * | 2016-04-13 | 2016-08-31 | 苏州大成电子科技有限公司 | Multidirectional management system and multidirectional management method of RFID supply chain |
| US20170116305A1 (en) * | 2015-10-23 | 2017-04-27 | Numerify, Inc. | Input Gathering System and Method for Refining, Refining or Validating Star Schema for a Source Database |
| JP2018116706A (en) * | 2017-01-22 | 2018-07-26 | 株式会社日立製作所 | Data multidimensional model generation system and data multidimensional model generation method |
| CN108874971A (en) * | 2018-06-07 | 2018-11-23 | 北京赛思信安技术股份有限公司 | A kind of tool and method applied to the storage of magnanimity labeling solid data |
| CN109635029A (en) * | 2018-12-07 | 2019-04-16 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and medium based on label index system |
| CN110019396A (en) * | 2017-12-01 | 2019-07-16 | 中国移动通信集团广东有限公司 | A kind of data analysis system and method based on distributed multidimensional analysis |
| CN112527917A (en) * | 2020-12-04 | 2021-03-19 | 广州橙行智动汽车科技有限公司 | Data processing method and device |
| CN112527918A (en) * | 2020-12-04 | 2021-03-19 | 广州橙行智动汽车科技有限公司 | Data processing method and device |
-
2020
- 2020-12-04 CN CN202011406985.XA patent/CN112527919A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2468406A1 (en) * | 2003-05-27 | 2004-11-27 | Cognos Incorporated | System and method of modelling of a multi-dimensional data source in an entity-relationship model |
| US20120179644A1 (en) * | 2010-07-09 | 2012-07-12 | Daniel Paul Miranker | Automatic Synthesis and Presentation of OLAP Cubes from Semantically Enriched Data Sources |
| US20170116305A1 (en) * | 2015-10-23 | 2017-04-27 | Numerify, Inc. | Input Gathering System and Method for Refining, Refining or Validating Star Schema for a Source Database |
| CN105787052A (en) * | 2016-02-26 | 2016-07-20 | 广州品唯软件有限公司 | Data processing model building method and data screening method based on data processing model |
| CN105913188A (en) * | 2016-04-13 | 2016-08-31 | 苏州大成电子科技有限公司 | Multidirectional management system and multidirectional management method of RFID supply chain |
| JP2018116706A (en) * | 2017-01-22 | 2018-07-26 | 株式会社日立製作所 | Data multidimensional model generation system and data multidimensional model generation method |
| CN110019396A (en) * | 2017-12-01 | 2019-07-16 | 中国移动通信集团广东有限公司 | A kind of data analysis system and method based on distributed multidimensional analysis |
| CN108874971A (en) * | 2018-06-07 | 2018-11-23 | 北京赛思信安技术股份有限公司 | A kind of tool and method applied to the storage of magnanimity labeling solid data |
| CN109635029A (en) * | 2018-12-07 | 2019-04-16 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and medium based on label index system |
| CN112527917A (en) * | 2020-12-04 | 2021-03-19 | 广州橙行智动汽车科技有限公司 | Data processing method and device |
| CN112527918A (en) * | 2020-12-04 | 2021-03-19 | 广州橙行智动汽车科技有限公司 | Data processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 汪辉等: "金融业数据仓库中星型衍生子维度模式", 湖南科技大学学报(自然科学版), no. 01, 31 March 2013 (2013-03-31), pages 80 - 83 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038222B (en) | System of entity-attribute framework for information system modeling and data access | |
| US11526338B2 (en) | System and method for inferencing of data transformations through pattern decomposition | |
| CN101971165B (en) | Graphic representations of data relationships | |
| CN110674228A (en) | Data warehouse model construction and data query method, device and equipment | |
| US8543535B2 (en) | Generation of star schemas from snowflake schemas containing a large number of dimensions | |
| CN110647512B (en) | Data storage and analysis method, device, equipment and readable medium | |
| US9547646B2 (en) | User-created members positioning for OLAP databases | |
| US8682936B2 (en) | Inherited entity storage model | |
| US8762411B2 (en) | Progressive exploration of data relationships | |
| CN112527920A (en) | Data processing method and device | |
| CN112396462B (en) | Crowd circling method and device based on click house | |
| CN104102670A (en) | Performance indicator analytical framework | |
| CN112527918A (en) | Data processing method and device | |
| CN101901277A (en) | Dynamic ontology modeling method and system based on user situation | |
| CN112527917A (en) | Data processing method and device | |
| CN116089490A (en) | Data analysis method, device, terminal and storage medium | |
| CN112527919A (en) | Data processing method and device | |
| CN101159049A (en) | Dynamic configuring arrange method and system | |
| CN114281494A (en) | Data full life cycle management method, system, terminal device and storage medium | |
| CN115017185A (en) | Data processing method, device and storage medium | |
| CN112732841A (en) | Data processing method and device | |
| US20140143278A1 (en) | Application programming interface layers for analytical applications | |
| CN112162971B (en) | Method, device and system for generating multidimensional data cube | |
| KR20170059604A (en) | System for planning document databse based in collection diagram and apparatus used in it | |
| CN116644063A (en) | Label generating method, label generating device, and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |