WO2025256476A1 - 维度数据的管理方法及维度管理系统 - Google Patents
维度数据的管理方法及维度管理系统Info
- Publication number
- WO2025256476A1 WO2025256476A1 PCT/CN2025/099648 CN2025099648W WO2025256476A1 WO 2025256476 A1 WO2025256476 A1 WO 2025256476A1 CN 2025099648 W CN2025099648 W CN 2025099648W WO 2025256476 A1 WO2025256476 A1 WO 2025256476A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- dimension
- graph
- dimensional
- entities
- dimensions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
Definitions
- This application relates to the field of data warehouse analysis, and in particular to a method for managing dimensional data and a dimensional management system.
- This application provides a method and system for managing dimensional data, which can improve the efficiency of dimensional data management.
- the dimension management system can update the first dimension graph according to the dimension update information.
- the entities and relationships between entities represented by the updated second dimension graph can reflect the updated content.
- the dimension management system before the dimension management system obtains dimension update information, the dimension management system further performs the following steps: The dimension management system obtains a dimension update request, wherein the dimension update request includes the identifier of the object to be updated, and the object to be updated is a dimension or a relationship between dimensions. Then, the dimension management system obtains a first sub-graph from the first dimension graph based on the identifier of the object to be updated, and displays the first sub-graph to the user, wherein the first sub-graph represents the object to be updated and its associated objects.
- users can determine which objects might be affected after the object to be updated is updated. This allows users to update the objects affected by the updated object after updating the object they want to update, thereby improving the accuracy of dimensional data updates.
- the dimension management system also acquires multiple dimension models, which describe dimensions or relationships between dimensions. Then, the dimension management system extracts entity information and relationships between multiple entities from these multiple dimension models, and generates a first dimension graph based on this information.
- This first dimension graph is a multi-layered topological structure composed of multiple nodes and edges. Nodes in the first dimension graph indicate entities, and edges indicate the relationships between entities indicated by the nodes at both ends of the edges.
- the entities represented by the first dimension graph also include the business, the dimension table used by the business, and at least one of the dimension table fields.
- the dimension management system also includes a sending module. This module sends notification messages to the user, including information about any anomalies or issues with the dimension update information.
- the acquisition module is further configured to acquire a dimension update request, which includes the identifier of the object to be updated, and the object to be updated is a dimension or a relationship between dimensions.
- the graph management module is further configured to acquire a first sub-graph from the first dimension graph based on the identifier of the object to be updated, and display the first sub-graph to the user, whereby the first sub-graph represents the object to be updated and its associated objects.
- the entities represented by the first dimension graph also include the business, the dimension table used by the business, and at least one of the dimension table fields.
- the computing device includes a processor and a memory, wherein the processor is configured to execute instructions stored in the memory to cause the computing device to perform some or all of the methods described in the first aspect and any implementation thereof.
- the computing device cluster includes at least one computing device, each computing device including a processor and a memory.
- the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, causing the computing device cluster to perform some or all of the methods described in the first aspect and any implementation thereof.
- this application provides a computer program product containing instructions.
- This computer program product may be a software or program product containing instructions that can run on a computing device or be stored on any usable medium.
- the computer program product When the computer program product is run on a computing device, it causes the computing device to perform some or all of the methods described in the first aspect and any implementation thereof.
- the computer storage medium includes computer program instructions that, when executed by a computing device, cause the computing device to perform some or all of the methods described in the first aspect and any implementation thereof.
- Figure 1 is a schematic diagram of a dimensional data management scenario provided in this application.
- Figure 2 is a schematic diagram of an application scenario provided in this application.
- Figure 3 is a flowchart illustrating a method for managing dimensional data provided in this application.
- Figure 4 is a flowchart illustrating a method for constructing a dimensional map provided in this application
- Figures 5A and 5B are schematic diagrams of a dimension map provided in this application.
- Figure 6 is a schematic diagram of the structure of a dimension management system provided in this application.
- Figure 7 is a schematic diagram of the structure of a computing device provided in this application.
- Figure 8 is a schematic diagram of the structure of a computing device cluster provided in this application.
- Figure 9 is a schematic diagram of another computing device cluster provided in this application.
- this application provides a method for managing dimensional data.
- This method can be applied to a dimensional management system.
- the dimensional management system has multiple pre-built dimensional models, which are data models used to describe dimensions or relationships between dimensions.
- the dimensional management system can generate a dimensional graph based on these multiple dimensional models.
- the dimensional graph represents dimensions, the attributes of dimension members, the relationships between dimensions, and the relationships between dimensions and the attributes of dimension members.
- the dimensional management system can check the update for any anomalies that might affect the use of the dimension based on the dimensional graph. If an anomaly is found, the dimensional management system will prompt the user to correct it promptly. This achieves automated checking of updated dimensional data, improving the efficiency of dimensional data management.
- Figure 1 illustrates a dimensional data management scenario to which this application applies.
- the scenario includes a client 100 and a dimensional management system 200, which are connected via a network, which can be a wide area network (WAN) or a local area network (LAN).
- WAN wide area network
- LAN local area network
- Client 100 can be software or applications deployed on a terminal device, such as a browser or application (APP) software.
- the aforementioned terminal devices may include desktop computers, personal computers, smartphones, wearable devices, handheld processing devices, tablet computers, laptop computers, all-in-one handheld devices, in-vehicle devices, smart conferencing devices, augmented reality (AR) devices, virtual reality (VR) devices, etc., and this application does not specifically limit them.
- the dimension management system 200 may include one or more computing devices, which may be servers, desktop computers, laptops, or smartphones.
- the dimension management system 200 includes a computing device 210.
- the computing device 210 may include at least a bus 211, a communication interface 212, a processor 213, and a memory 214, wherein the communication interface 212, the processor 213, and the memory 214 communicate with each other via the bus 211.
- the dimension management system 200 may include more computing devices 210, but for simplicity, Figure 1 only describes one computing device 210 as an example.
- this application does not limit the number of processors 213 and memory 214 in the computing device 210, but for simplicity, Figure 1 only describes one processor 213 and one memory 214 as an example.
- Bus 211 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc., where the Unified Bus is also known as the Lingqu bus.
- PCI Peripheral Component Interconnect
- EISA Extended Industry Standard Architecture
- Ubus or UB Unified Bus
- CXL Compute Express Link
- CCIX Cache Coherent Interconnect for Accelerators
- Bus 211 can include pathways for transmitting information between various components in the computing device 210 (e.g., communication interface 212, processor 213, and memory 214).
- the communication interface 212 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable the computing device 210 to communicate with other devices or communication networks.
- the computing device 210 can communicate with the client 100 through the communication interface 212.
- the processor 213 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD).
- the PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system-on-chip (SoC), an offload card, an accelerator card, or any combination thereof.
- CPLD complex programmable logical device
- FPGA field-programmable gate array
- GAL generic array logic
- DPU data processing unit
- NPU neural network processing unit
- SoC system-on-chip
- Memory 214 may include volatile memory, such as random access memory (RAM). Memory 214 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 214 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.
- RAM random access memory
- ROM read-only memory
- HDD hard disk drive
- SSD solid state drive
- SCM storage class memory
- PCM phase change memory
- the computing device 210 can be configured with the same type of storage medium to realize the function of the memory 214, or it can be configured with two or more types of storage media to realize the function of the memory 214. This application does not limit this.
- the memory 214 stores executable program code.
- the processor 213 can generate a first dimensional graph based on these dimensional models and store the first dimensional graph in the memory 214.
- the first dimensional graph represents the relationships between dimensions and the relationships between the attributes of dimensions and their members.
- the processor 213 can also retrieve the first dimensional graph from the memory 214, identify abnormal content in the dimensional update information based on the first dimensional graph, and then send a prompt message to the client 100 through the communication interface 212 to prompt the user to modify the aforementioned abnormal content.
- dimensional update information i.e., updates made by the user to dimensions or relationships between dimensions
- the Dimension Management System 200 can be deployed in a cloud data center and provided to users as a cloud service based on the basic resources (including computing resources, storage resources, and network resources) provided by the cloud data center.
- the client 100 can be deployed on the terminal devices used by users.
- the cloud management platform provides the Dimension Management System 200 as a cloud service to enterprises. Enterprises can obtain access to the Dimension Management System 200 by purchasing cloud services and provide their employees with accounts for using the aforementioned cloud services. This allows employees to log in to their assigned accounts on the client 100 to manage dimensional data using the Dimension Management System 200.
- Step 101 Client 100 sends a dimension update request to dimension management system 200.
- communication interface 212 in dimension management system 200 receives the dimension update request.
- a dimension update request indicates the object to be updated.
- This object can be the first dimension or the first relationship the user wants to update.
- the first relationship refers to the relationship between the second and third dimensions, and can specifically be a convergence relationship or a combination relationship.
- a convergence relationship means that the second dimension converges to the third dimension; for example, the second dimension is the product dimension sold by the company, and the third dimension is the product line dimension managed by the company.
- a combination relationship means that the third dimension includes some or all of the members of the second dimension; for example, the second dimension is the product dimension sold by the company, and the third dimension is the product portfolio dimension sold by the company.
- the updates a user wants to make to the first dimension can include one or more of the following: adding a first dimension to an existing dimension; deleting an existing first dimension; modifying the first dimension, such as adding new dimension members to the first dimension, deleting existing dimension members in the first dimension, modifying the attribute values of dimension members in the first dimension, modifying the encoding or name of the first dimension, etc.
- the updates a user wants to make to the first relation can include one or more of the following: adding a first relation to the second and third dimensions; deleting the first relation between the second and third dimensions; modifying the first relation, such as changing its type, direction, or other components.
- the type of the first relation is either the convergence relation or the combination relation mentioned above.
- the second dimension converges to the third dimension
- the direction of the first relation is from the third dimension to the second dimension.
- the third dimension includes some or all of the members of the second dimension
- the direction of the first relation is from the second dimension to the third dimension.
- Step 102 Processor 213 obtains the first sub-graph from the first dimension graph according to the dimension update request.
- memory 214 pre-stores a first-dimensional graph, which can be generated by processor 213.
- the generation process is described in Figure 4 and its related description below.
- the first-dimensional graph includes multiple nodes and multiple edges. Nodes in the first-dimensional graph indicate entities, and edges indicate relationships between entities.
- An entity can be a dimension or an attribute of a dimension member.
- an entity can also be a business function, a dimension table used by the business, or a dimension table field. Therefore, processor 213 can retrieve the first-dimensional graph from memory 214 and then, based on the object to be updated indicated by the dimension update request, retrieve the first sub-graph from the first-dimensional graph.
- the dimension update request includes the identifier of the object to be updated.
- Nodes in the first dimension graph also have identifiers.
- the node's identifier can be the dimension's code, name, or other relevant information indicating that dimension.
- the processor 213 obtains the first subgraph from the first dimension graph based on the object to be updated indicated by the dimension update request (i.e., the identifier of the object to be updated), including at least two of the following implementation methods:
- the dimension update request includes the identifier of the first dimension.
- the processor 213 can compare the identifier of the first dimension with the identifiers of nodes in the first-dimensional graph. If the identifier of the first dimension matches the identifier of the first node in the first-dimensional graph, the processor 213 further determines the nodes associated with the first node and the edges between the first node and its associated nodes based on the first-dimensional graph, thus obtaining a first sub-graph.
- the first sub-graph includes the first node, the nodes associated with the first node, and the edges between the first node and its associated nodes. If the identifier of the first dimension does not match the identifiers of any node in the first-dimensional graph, then the first sub-graph is the first-dimensional graph.
- the dimension update request includes the identifiers of the second and third dimensions.
- the processor 213 can compare the identifiers of the second and third dimensions with the identifiers of nodes in the first-dimensional graph to determine the second and third nodes in the first-dimensional graph. Specifically, the identifier of the second node matches the identifier of the second dimension, and the identifier of the third node matches the identifier of the third dimension. That is, the second node indicates the second dimension, and the third node indicates the third dimension. Then, the processor 213 determines whether the second and third nodes are connected by an edge according to the first-dimensional graph.
- the first sub-graph includes the second node, the third node, and the edge between them; if the second and third nodes are not connected by an edge, the first sub-graph includes the second node and the third node.
- the "comparison between identifiers" involved in the above implementation method can be achieved by the processor 213 by calculating the similarity between the identifiers.
- the processor 213 calculates the similarity between the identifiers.
- the similarity between the identifiers is greater than or equal to a threshold, the two identifiers are determined to match; otherwise, when the similarity between the identifiers is less than the threshold, the two identifiers are determined to not match.
- Step 103 Processor 213 instructs communication interface 212 to send the first sub-map to client 100. Accordingly, client 100 receives the first sub-map.
- steps 101 to 103 above are optional.
- the dimension management system 200 can choose to execute steps 101 to 103 above, or choose not to execute steps 101 to 103 above, according to the user's actual needs.
- the communication interface 212 can provide a user interface displaying a "hint” option.
- a "hint” option When the user selects the "hint” option, it indicates that the user wants to view the first sub-map, and the dimension management system 200 will execute steps 101 to 103 as described above. Conversely, when the user does not select the "hint” option, it indicates that the user does not need to view the first sub-map, and the dimension management system 200 will not execute steps 101 to 103 as described above.
- the client 100 can display the first subgraph to the user. Since the first subgraph can represent not only the object to be updated but also its associated objects, specifically, when the object to be updated is a first dimension, the associated objects include the entities indicated by the nodes associated with the first node, and the relationships between entities indicated by the edges between the first node and its associated nodes; when the object to be updated is a first relationship, the associated objects include the second and third dimensions. Therefore, based on the first subgraph, the user can intuitively understand which objects might be affected after the object to be updated is updated. After updating the object to be updated, the user can update the affected objects based on the first subgraph, improving the accuracy of dimension data updates.
- Step 104 Client 100 sends dimension update information to dimension management system 200.
- communication interface 212 in dimension management system 200 receives the dimension update information.
- Dimension update information indicates the object to be updated and the content of the updated object.
- the updated object can be either the first dimension or the first relationship.
- the update content indicates the user's updates to the first dimension, specifically including newly added first dimensions, deleted first dimensions, and modifications made to the first dimension.
- the update content indicates the user's updates to the first relationship, specifically including newly added first relationships, deleted first relationships, and modifications made to the first relationship.
- memory 214 pre-stores multiple dimensional models.
- a dimensional model is a data model used to describe dimensions or relationships between dimensions. These dimensional models can be created by processor 213 according to user configuration; a detailed description can be found in Figure 4 and its related description below.
- Communication interface 212 provides a dimensional model update interface, which can be implemented through an application programming interface (API) or a graphical user interface (GUI).
- API application programming interface
- GUI graphical user interface
- the dimensional model update interface presents the modifiable content of the dimensional models to the user, allowing the user to operate on the dimensional model update interface through client 100 to update the first dimension or the first relationship.
- communication interface 212 can provide a dimension data entry interface, which can be implemented via API or GUI.
- the dimension data entry interface allows users to add a first dimension. It can also present modifiable dimensions to the user, enabling the user to modify the dimension members included in the first dimension or delete the first dimension through client 100 on the dimension data entry interface.
- Step 105 Processor 213 performs an update operation on the first dimension map based on the dimension update information to obtain the second dimension map.
- the processor 213 can perform an update operation on the first dimension map according to the update object and update content of the update object indicated by the dimension update information to obtain the second dimension map.
- the changes between the second dimension map and the first dimension map are the update content of the update object.
- the dimension update information includes the identifier of the first dimension and the update content of the first dimension.
- the processor 213 can compare the identifier of the first dimension with the identifiers of nodes in the first-dimensional graph. If the identifier of the first dimension matches the identifier of the first node in the first-dimensional graph, the processor 213 updates the first node and its associated other nodes and edges according to the update content of the first dimension. If the identifier of the first dimension does not match the identifiers of any node in the first-dimensional graph, the processor 213 adds nodes and edges between nodes in the first-dimensional graph according to the update content of the first dimension.
- the added nodes include nodes indicating the first dimension and nodes indicating the attributes of the dimension members of the first dimension.
- the added edges include the edges between the newly added nodes.
- the dimension update information can include the identifier of the second dimension, the identifier of the third dimension, and the updated content of the first relation.
- the processor 213 can compare the identifiers of the second and third dimensions with the identifiers of nodes in the first-dimensional graph to determine the second and third nodes in the first-dimensional graph, and then determine whether the second and third nodes are connected by an edge based on the first-dimensional graph. If the second and third nodes are connected by an edge, the processor 213 will update the first relation according to the updated content. If the second and third nodes are not connected by an edge, the processor 213 will add an edge to the first-dimensional graph according to the updated content of the first relation; the added edge is used to indicate the first relation.
- Step 106 Processor 213 determines whether there are any abnormalities in the dimension update information based on the second dimension map. If there are, proceed to step 107; otherwise, proceed to step 108.
- the processor 213 can determine whether there are any abnormal issues with the dimension update information based on the entities and relationships between entities represented by the second dimension graph (i.e., the nodes and relationships between nodes in the second dimension graph).
- memory 214 pre-stores graph verification rules. These rules can be set by the user or processor 213 based on graph integrity and conflict checks. Therefore, the graph verification rules are used to verify the integrity of the graph and whether there are conflicting nodes and edges. When the graph does not meet the verification rules, it indicates that the dimensional information contained in the graph is abnormal; conversely, when the graph meets the verification rules, it indicates that the dimensional information contained in the graph is correct. Therefore, processor 213 can determine whether there are any abnormalities in the dimensional update information by judging whether the second-dimensional graph meets the graph verification rules.
- graph verification rules may include one or more of the following: there are no loops in the graph; there are no isolated nodes or edges in the graph. It should be understood that the above examples are merely illustrative of two possible graph verification rules, and other graph verification rules may exist in practical applications, which are not limited in this application.
- Step 107 Processor 213 instructs communication interface 212 to send a prompt message to client 100. Accordingly, client 100 receives the prompt message.
- the prompt message is used to remind users to modify dimension update information.
- the prompt message may include any issues or problems with the dimension update information. It is understood that when the client 100 presents the prompt message to the user, the user can understand the issues with the dimension update information and modify the update information accordingly, thus improving update efficiency.
- Step 108 Processor 213 instructs memory 214 to store dimension update information and the second dimension map.
- the processor 213 can also compare the manually updated second-dimensional map with the second-dimensional map generated in step 105. When differences exist, a difference report is generated based on these differences.
- the difference report includes the differences between the manually updated second-dimensional map and the second-dimensional map automatically generated by the processor 213.
- the processor 213 also instructs the communication interface 212 to provide the difference report to the administrator of the dimension map, enabling the administrator to promptly correct the map generation function provided by the dimension management system 200.
- the processor 213 can also instruct the communication interface 212 to present a second-dimensional graph to the user, allowing the user to select a dimension of interest based on the second-dimensional graph.
- the processor 213 can also generate a dimension consumption model based on the user-selected dimension and the second-dimensional graph, and display the dimension consumption model to the user.
- the dimension consumption model includes relevant information about the dimensions that the user can use, such as the dimension's encoding and its members.
- the processor 213 can also generate recommendation information based on historical dimension update information and historical dimension usage information, and provide the recommendation information to the user.
- the recommendation information is used to indicate the dimensions that the user updates frequently or the dimensions that the user uses frequently, so as to assist the user in selecting the dimension to view.
- Step 201 Processor 213 creates a multi-dimensional model based on the user's initial configuration.
- communication interface 212 provides an access interface, which can be implemented via API or GUI.
- the user can provide a first configuration to processor 213 through this access interface.
- processor 213 obtains the user's first configuration and creates multiple dimensional models based on it.
- the first configuration provided by the user may include specific content from the dimensional models.
- a dimensional model is a data model (specifically, a meta-model) used to describe dimensions or the relationships between dimensions.
- the multiple dimensional models created by the Dimension Management System 200 can be divided into two categories: dimensional tree models and tree-to-tree relationship models. These two types of models are described below:
- a dimension tree model is a data model used to describe dimensions.
- One dimension tree corresponds to one dimension, which represents an attribute or metric of business data and provides a business analysis perspective.
- a dimension can include multiple dimension members, with hierarchical relationships between them. Different levels represent different granularities of describing business data.
- a time dimension can include year and month levels. The year level represents business data described at the year level (e.g., product sales in a given year), and the month level represents business data described at the month level (e.g., product sales in a given month). Therefore, multiple dimension members of a dimension can form a tree structure, i.e., a dimension tree.
- a dimension tree model can also be understood as a data model used to describe dimension trees.
- a dimension tree model can include the following: the dimension name, dimension code, dimension type, the user responsible for maintaining the dimension (hereinafter referred to as the "dimensional administrator"), the dimension display method, the code and name of the attributes of the dimension members, and the attribute values of the dimension members, etc., all information related to the dimension.
- the dimension name can be set by the dimension administrator and can specifically be the dimension's name.
- the dimension code can be set by the dimension management system 200 and can specifically be the dimension's code.
- the dimension type can include two categories: basic type and composite type.
- Basic type dimensions refer to the basic dimensions set by the dimension administrator, while composite type dimensions are dimensions created by combining dimension members from basic type dimensions based on different business analysis perspectives.
- the dimension display method can include displaying it through text boxes, dropdown lists, etc.
- the inter-tree relationship model is a data model used to describe the relationships between dimensions. Since dimensions can be represented using a dimension tree, the inter-tree relationship model can also be understood as a data model used to describe the relationships between dimension trees.
- a tree-based relationship model describes the relationship between a dimension (hereinafter referred to as the "primary dimension") and its associated dimensions (hereinafter referred to as "associated dimensions").
- the relationship between the primary dimension and associated dimensions can be a convergence relationship or a combination relationship.
- a convergence relationship generally refers to multiple dimensions converging to a single dimension. For example, multiple products sold by a company correspond to multiple product dimensions, and the product lines managed by the company correspond to the product line dimension. There is a convergence relationship between these multiple product dimensions and the product line dimension, and the multiple product dimensions converge to the product line dimension.
- a combination relationship refers to one dimension including some or all of the members of another dimension. For example, the product portfolio sold by a company corresponds to the product portfolio dimension.
- the product portfolio dimension includes some of the members of the product dimensions. It should be understood that, in addition to the above convergence and combination relationships, other relationships may exist between the primary dimension and associated dimensions, depending on the actual business or enterprise requirements.
- the tree-based relationship model can include the following: information about the primary dimension, information about related dimensions, the type and direction of the relationship between the primary and related dimensions, and the user responsible for the above relationship (hereinafter referred to as the "relationship administrator"), etc., related to the relationship between dimensions.
- the information about the primary dimension can include its name, code, etc.
- the information about related dimensions can include their names, codes, etc.
- the relationship between the primary and related dimensions can be of two types: convergence relationship and combination relationship.
- the direction of a convergence relationship points to the converged dimension (e.g., from the product dimension to the product line dimension), and the direction of a combination relationship points to the combined dimension (e.g., from the product dimension to the product combination dimension).
- Step 202 Processor 213 generates the first-dimensional map based on the multi-dimensional model.
- processor 213 extracts entity information and relationships between multiple entities from a multi-dimensional model.
- the entity information can be the name or code of the dimension, or other information indicating that dimension.
- the entity information can be the name or code of the attribute, or other information indicating that attribute.
- Relationships between entities can be relationships between dimensions, or relationships between a dimension and the attributes of its members. Relationships between a dimension and the attributes of its members are compositional relationships, meaning that a dimension includes the attributes of its members.
- processor 213 generates a first-dimensional graph based on the multiple entities and their relationships.
- the first-dimensional graph is a multi-layered topological structure composed of multiple nodes and edges. Nodes in the first-dimensional graph indicate entities, and edges indicate the relationships between entities indicated by the nodes at both ends of the edges.
- the dimensional model is stored in JSON file format, which includes "keys” and "values.”
- the "keys” are used to access the specific content contained in the JSON file.
- processor 213 can extract the dimension name (or encoding), the attribute name (or attribute encoding) of dimension members, and the relationship between the attributes of dimensions and dimension members from the dimensional tree model through keyword detection, and extract the relationship between dimensions from the tree relationship model.
- the keywords used here can be the "keys” in the JSON file, such as "dimensional name,” “dimensional encoding,” “attribute name of dimension member,” “attribute encoding of dimension member,” “relationship between dimensions,” etc.
- the processor 213 may further perform the following steps: The processor 213 updates the first dimension graph according to the user's second configuration.
- the user's second configuration may include a business identifier, the identifier of the dimension table used by the business, the identifier of the dimension that has a mapping relationship with the dimension table, the identifier of the dimension table field, the identifier of the attribute of the dimension member that has a mapping relationship with the dimension table field, and the compositional relationship between the dimension table and the dimension table field.
- the processor 213 may compare the identifier of the dimension and the identifier of the attribute of the dimension member in the second configuration with the identifier of the node in the first dimension graph, and update the first dimension graph according to the comparison result.
- the new first dimension graph adds multiple nodes and multiple edges.
- the entities indicated by the newly added nodes can be businesses, dimension tables used by the business, or dimension table fields.
- the newly added edges are used to indicate the authorization relationship between the business and the dimension table, the mapping relationship between the dimension table and the dimension, the compositional relationship between the dimension table and the dimension table field, and the mapping relationship between the dimension table field and the attribute of the dimension member.
- This dimensionality graph includes five nodes, which are used to indicate dimension A, dimension B, dimension C, dimension member attribute 1, and dimension member attribute 2, respectively.
- Dimension A includes dimension member attribute 1 and dimension member attribute 2.
- Dimension A and dimension B have a combination relationship, and dimension B includes dimension member attribute 1.
- Dimension A and dimension C have a convergence relationship, and dimension A converges to dimension C.
- Dimension table 1 includes dimension table field 1 and dimension table field 2.
- dimension table field 1 includes dimension table field 1 and dimension table field 2.
- mapping relationship between dimension table field 1 and dimension member attribute 1 includes a mapping relationship between dimension table field 2 and dimension member attribute 2.
- processor 213 adds 4 nodes, which are used to indicate business D, dimension table 1, dimension table field 1, and dimension field 2, respectively.
- processor 213 establishes the edges between the node corresponding to business D and the node corresponding to dimension table 1, the edges between the node corresponding to dimension table 1 and the node corresponding to dimension tree A, the relationships between the node corresponding to dimension table 1 and the node corresponding to dimension table field 1, the edges between the node corresponding to dimension table 1 and the node corresponding to dimension member attribute 1, and the edges between the node corresponding to dimension table field 2 and the node corresponding to dimension member attribute 2, thereby obtaining the new dimension graph as shown in Figure 5B.
- the processor 213 can also instruct the communication interface 212 to present a new first-dimensional graph to the user, allowing the user to add new services, new dimension tables, or new dimension table fields to the dimension management system 200 based on the new first-dimensional graph, or delete existing dimension tables or dimension table fields, or modify the authorization relationship between services and dimension tables, etc.
- the processor 213 can also check whether there are any abnormal problems in the user's update based on the new first-dimensional graph, and if there are any abnormal problems in the user's update, prompt the user to update again.
- the specific implementation process is similar to the implementation process described in steps 105 to 108 above, so it will not be described in detail here.
- the dimensional management system 200 can configure multiple dimensional models based on a metamodel and generate a dimensional graph based on these models.
- the dimensional management system 200 can determine whether the user's update has any abnormalities based on the aforementioned dimensional graph. Therefore, the method provided in this application can automatically check the user's updated content, improving the efficiency of dimensional data management.
- the dimensional management system 200 can present the dimensional graph to the user to assist in maintaining the dimensional data, lowering the maintenance threshold for dimensional data.
- Figure 6 shows a schematic diagram of the structure of a dimension management system 200. It should be understood that Figure 6 is only an exemplary illustration of one way to divide the structure of the dimension management system 200. In practical applications, the structure of the dimension management system 200 can also be divided in other ways, which are not limited in this embodiment.
- the dimension management system 200 includes an acquisition module 301, a map management module 302, a data verification module 303, a sending module 304, a storage module 305, and a configuration module 306.
- the acquisition module 301 is used to perform the step of receiving the dimension update request in step 101 above; step 104 above; receiving the user's first configuration in step 201 above; and receiving the user's second configuration.
- the graph management module 302 is used to perform the steps 102 above; step 105 above; step 108 above; and step 202 above; update the first dimension graph according to the user's second configuration; compare the manually updated second dimension graph with the second dimension graph generated in step 105 above, and generate a difference report based on the difference when there is a difference between the two; generate a dimension consumption model based on the dimension selected by the user and the second dimension graph; generate recommendation information based on historical dimension update information and historical dimension usage information; check whether there are any abnormal problems in the user's update based on the new first dimension graph, and prompt the user to update again if there are any abnormal problems in the user's update.
- the data verification module 303 is used to perform the step 106 above.
- the sending module 304 is used to perform the steps 103 above; and step 107 above; provide a difference report to the dimension graph administrator; display the dimension consumption model to the user; display the recommendation information to the user; and display the new first dimension graph to the user.
- Storage module 305 is used to perform the steps of storing dimension update information and the second dimension map in step 108 above; storing multiple dimension models; storing the first dimension map; storing the new first dimension map; and storing the dimension consumption model.
- Configuration module 306 is used to perform step 201 above.
- the acquisition module 301, the map management module 302, the data verification module 303, the sending module 304, the storage module 305, and the configuration module 306 can all be implemented in software or in hardware.
- the implementation of the acquisition module 301 will be described below.
- the implementation of the map management module 302, the data verification module 303, the sending module 304, the storage module 305, and the configuration module 306 can refer to the implementation of the acquisition module 301.
- the acquisition module 301 may include code running on a computing instance.
- the computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more.
- the acquisition module 301 may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts/virtual machines/containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.
- AZ availability zone
- VPC Virtual Private Cloud
- multiple hosts/virtual machines/containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs.
- VPC Virtual Private Cloud
- a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.
- the acquisition module 301 may include at least one computing device, such as a server.
- the acquisition module 301 may also be a device implemented using a CPU, ASIC, or PLD.
- the aforementioned PLD may be implemented using a CPLD, FPGA, GAL, DPU, NPU, SoC, offload card, accelerator card, or any combination thereof.
- the multiple computing devices included in the acquisition module 301 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 301 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 301 can be distributed in the same VPC or in multiple VPCs. Furthermore, the multiple computing devices included in the acquisition module 301 can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offloading cards, and accelerator cards.
- AZ Availability Zone
- VPC VPC
- the multiple computing devices included in the acquisition module 301 can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offloading cards, and accelerator cards.
- the acquisition module 301 can be used to execute any step in the above-described dimension management method and dimension graph construction method; the graph management module 302 can be used to execute any step in the above-described dimension management method and dimension graph construction method; the data verification module 303 can be used to execute any step in the above-described dimension management method and dimension graph construction method; the sending module 304 can be used to execute any step in the above-described dimension management method and dimension graph construction method; the storage module 305 can be used to execute any step in the above-described dimension management method and dimension graph construction method; and the configuration module 306 can be used to execute any step in the above-described dimension management method and dimension graph construction method.
- the steps implemented by the acquisition module 301, the map management module 302, the data verification module 303, the sending module 304, the storage module 305, and the configuration module 306 can be specified according to actual needs.
- the acquisition module 301, the map management module 302, the data verification module 303, the sending module 304, the storage module 305, and the configuration module 306 respectively implement different steps in the above-mentioned dimension management method and dimension map construction method to realize all the functions of the dimension management system 200.
- the computing device can be a server, such as a central server, an edge server, or a local server in a local data center.
- the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.
- Figure 7 exemplarily illustrates a schematic diagram of the computing device provided in this application.
- the computing device 400 includes a bus 401, a processor 402, a memory 403, and a communication interface 404, and the processor 402, the memory 403, and the communication interface 404 communicate with each other via the bus 401.
- this application does not limit the number of processors 402 and memory 403 in the computing device 400.
- Figure 7 describes an example using one processor 402 and one memory 403.
- Bus 401 can be a PCIe bus, EISA bus, UB, CXL, CCIX, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in Figure 7, but this does not mean that the computing device 400 has only one bus or one type of bus. Bus 401 can include pathways for transmitting information between various components of the computing device 400 (e.g., processor 402, memory 403, and communication interface 404).
- the processor 402 may include any one or more computing units with computing capabilities, such as CPU, GPU, MP or DSP, ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.
- Memory 403 may include volatile memory, such as RAM. Memory 403 may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD. In addition, memory 403 may be implemented using SCM, PCM, or other types of storage media.
- the memory 403 stores executable program code.
- the processor 402 executes this executable program code to perform the operation steps performed by the processor 213 in Figures 3 and 4, realizing the functions of the acquisition module 301, the map management module 302, the data verification module 303, the sending module 304, the storage module 305, and the configuration module 306, thereby realizing the aforementioned dimension management method and dimension map construction method. That is to say, the memory 403 stores instructions for executing the aforementioned dimension management method and dimension map construction method.
- the communication interface 404 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 400 and other devices or communication networks. For example, the computing device 400 communicates with the client 100 through the communication interface 404.
- transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 400 and other devices or communication networks. For example, the computing device 400 communicates with the client 100 through the communication interface 404.
- the computing device 400 provided according to this application can correspond to the dimension management system 200 shown in FIG1 provided according to this application, and can correspond to the dimension management system 200 executing the methods shown in FIG3 and FIG4 provided according to this application.
- the above and other operations and/or functions of each module in the computing device 400 are respectively for implementing the corresponding processes of each method in FIG3 and FIG4. For the sake of brevity, they will not be described in detail here.
- the computing device cluster includes at least one computing device.
- the computing device can be a server, such as a central server, an edge server, or a local server in a local data center.
- the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.
- Figure 8 exemplarily illustrates the structural diagram of the computing device cluster provided in this application.
- the computing device cluster 500 includes at least one computing device 400.
- the memory 403 of one or more computing devices 400 in the computing device cluster 500 may store the same instructions for executing the above-described dimension management method and dimension map construction method.
- the memory 403 of one or more computing devices 400 in the computing device cluster 500 may also store partial instructions for executing the above-mentioned dimension management method and dimension graph construction method, that is, a combination of one or more computing devices 400 can jointly execute the above-mentioned dimension management method and dimension graph construction method.
- the memory 403 in different computing devices 400 within the computing device cluster 500 can also store different instructions, which are used to execute certain functions of the dimension management system 200. That is, the instructions stored in the memory 403 of different computing devices 400 can implement the functions of one or more of the modules mentioned above, including the acquisition module 301, the map management module 302, the data verification module 303, the sending module 304, the storage module 305, and the configuration module 306.
- multiple computing devices 400 in the computing device cluster 500 can be connected via a network, which can be a wide area network (WAN) or a local area network (LAN).
- a network which can be a wide area network (WAN) or a local area network (LAN).
- Figure 9 illustrates one possible implementation.
- computing devices 400A and 400B are connected via a network. Specifically, they are connected to the network through communication interfaces in computing devices 400A and 400B.
- the memory 403 in both computing devices 400A and 400B stores instructions for executing the functions of the acquisition module 301, the map management module 302, the data verification module 303, the transmission module 304, the storage module 305, and the configuration module 306.
- connection method between the computing device cluster 500 shown in Figure 9 can be considered for high-concurrency computing scenarios. For example, if computing device 400A receives a large amount of dimension update information in the same time period, in order to respond to these dimension update information as quickly as possible, computing device 400A can delegate some dimension update tasks to computing device 400B. It should be understood that the function of computing device 400A in Figure 9 can also be performed by multiple computing devices 400, and similarly, the function of computing device 400B can also be performed by multiple computing devices 400.
- the computing device cluster 500 provided according to this application can correspond to the dimension management system 200 shown in Figure 1 of this application, and can correspond to the dimension management system 200 that executes the methods shown in Figures 3 and 4 of this application.
- the above and other operations and/or functions of each module in the computing device cluster 500 are respectively for implementing the corresponding processes of each method in Figures 3 and 4. For the sake of brevity, they will not be described in detail here.
- This application also provides a computer program product containing instructions.
- This computer program product may be a software or program product containing instructions that can run on a computing device or be stored on any available medium. When the computer program product is run on a computing device, it causes the computing device to perform the task execution method described above.
- the computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media.
- the aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).
- the computer-readable storage medium includes instructions that instruct the computing device to perform the task execution method described above.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请提供了一种维度数据的管理方法及维度管理系统,能够提高维度数据管理的效率。上述维度数据的管理方法可应用于维度管理系统。具体地,维度管理系统获取维度更新信息,维度更新信息包括更新内容,更新内容包括维度或维度间关系的更新。维度管理系统根据维度更新信息对第一维度图谱执行更新操作,得到第二维度图谱,其中,第二维度图谱与第一维度图谱变化的内容为上述更新内容,第一维度图谱用于表示多个实体及多个实体间的关系,第一维度图谱所表示的实体包括维度或维度成员的属性。之后,维度管理系统根据第二维度图谱所表示的实体及实体间的关系,确定上述维度更新信息存在的异常问题,这里的异常问题包括影响维度使用的问题。
Description
本申请要求在2024年6月13日提交中国国家知识产权局、申请号为202410764025.2的中国专利申请的优先权,发明名称为“维度数据的管理方法及维度管理系统”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及数据仓库分析领域,尤其涉及一种维度数据的管理方法及维度管理系统。
数据仓库(data warehouse,DW)是为了帮助企业制定决策而设计的一种数据管理系统。在数据仓库中,数据通常以多维数据模型的形式进行组织,多维数据模型是基于维表和事实表所构建的,其中,维表描述了数据的各种维度,事实表包含了这些维度关联的业务数据,通过维表和事实表的组合可以将数据组织成多维度的数据立方体(data cube),方便用户从不同的维度分析数据。
随着业务的发展,分析需求也会相应地发生变化。为了支持新的分析需求,通常采用人工补录数据的方式来维护数据仓库中的维表(例如,新增维度,或修改维度的属性等)。目前,补录数据的准确性也需要人为进行检查。可以理解的是,当业务涉及的维度多且维度之间的关系复杂时,如果完全依赖人为检查,则需要耗费大量的时间,维度数据管理的效率低。
本申请提供了一种维度数据的管理方法及维度管理系统,能够提高维度数据管理的效率。
第一方面,本申请提供了一种维度数据的管理方法。该方法可以应用于维度管理系统。具体地,维度管理系统获取维度更新信息,其中,维度更新信息包括更新内容,更新内容包括维度或维度间关系的更新。维度管理系统还根据维度更新信息对第一维度图谱执行更新操作,得到第二维度图谱,其中,第二维度图谱与第一维度图谱变化的内容为上述更新内容,第一维度图谱用于表示多个实体及多个实体间的关系,第一维度图谱所表示的实体包括维度或维度成员的属性。之后,维度管理系统根据第二维度图谱所表示的实体及实体间的关系,确定上述维度更新信息存在的异常问题,这里的异常问题包括影响维度使用的问题。
本申请提供的技术方案中,维度管理系统可以根据维度更新信息对第一维度图谱进行相应的更新,更新所得的第二维度图谱所表示的实体及实体间的关系能够反映更新内容,通过对第二维度图谱进行校验以确定更新内容中是否存在异常问题。因此,通过上述方案可以实现自动化地对更新内容进行校验,提高维度数据管理的效率。
在一种可能的实现方式中,维度管理系统还向用户发送提示信息,其中,提示信息包括维度更新信息存在的异常问题。
通过上述实现方式,用户可以了解维度更新信息存在的异常问题,并基于上述异常问题重新修改维度更新信息,提高更新效率。
在一种可能的实现方式中,在维度管理系统获取维度更新信息之前,维度管理系统还执行以下步骤:维度管理系统获取维度更新请求,其中,维度更新请求包括待更新对象的标识,待更新对象为维度或维度间的关系。之后,维度管理系统根据待更新对象的标识从第一维度图谱中获取第一子图谱,并向用户展示第一子图谱,其中,第一子图谱用于表示待更新对象及待更新对象的关联对象。
通过上述实现方式,用户可以确定待更新对象被更新后,哪些对象可能会受到影响,使得用户在对想要更新的对象更新完毕后,对上述更新对象所影响的对象进行更新,提高维度数据更新的准确性。
在一种可能的实现方式中,维度管理系统还获取多个维度模型,其中,维度模型用于描述维度或维度间的关系。之后,维度管理系统从上述多个维度模型中抽取多个实体的实体信息及多个实体间的关系,并根据多个实体的实体信息及多个实体间的关系,生成第一维度图谱,其中,第一维度图谱是由多个节点和多条边构成的多层拓扑结构,第一维度图谱中的节点用于指示实体,第一维度图谱中的边用于指示位于边两端的节点所指示的实体间的关系。
在一种可能的实现方式中,第一维度图谱所表示的实体还包括业务、业务使用的维表及维表字段中的至少一个。
第二方面,本申请提供了一种维度管理系统。该系统包括获取模块、图谱管理模块和数据校验模块。获取模块用于获取维度更新信息,其中,维度更新信息包括更新内容,更新内容包括维度或维度间关系的更新。图谱管理模块用于根据维度更新信息对第一维度图谱执行更新操作,得到第二维度图谱,其中,第二维度图谱与第一维度图谱变化的内容为上述更新内容,第一维度图谱用于表示多个实体及多个实体间的关系,第一维度图谱所表示的实体包括维度或维度成员的属性。数据校验模块用于根据第二维度图谱所表示的实体及实体间的关系,确定上述维度更新信息存在的异常问题,这里的异常问题包括影响维度使用的问题。
在一种可能的实现方式中,维度管理系统还包括发送模块。发送模块用于向用户发送提示信息,其中,提示信息包括维度更新信息存在的异常问题。
在一种可能的实现方式中,获取模块还用于获取维度更新请求,维度更新请求包括待更新对象的标识,待更新对象为维度或维度间的关系。图谱管理模块还用于根据待更新对象的标识从第一维度图谱中获取第一子图谱,并向用户展示第一子图谱,第一子图谱用于表示待更新对象及待更新对象的关联对象。
在一种可能的实现方式中,获取模块还用于获取多个维度模型,其中,维度模型用于描述维度或维度间的关系。图谱管理模块还用于从上述多个维度模型中抽取多个实体的实体信息及多个实体间的关系,并根据多个实体的实体信息及多个实体间的关系,生成第一维度图谱,其中,第一维度图谱是由多个节点和多条边构成的多层拓扑结构,第一维度图谱中的节点用于指示实体,第一维度图谱中的边用于指示位于边两端的节点所指示的实体间的关系。
在一种可能的实现方式中,第一维度图谱所表示的实体还包括业务、业务使用的维表及维表字段中的至少一个。
第三方面,本申请提供了一种计算设备。该计算设备包括处理器和存储器,其中,处理器用于执行存储器中存储的指令,以使该计算设备执行前述第一方面及其任一种实现方式所描述的部分或全部方法。
第四方面,本申请提供了一种计算设备集群。该计算设备集群包括至少一个计算设备,每个计算设备包括处理器和存储器。上述至少一个计算设备的处理器用于执行上述至少一个计算设备的存储器中存储的指令,以使得上述计算设备集群执行前述第一方面及其任一种实现方式所描述的部分或全部方法。
第五方面,本申请提供了一种包含指令的计算机程序产品。该计算机程序产品可以是包含指令的、能够运行在计算设备上或被储存在任何可用介质中的软件或程序产品。当该计算机程序产品在计算设备上运行时,使得该计算设备执行前述第一方面及其任意一种实现方式所描述的部分或全部方法。
第六方面,本申请提供了一种计算机可读存储介质。该计算机存储介质包括计算机程序指令,当所述计算机程序指令被计算设备执行时,使得该计算设备执行前述第一方面及其任意一种实现方式所描述的部分或全部方法。
图1是本申请提供的一种维度数据的管理场景的示意图;
图2是本申请提供的一种应用场景的示意图;
图3是本申请提供的一种维度数据的管理方法的流程示意图;
图4是本申请提供的一种维度图谱的构建方法的流程示意图;
图5A至图5B是本申请提供的一种维度图谱的示意图;
图6是本申请提供的一种维度管理系统的结构示意图;
图7是本申请提供的一种计算设备的结构示意图;
图8是本申请提供的一种计算设备集群的结构示意图;
图9是本申请提供的另一种计算设备集群的结构示意图。
为了解决人为检查补录数据效率低的问题,本申请提供了一种维度数据的管理方法。该方法可以应用于维度管理系统。维度管理系统预置有多个维度模型,维度模型是用于描述维度或维度间关系的数据模型。维度管理系统可以根据上述多个维度模型生成维度图谱,其中,维度图谱用于表示维度、维度成员的属性、维度间的关系、以及维度与维度成员的属性之间的关系。当用户对某个维度或维度间的关系进行更新后,维度管理系统可以基于上述维度图谱检查用户的更新是否存在影响维度使用的异常问题。其中,当用户的更新存在异常问题时,维度管理系统会提示用户及时修正上述异常问题。由此,实现自动化的检查更新的维度数据,提高维度数据管理的效率。
下面结合附图,对本申请提供的技术方案进行描述。
参见图1,图1示出了本申请所适用的一种维度数据的管理场景。如图1所示,该场景包括客户端100和维度管理系统200,客户端100和维度管理系统200通过网络连接,该网络可以是广域网或局域网。
客户端100可以是部署在终端设备上的软件或应用程序,例如,浏览器或应用(application,APP)软件。上述终端设备可以包括台式机、个人电脑、智能手机、可穿戴设备、掌上处理设备、平板电脑、笔记本电脑、一体化掌机、车载设备、智能会议设备、增强现实(augmented reality,AR)设备、虚拟现实(virtual reality,VR)设备等,对此本申请不作具体限定。
维度管理系统200可以包括一个或多个计算设备,上述计算设备可以是服务器,也可以是台式机、笔记本电脑或者智能手机等终端设备。
如图1所示,维度管理系统200包括一个计算设备210,计算设备210至少可以包括总线211、通信接口212、处理器213和存储器214,其中,通信接口212、处理器213和存储器214之间通过总线211通信。应理解,在具体实现中,维度管理系统200还可以包括更多的计算设备210,但为了简便,图1仅以一个计算设备210为例进行描述。同样,本申请也不限定计算设备210中的处理器213和存储器214的个数,但为了简便,图1仅以一个处理器213和一个存储器214为例进行描述。
总线211可以是外设部件互连标准(peripheral component interconnect,PCI)总线或扩展工业标准结构(extended industry standard architecture,EISA)总线、统一总线(unified bus,Ubus或UB)、计算机快速链接(compute express link,CXL)、缓存一致互联协议(cache coherent interconnect for accelerators,CCIX)等,其中,统一总线也被称为灵衢总线。总线可以分为地址总线、数据总线、控制总线等。为了便于表示,图1中仅用一条线表示,但这并不表示计算设备210仅有一根总线或一种类型的总线。总线211可以包括在计算设备210中的各个部件(例如,通信接口212、处理器213和存储器214)之间传送信息的通路。
通信接口212使用例如但不限于网络接口卡、收发器一类的收发模块,来实现计算设备210与其他设备或通信网络的通信,例如,计算设备210可通过通信接口212与客户端100通信。
处理器213可以包括中央处理单元(central processing unit,CPU)、专用集成电路(application-specific integrated circuit,ASIC)、或者可编程逻辑器件(programmable logic device,PLD)。上述PLD可以是复杂程序逻辑器件(complex programmable logical device,CPLD)、现场可编程门阵列(field-programmable gate array,FPGA)、通用阵列逻辑(generic array logic,GAL)、数据处理单元(data processing unit,DPU)、神经网络处理单元(neural network processing unit,NPU)、片上系统(system on chip,SoC)、卸载卡、加速卡或其任意组合。
存储器214可以包括易失性存储器(volatile memory),例如,随机存取存储器(random access memory,RAM)。存储器214还可以包括非易失性存储器(non-volatile memory),例如,只读存储器(read-only memory,ROM)、快闪存储器、机械硬盘(hard disk drive,HDD)或固态硬盘(solid state drive,SSD)。此外,存储器214还可以通过存储级内存(storage class memory,SCM)、相变存储(phase change memory,PCM)、或其他类型存储介质实现。
值得说明的是,计算设备210中可配置相同类型的存储介质以实现存储器214的功能,也可以配置两种或两种以上类型的存储介质以实现存储器214的功能,本申请对此不作限定。
存储器214中存储有可执行的程序代码。处理器213通过执行存储器214中的程序代码,可以实现本申请提供的维度数据的管理方法。具体地,存储器214预先存储有多个维度模型,维度模型是用于描述维度或维度之间的关系的数据模型。处理器213可以根据这些维度模型生成第一维度图谱,并将第一维度图谱存储至存储器214,其中,第一维度图谱用于表示维度与维度之间的关系,以及维度与维度成员的属性之间的关系。当通信接口212接收到客户端100发送的维度更新信息(即用户对维度或维度之间的关系作出的更新)时,处理器213还可以从存储器214获取第一维度图谱,并根据第一维度图谱识别出维度更新信息中的异常内容,然后通过通信接口212向客户端100发送提示信息,以提示用户修改上述异常内容。
作为一种可能的实现方式,维度管理系统200可以部署在云数据中心,并基于云数据中心提供的基础资源(包括计算资源、存储资源和网络资源)以云服务的形式提供给用户,客户端100可以部署在用户使用的终端设备上。如图2所示,云管理平台将维度管理系统200作为一项云服务提供给企业,企业可以通过购买云服务的方式,获取维度管理系统200的使用权限,并向企业员工提供上述云服务的使用账号,使得企业员工可以基于客户端100登陆各自分配到的使用账号使用维度管理系统200实现维度数据的管理。
接下来,结合图3示出的维度数据的管理方法的流程示意图,对上述维度管理系统200实现维度数据的管理的过程进行更为详细地介绍。
步骤101、客户端100向维度管理系统200发送维度更新请求。相应地,维度管理系统200中的通信接口212接收维度更新请求。
维度更新请求用于指示待更新对象,待更新对象可以是用户想要更新的第一维度或第一关系,第一关系是指第二维度与第三维度之间的关系,第一关系具体可以是收敛关系或组合关系。其中,收敛关系是指第二维度收敛于第三维度,例如,第二维度为企业销售的产品维度,第三维度为企业管理的产品线维度。组合关系是指第三维度包括第二维度中的部分或全部维度成员,例如,第二维度为企业销售的产品维度,第三维度为企业销售的产品组合维度。
用户想要对第一维度进行的更新可以包括以下一种或多种:在已有维度的基础上,增加第一维度;删除已有的第一维度;修改第一维度,例如,为第一维度增加新的维度成员、删除第一维度中已有的维度成员、修改第一维度中的维度成员的属性值、修改第一维度的编码或名称等第一维度所包括的内容。
用户想要对第一关系进行的更新可以包括以下一种或多种:为第二维度和第三维度添加第一关系;删除第二维度和第三维度之间的第一关系;修改第一关系,例如,修改第一关系的类型、方向等第一关系所包括的内容。其中,第一关系的类型即为上述收敛关系或组合关系。当第二维度收敛于第三维度时,第一关系的方向为第三维度指向第二维度。当第三维度包括第二维度中部分或全部维度成员时,第一关系的方向为第二维度指向第三维度。
步骤102、处理器213根据维度更新请求从第一维度图谱中获取第一子图谱。
具体地,存储器214预先存储有第一维度图谱,第一维度图谱可以由处理器213生成,其生成过程可参见后文的图4及其相关描述。第一维度图谱包括多个节点和多条边,第一维度图谱中的节点用于指示实体,第一维度图谱中的边用于指示实体间的关系,其中,实体可以是维度或维度成员的属性,可选地,实体还可以是业务、业务使用的维表或维表字段。因此,处理器213可以从存储器214获取第一维度图谱,然后根据维度更新请求指示的待更新对象,从第一维度图谱中获取第一子图谱。
更具体地,维度更新请求包括待更新对象的标识。第一维度图谱中的节点也具有标识。例如,当节点指示维度时,节点的标识可以是维度的编码、名称等能够指示该维度的相关信息。又例如,当节点指示维度成员的属性时,节点的标识可以是维度成员的属性的编码、名称等能够指示该属性的相关信息。因此,处理器213根据维度更新请求指示的待更新对象(即待更新对象的标识),从第一维度图谱中获取第一子图谱,包括以下至少两种实现方式:
方式①、当待更新对象为第一维度时,维度更新请求包括第一维度的标识。处理器213可以将第一维度的标识和第一维度图谱中的节点的标识进行比对。如果第一维度的标识与第一维度图谱中的第一节点的标识匹配,则处理器213还根据第一维度图谱确定第一节点关联的节点,以及第一节点与其关联节点之间的边,得到第一子图谱,其中,第一子图谱包括第一节点、第一节点关联的节点、以及第一节点与其关联节点之间的边。如果第一维度的标识与第一维度图谱中的所有节点的标识均不匹配,则第一子图谱为第一维度图谱。
方式②、当待更新对象为第一关系时,维度更新请求包括第二维度的标识和第三维度的标识。处理器213可以将第二维度的标识和第三维度的标识与第一维度图谱中的节点的标识进行比对,确定第一维度图谱中的第二节点和第三节点,其中,第二节点的标识与第二维度的标识匹配,第三节点的标识与第三维度的标识匹配,也即是说,第二节点用于指示第二维度,第三节点用于指示第三维度。之后,处理器213根据第一维度图谱确定第二节点与第三节点之间是否通过边连接,其中,如果第二节点和第三节点之间通过边连接,则第一子图谱包括第二节点、第三节点以及第二节点和第三节点之间的边;如果第二节点和第三节点之间未通过边连接,则第一子图谱包括第二节点和第三节点。
上述实现方式中涉及的“标识之间的比对”,处理器213可以通过计算标识的相似度来实现,其中,当标识之间的相似度大于或等于阈值时,确定这两个标识匹配,反之,当标识之间的相似度小于阈值时,确定这两个标识不匹配。
步骤103、处理器213指示通信接口212向客户端100发送第一子图谱。相应地,客户端100接收第一子图谱。
值得注意的是,上述步骤101至步骤103为可选步骤。具体实现中,维度管理系统200可以根据用户的实际需求选择执行上述步骤101至步骤103,或选择不执行上述步骤101至步骤103。
作为一种可能的实现方式,通信接口212可以提供用户界面,该界面显示有“提示”选项。当用户选择“提示”选项时,说明用户想要查看第一子图谱,那么维度管理系统200将会执行上述步骤101至步骤103。反之,当用户未选择“提示”选项时,说明用户无需查看第一子图谱,那么维度管理系统200将不执行上述步骤101至步骤103。
可以理解的是,当维度管理系统200执行上述步骤101至步骤103时,客户端100便可以向用户展示第一子图谱。由于第一子图谱除了可以表示待更新对象,还可以表示待更新对象的关联对象,其中,当待更新对象为第一维度时,待更新对象关联的对象包括第一节点关联的节点所指示的实体、以及第一节点与其关联节点之间的边所指示的实体间关系;当待更新对象为第一关系时,待更新对象关联的对象包括第二维度和第三维度。因此,用户基于第一子图谱可以直观了解到:待更新对象被更新后,哪些对象可能会受到影响。用户在对想要更新的对象更新完毕后,可以基于第一子图谱对更新对象所影响的对象进行相应的更新,提高维度数据更新的准确性。
步骤104、客户端100向维度管理系统200发送维度更新信息。相应地,维度管理系统200中的通信接口212接收维度更新信息。
维度更新信息用于指示更新对象及更新对象的更新内容。更新对象可以为第一维度或第一关系。当更新对象为第一维度时,更新对象的更新内容用于指示用户对第一维度作出的更新,具体可以包括新增的第一维度、被删除的第一维度、对第一维度作出的修改等。当更新对象为第一关系时,更新对象的更新内容用于指示用户对第一关系作出的更新,具体可以包括新增的第一关系、被删除的第一关系、对第一关系作出的修改等。
在一种实现方式中,存储器214预先存储有多个维度模型,维度模型是用于描述维度或维度间的关系的数据模型,维度模型可以由处理器213根据用户的配置所创建,其具体描述可以参见后文的图4及其相关描述。通信接口212可以提供维度模型更新接口,其中,维度模型更新接口可以通过应用程序编程接口(applicationprogramming interface,API)或图形用户界面(graphicaluserinterface,GUI)实现。维度模型更新接口向用户呈现维度模型中可修改的内容,使得用户可以通过客户端100在维度模型更新接口上操作,以实现第一维度或第一关系的更新。
在另一种实现方式中,通信接口212可以提供维度数据补录接口,其中,维度数据补录接口可以通过API或GUI实现。维度数据补录接口可以支持用户新增第一维度。维度数据补录接口还可以向用户呈现可修改的维度,使得用户可通过客户端100在维度数据补录接口上操作以修改第一维度所包括的维度成员或删除第一维度。
步骤105、处理器213根据维度更新信息对第一维度图谱执行更新操作,得到第二维度图谱。
具体地,处理器213可以根据维度更新信息指示的更新对象及更新对象的更新内容,对第一维度图谱执行更新操作,得到第二维度图谱,第二维度图谱与第一维度图谱变化的内容为更新对象的更新内容。
当更新对象为第一维度时,维度更新信息包括第一维度的标识以及第一维度的更新内容。处理器213可以将第一维度的标识与第一维度图谱中的节点的标识进行比对。如果第一维度的标识与第一维度图谱中的第一节点的标识匹配,则处理器213会根据第一维度的更新内容更新第一节点及其关联的其他节点和边。如果第一维度的标识与第一维度图谱中的所有节点的标识均不匹配,则处理器213会根据第一维度的更新内容在第一维度图谱中新增节点和节点之间的边,其中,新增的节点包括用于指示第一维度的节点,以及用于指示第一维度的维度成员的属性的节点,新增的边包括上述新增节点之间的边。
当更新对象为第一关系时,维度更新信息可以包括第二维度的标识、第三维度的标识以及第一关系的更新内容。处理器213可以将第二维度的标识和第三维度的标识与第一维度图谱中的节点的标识进行比对,确定第一维度图谱中的第二节点和第三节点,再根据第一维度图谱确定第二节点和第三节点之间是否通过边连接。如果第二节点和第三节点之间通过边连接,则处理器213会根据第一关系的更新内容更新第一关系。如果第二节点和第三节点之间未通过边连接,则处理器213会根据第一关系的更新内容在第一维度图谱中新增边,其中,新增的边用于指示第一关系。
步骤106、处理器213根据第二维度图谱判断维度更新信息是否存在异常问题。如果存在,则执行步骤107;如果不存在,则执行步骤108。
具体地,处理器213可根据第二维度图谱所表示的实体及实体间的关系(即第二维度图谱中的节点及节点间的关系),判断维度更新信息是否存在异常问题。
作为一种可能的实现方式,存储器214预先存储有图谱校验规则,图谱校验规则可以由用户或处理器213基于图谱的完整性校验和冲突性校验所设置,因此图谱校验规则用于校验图谱的完整性以及图谱中是否存在冲突的节点和边,其中,当图谱未满足图谱校验规则时,说明图谱中包含的维度信息出现异常,反之,当图谱满足图谱校验规则时,说明图谱中包含的维度信息正确。因此,处理器213可以通过判断第二维度图谱是否满足图谱校验规则,来确定维度更新信息是否存在异常问题。其中,当第二维度图谱满足图谱校验规则时,说明第二维度图谱中包含的维度信息是正确的,即维度更新信息不存在异常问题;当第二维度图谱不满足图谱校验规则时,说明第二维度图谱中包含的维度信息出现异常,即维度更新信息存在异常问题。这里的异常问题是指影响维度使用的问题,比如说,维度1包括维度成员1和维度成员2,维度2包括维度成员1、维度成员2以及维度成员3,当用户删除维度1中的维度成员1,但不删除维度2中的维度成员1时,会影响维度2的正常使用。
示例性地,图谱校验规则可以包括以下一个或多个:图谱中不存在环路;图谱中不存在孤立的节点或边。应理解,上述举例仅是示例性地介绍了两个可能的图谱校验规则,在实际应用中,还可以有其他的图谱校验规则,对此本申请不作限定。
步骤107、处理器213指示通信接口212向客户端100发送提示信息。相应地,客户端100接收上述提示信息。
提示信息用于提示用户修改维度更新信息。提示信息具体可以包括维度更新信息存在的异常问题。可以理解的是,当客户端100向用户呈现提示信息时,用户可以了解维度更新信息存在的异常问题,并基于上述异常问题重新修改维度更新信息,提高更新效率。
步骤108、处理器213指示存储器214存储维度更新信息以及第二维度图谱。
可选地,处理器213还可以将人工更新的第二维度图谱与上述步骤105生成的第二维度图谱进行比对,当二者存在差异时,根据二者之间的差异生成差异报告,其中,差异报告包含人工更新的第二维度图谱与处理器213自动生成的第二维度图谱之间的差异内容。处理器213还指示通信接口212向维度图谱的管理员提供差异报告,使得该管理员可以及时修正维度管理系统200所提供的图谱生成功能。
可选地,处理器213还可以指示通信接口212向用户呈现第二维度图谱,使得用户可以基于第二维度图谱选择感兴趣的维度。处理器213还可以根据用户选择的维度和第二维度图谱,生成维度消费模型,并将维度消费模型展示给用户,其中,维度消费模型包括用户可以使用的维度的相关信息,例如,维度的编码、维度中的维度成员等。
可选地,处理器213还可以根据历史维度更新信息以及维度的历史使用信息生成推荐信息,并向用户提供该推荐信息,其中,该推荐信息用于指示用户频繁更新的维度或用户频繁使用的维度,以辅助用户选择待查看的维度。
下面结合图4示出的维度图谱构建方法的流程示意图,对上述步骤102中涉及的第一维度图谱的构建过程进行详细地介绍。
步骤201、处理器213根据用户的第一配置创建多个维度模型。
具体地,通信接口212提供访问接口,该访问接口可通过API或GUI实现。用户可通过上述访问接口向处理器213提供第一配置,相应地,处理器213获得用户的第一配置,并根据用户的第一配置创建多个维度模型。其中,用户提供的第一配置可以包括维度模型中的具体内容。
维度模型是用于描述维度或维度间的关系的数据模型(具体可以是元模型)。维度管理系统200创建的多个维度模型可以分为两类,一类是维度树模型,另一类是树间关系模型。下面分别对这两类模型进行介绍:
(1)维度树模型
维度树模型是用于描述维度的数据模型。一个维度树模型对应一个维度,一个维度表示业务数据的一个属性或指标,也是对业务数据进行分析的一个业务分析视角。一个维度可以包括多个维度成员,多个维度成员之间存在层级关系,其中,不同层级用于表示描述业务数据的不同粒度。比如说,时间维度可以包括年份和月份两个层级,年份层级用于表示以年为粒度来描述业务数据(如产品在某年的销售额),月份层级用于表示以月为粒度来描述业务数据(如产品在某月的销售额)。因此,一个维度的多个维度成员之间可以形成树状结构,即维度树。也就是说,维度树模型还可以理解为用于描述维度树的数据模型。
维度树模型可以包括以下内容:维度的名称、维度的编码、维度的类型、负责维护该维度的用户(以下称为“维度管理员”)、维度的展示方式、维度中维度成员的属性的编码及名称、维度成员的属性值等与该维度相关的信息。其中,维度的名称可以由维度管理员设置,具体可以是维度的名称。维度的编码可以由维度管理系统200设置,具体可以是维度的编码。维度的类型可以包括基础类型和组合类型两类,其中,基础类型的维度是指维度管理员设置的基本维度,组合类型是指维度管理员基于不同的业务分析视角,对基础类型的维度中的维度成员进行组合所构成的维度。维度的展示方式可以包括通过文本框、下拉框等形式进行展示。
(2)树间关系模型
树间关系模型是用于描述维度间的关系的数据模型。由于维度可以使用维度树表示,因此树间关系模型还可以理解为用于描述维度树之间的关系的数据模型。
一个树间关系模型用于描述一个维度(以下称为“主维度”)与其关联的维度(以下称为“关联维度”)之间的关系。主维度与关联维度之间的关系可以为收敛关系或组合关系。其中,收敛关系一般指的是多个维度收敛于一个维度,比如说,企业销售的多个产品分别对应于多个产品维度,企业管理的产品线对应于产品线维度,上述多个产品维度与产品线维度之间存在收敛关系,且多个产品维度收敛于产品线维度。组合关系指的是一个维度包括另一个维度的部分或全部维度成员,比如说,企业销售的产品组合对应于产品组合维度,上述产品组合维度与产品维度之间存在组合关系,且产品组合维度包括产品维度中的部分维度成员。应理解,除上述收敛关系和组合关系之外,主维度与关联维度之间还可能存在其他的关系,具体取决于实际的业务需求或企业需求。
树间关系模型可以包括以下内容:主维度的信息、关联维度的信息、主维度与关联维度之间的关系的类型和方向、负责维度上述关系的用户(以下称为“关系管理员”)等与维度间关系相关的信息。其中,主维度的信息可以包括主维度的名称、编码等信息。关联维度的信息可以包括关联维度的名称、编码等信息。主维度与关联维度之间的关系的类型可以为收敛关系和组合关系两类,其中,收敛关系的方向用于指向收敛到的维度(例如,由产品维度指向产品线维度),组合关系的方向用于指向组合得到的维度(例如,由产品维度指向产品组合维度)。
步骤202、处理器213根据多个维度模型生成第一维度图谱。
具体地,处理器213从多个维度模型中抽取多个实体的实体信息及实体间的关系。其中,当实体为维度时,实体信息可以是维度的名称或编码等能够指示该维度的相关信息;当实体为维度成员的属性时,实体信息可以是维度成员的属性名称或属性编码等能够指示该属性的相关信息。实体间的关系可以是维度间的关系,或维度与维度成员的属性之间的关系,维度与维度成员的属性之间的关系为组成关系,即维度包括维度成员的属性。之后,处理器213根据多个实体及实体间的关系生成第一维度图谱,第一维度图谱是由多个节点和多条边构成的多层拓扑结构,其中,第一维度图谱中的节点用于指示实体,第一维度图谱中的边用于指示位于边两端的节点所指示的实体间的关系。
作为一种可能的实现方式,维度模型是以JSON文件的格式进行存储的,JSON文件包括“键”和“值”,其中,“键”用于访问JSON文件所包含的具体内容,例如,在维度树模型的JSON文件中,当“键”为“编码”时,“值”为维度的编码。因此,处理器213可以通过关键字检测的方式从维度树模型中抽取维度的名称(或编码)、维度成员的属性名称(或属性编码)以及维度与维度成员的属性之间的关系,从树间关系模型中抽取维度间的关系。这里使用的关键字可以为JSON文件中的“键”,比如说“维度名称”、“维度编码”、“维度成员的属性名称”、“维度成员的属性编码”、“维度间的关系”等。
可选地,处理器213还可以执行以下步骤:处理器213根据用户的第二配置更新第一维度图谱。其中,用户的第二配置可以包括业务标识、业务所使用的维表的标识、与维表具有映射关系的维度的标识、维表字段的标识、与维表字段具有映射关系的维度成员的属性的标识、以及维表与维表字段之间的组成关系。具体实现中,处理器213可以将第二配置中的维度的标识以及维度成员的属性的标识与第一维度图谱中的节点的标识进行比对,根据比对结果更新第一维度图谱。新的第一维度图谱新增了多个节点和多条边,其中,新增节点指示的实体可以为业务、业务使用的维表或维表字段,新增的边用于指示业务与维表之间的授权关系、维表与维度之间的映射关系、维表与维表字段之间的组成关系、维表字段与维度成员的属性之间的映射关系。
为了便于理解,下面以一个具体的示例进行说明:
如图5A所示,图5A示出了一种可能的维度图谱。该维度图谱包括5个节点,这5个节点分别用于指示维度A、维度B、维度C、维度成员属性1、维度成员属性2。其中,维度A包括维度成员属性1和维度成员属性2,维度A与维度B之间存在组合关系,且维度B包括维度成员属性1,维度A与维度C之间存在收敛关系,且维度A收敛于维度C。
假设业务D使用到维表1,维表1与维度A之间存在映射关系,维表1包括维表字段1和维表字段2,维表字段1与维度成员属性1之间存在映射关系,维表字段2与维度成员属性2之间存在映射关系。那么,处理器213更新第一维度图谱的过程为:处理器213新增4个节点,这4个节点分别用于指示业务D、维表1、维表字段1和维度字段2。之后,处理器213建立业务D对应的节点与维表1对应的节点之间的边,维表1对应的节点与维度树A对应的节点之间的边,维表1对应的节点与维表字段1对应的节点之间的关系,维表1对应的节点与维表字段2对应的节点之间的边,维表字段1对应的节点与维度成员属性1对应的节点之间的边,维表字段2对应的节点与维度成员属性2对应的节点之间的边,从而得到如图5B所示的新的维度图谱。
可选地,处理器213还可以指示通信接口212向用户呈现新的第一维度图谱,使得用户可以基于新的第一维度图谱,在维度管理系统200上添加新的业务、新的维表、新的维表字段,或者删除已有的维表或维表字段,又或者修改业务与维表之间的授权关系等。处理器213还可以基于新的第一维度图谱检查用户的更新是否存在异常问题,并在用户的更新存在异常问题时,提示用户再次更新,其具体实现过程与上述步骤105至步骤108所描述的实现过程类似,因此这里不展开描述。
可以看出,本申请所提供的维度数据的管理方法中,维度管理系统200可以基于元模型驱动配置多个维度模型,并基于多个维度模型生成维度图谱。当用户对维度或维度间的关系进行更新时,维度管理系统200可以基于上述维度图谱确定用户的更新是否存在异常问题,因此本申请提供的上述方法可以自动化地对用户的更新内容进行检查,提高维度数据的管理效率。另一方面,维度管理系统200可以向用户呈现维度图谱,以辅助用户对维度数据进行维护,降低了维度数据的维护门槛。
上文结合图3至图5B详细介绍了本申请所提供的维度数据管理方法及维度图谱构建方法,下面结合图6对执行上述方法的维度管理系统200的结构进行描述。
参见图6,图6示出了一种维度管理系统200的结构示意图。应理解,图6仅是示例性地展示了维度管理系统200的结构的一种划分方式,在实际应用中,维度管理系统200的结构还可以有其他的划分方式,对此本实施例不作限定。如图6所示,维度管理系统200包括获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306。
获取模块301用于执行上述步骤101中接收维度更新请求的步骤;上述步骤104;上述步骤201中接收用户的第一配置;接收用户的第二配置。图谱管理模块302用于执行上述步骤102;上述步骤105;上述步骤108;上述步骤202;根据用户的第二配置更新第一维度图谱;将人工更新的第二维度图谱和上述步骤105生成的第二维度图谱进行对比,并在二者存在差异时,根据二者之间的差异生成差异报告;根据用户选择的维度和第二维度图谱生成维度消费模型;根据历史维度更新信息以及维度的历史使用信息生成推荐信息;根据新的第一维度图谱检查用户的更新是否存在异常问题,并在用户的更新存在异常问题时,提示用户再次更新。数据校验模块303用于执行上述步骤106。发送模块304用于执行上述步骤103;上述步骤107;向维度图谱的管理员提供差异报告;向用户展示维度消费模型;向用户展示推荐信息;向用户展示新的第一维度图谱。存储模块305用于执行上述步骤108中存储维度更新信息及第二维度图谱的步骤;存储多个维度模型;存储第一维度图谱;存储新的第一维度图谱;存储维度消费模型。配置模块306用于执行上述步骤201。
获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306均可以通过软件实现,或者可以通过硬件实现。示例性地,以获取模块301为例,介绍获取模块301的实现方式。类似的,图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306的实现方式可以参考获取模块301的实现方式。
作为软件功能单元的一种举例,获取模块301可以包括运行在计算实例上的代码。其中,计算实例可以包括物理主机(计算设备)、虚拟机、容器中的至少一种。进一步地,上述计算实例可以是一台或者多台。例如,获取模块301可以包括运行在多个主机/虚拟机/容器上的代码。需要说明的是,用于运行该代码的多个主机/虚拟机/容器可以分布在相同的区域(region)中,也可以分布在不同的region中。进一步地,用于运行该代码的多个主机/虚拟机/容器可以分布在相同的可用区(availability zone,AZ)中,也可以分布在不同的AZ中,每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中,通常一个region可以包括多个AZ。
同样,用于运行该代码的多个主机/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud,VPC)中,也可以分布在多个VPC中。其中,通常一个VPC设置在一个region内,同一region内两个VPC之间,以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关,经通信网关实现VPC之间的互连。
作为硬件功能单元的一种举例,获取模块301可以包括至少一个计算设备,如服务器等。或者,获取模块301也可以是利用CPU、ASIC、或PLD实现的设备。其中,上述PLD可以是CPLD、FPGA、GAL、DPU、NPU、SoC、卸载卡、加速卡或其任意组合实现。
获取模块301包括的多个计算设备可以分布在相同的region中,也可以分布在不同的region中。获取模块301包括的多个计算设备可以分布在相同的AZ中,也可以分布在不同的AZ中。同样,获取模块301包括的多个计算设备可以分布在同一个VPC中,也可以分布在多个VPC中。其中,获取模块301包括的多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA、GAL、DPU、NPU、SoC、卸载卡、加速卡等计算设备的任意组合。
需要说明的是,在其他实施例中,获取模块301可以用于执行上述维度管理方法及维度图谱构建方法中的任意步骤,图谱管理模块302可以用于执行上述维度管理方法及维度图谱构建方法中的任意步骤,数据校验模块303可以用于执行上述维度管理方法及维度图谱构建方法中的任意步骤,发送模块304可以用于执行上述维度管理方法及维度图谱构建方法中的任意步骤,存储模块305可以用于执行上述维度管理方法及维度图谱构建方法中的任意步骤,配置模块306可以用于执行上述维度管理方法及维度图谱构建方法中的任意步骤。获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306负责实现的步骤可根据实际需要指定,通过获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306分别实现上述维度管理方法及维度图谱构建方法中不同的步骤来实现维度管理系统200的全部功能。
本申请还提供了一种计算设备。该计算设备可以是服务器,例如是中心服务器、边缘服务器、或者是本地数据中心中的本地服务器。该计算设备还可以是台式机、笔记本电脑或者智能手机等终端设备。
图7示例性地展示了本申请所提供的计算设备的结构示意图。如图7所示,计算设备400包括总线401、处理器402、存储器403和通信接口404,且处理器402、存储器403和通信接口404之间通过总线401通信。应理解,本申请不限定计算设备400中的处理器402和存储器403的个数,为了简便,图7以一个处理器402和一个存储器403为例进行描述。
总线401可以是PCIe总线或EISA总线、UB、CXL、CCIX等。总线可以分为地址总线、数据总线、控制总线等。为了便于表示,图7中仅用一条线表示,但这并不表示计算设备400仅有一根总线或一种类型的总线。总线401可以包括在计算设备400各个部件(例如,处理器402、存储器403和通信接口404)之间传送信息的通路。
处理器402可以包括CPU、GPU、MP或者DSP、ASIC、FPGA、CPLD、NPU、SoC、卸载卡、加速卡等具有计算能力的计算单元中的任意一种或多种。
存储器403可以包括易失性存储器,例如,RAM。存储器403还可以包括非易失性存储器,例如,ROM、快闪存储器、HDD或SSD。此外,存储器403还可通过SCM、PCM、或其他类型存储介质实现。
值得说明的是,同一计算设备中可配置相同类型的存储介质实现存储器403功能,也可以配置两种或两种以上类型的存储介质以实现存储器403的功能,本申请对此不作限定。
存储器403中存储有可执行的程序代码。处理器402执行该可执行的程序代码,以执行图3和图4中处理器213所执行的操作步骤,实现上述获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306的功能,从而实现上述维度管理方法及维度图谱构建方法。也即是说,存储器403上存储有用于执行上述维度管理方法及维度图谱构建方法的指令。
通信接口404使用例如但不限于网络接口卡、收发器一类的收发模块,来实现计算设备400与其他设备或通信网络的通信,例如,计算设备400通过通信接口404与客户端100进行通信。
应理解,根据本申请所提供的计算设备400可对应于本申请所提供的图1所示的维度管理系统200,并可以对应于执行根据本申请提供的图3和图4所示的方法中的维度管理系统200,并且计算设备400中的各个模块的上述和其它操作和/或功能分别为了实现图3和图4中的各个方法的相应流程,为了简洁,在此不再赘述。
本申请还提供了一种计算设备集群。该计算设备集群包括至少一台计算设备。该计算设备可以是服务器,例如是中心服务器、边缘服务器,或者是本地数据中心中的本地服务器。该计算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。
图8示例性地展示了本申请所提供的计算设备集群的结构示意图。如图8所示,计算设备集群500包括至少一个计算设备400。计算设备集群500中的一个或多个计算设备400的存储器403中可以存储有相同的用于执行上述维度管理方法及维度图谱构建方法的指令。
在一种实现方式中,计算设备集群500中的一个或多个计算设备400的存储器403中也可以分别存储有用于执行上述维度管理方法及维度图谱构建方法的部分指令,即一个或多个计算设备400的组合可以共同执行上述维度管理方法及维度图谱构建方法。
需要说明的是,计算设备集群500中的不同的计算设备400中的存储器403也可以存储不同的指令,分别用于执行维度管理系统200的部分功能。也即,不同的计算设备400中的存储器403存储的指令可以实现上述获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306中的一个或多个模块的功能。
在一种实现方式中,计算设备集群500中的多个计算设备400可以通过网络连接,该网络可以是广域网或局域网等。图9示出了一种可能的实现方式。如图9所示,计算设备400A和计算设备400B之间通过网络进行连接。具体地,通过计算设备400A和计算设备400B中的通信接口与网络进行连接。在该实现方式中,计算设备400A中的存储器403以及计算设备400B中的存储器403中均存储执行获取模块301、图谱管理模块302、数据校验模块303、发送模块304、存储模块305以及配置模块306的功能的指令。
图9所示的计算设备集群500之间的连接方式可以是考虑到高并发计算的场景,例如,计算设备400A在同一时间段接收到大量的维度更新信息,为了尽快地响应这些维度更新信息,计算设备400A可以将一部分维度更新任务交由计算设备400B执行。应理解,图9中的计算设备400A的功能也可以由多个计算设备400完成,同样,计算设备400B的功能也可以由多个计算设备400完成。
应理解,根据本申请所提供的计算设备集群500可对应于本申请所提供的图1所示的维度管理系统200,并可以对应于执行根据本申请提供的图3和图4所示的方法中的维度管理系统200,并且计算设备集群500中的各个模块的上述和其它操作和/或功能分别为了实现图3和图4中的各个方法的相应流程,为了简洁,在此不再赘述。
本申请还提供了一种包含指令的计算机程序产品。该计算机程序产品可以是包含指令的、能够运行在计算设备上或被存储在任何可用介质中的软件或程序产品。当该计算机程序产品在计算设备上运行时,使得计算设备执行前文所描述的任务执行方法。
本申请还提供了一种计算机可读存储介质。该计算机可读存储介质可以是计算设备能够存储的任何可用介质或者是包含一个或多个可用介质的数据中心等数据存储设备。上述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如,固态硬盘)等。该计算机可读存储介质包括指令,所述指令指示计算设备执行前文所描述的任务执行方法。
最后应说明的是:以上实施例仅用以说明本申请的技术方案,而非对其限制。尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的保护范围。
Claims (13)
- 一种维度数据的管理方法,其特征在于,应用于维度管理系统,所述方法包括:获取维度更新信息,所述维度更新信息包括更新内容,所述更新内容包括维度或维度间关系的更新;根据所述维度更新信息对第一维度图谱执行更新操作,得到第二维度图谱,所述第二维度图谱与所述第一维度图谱变化的内容为所述更新内容,所述第一维度图谱用于表示多个实体及所述多个实体间的关系,所述第一维度图谱所表示的实体包括维度或维度成员的属性;根据所述第二维度图谱所表示的实体及实体间的关系,确定所述维度更新信息存在的异常问题,所述异常问题包括影响维度使用的问题。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:向用户发送提示信息,所述提示信息包括所述异常问题。
- 根据权利要求1所述的方法,其特征在于,在所述获取维度更新信息之前,所述方法还包括:获取维度更新请求,其中,所述维度更新请求包括待更新对象的标识,所述待更新对象为维度或维度间的关系;根据所述待更新对象的标识从所述第一维度图谱中获取第一子图谱,所述第一子图谱用于表示所述待更新对象以及所述待更新对象的关联对象;向用户展示所述第一子图谱。
- 根据权利要求1至3任一项所述的方法,其特征在于,所述方法还包括:获取多个维度模型,其中,维度模型用于描述维度或维度间的关系;从所述多个维度模型中抽取所述多个实体的实体信息及所述多个实体间的关系;根据所述多个实体的实体信息及所述多个实体间的关系,生成所述第一维度图谱,所述第一维度图谱是由多个节点和多条边构成的多层拓扑结构,所述第一维度图谱中的节点用于指示实体,所述第一维度图谱中的边用于指示位于边两端的节点所指示的实体间的关系。
- 根据权利要求4所述的方法,其特征在于,所述第一维度图谱所表示的实体还包括业务、业务使用的维表以及维表字段中的至少一个。
- 一种维度管理系统,其特征在于,所述系统包括:获取模块,用于获取维度更新信息,所述维度更新信息包括更新内容,所述更新内容包括维度或维度间关系的更新;图谱管理模块,用于根据所述维度更新信息对第一维度图谱执行更新操作,得到第二维度图谱,所述第二维度图谱与所述第一维度图谱变化的内容为所述更新内容,所述第一维度图谱用于表示多个实体及所述多个实体间的关系,所述第一维度图谱所表示的实体包括维度或维度成员的属性;数据校验模块,用于根据所述第二维度图谱所表示的实体及实体间的关系,确定所述维度更新信息存在的异常问题,所述异常问题包括影响维度使用的问题。
- 根据权利要求6所述的系统,其特征在于,所述系统还包括:发送模块,用于向用户发送提示信息,所述提示信息包括所述异常问题。
- 根据权利要求6所述的系统,其特征在于,所述获取模块,还用于获取维度更新请求,其中,所述维度更新请求包括待更新对象的标识,所述待更新对象为维度或维度间的关系;所述图谱管理模块,还用于根据所述待更新对象的标识从所述第一维度图谱中获取第一子图谱,所述第一子图谱用于表示所述待更新对象以及所述待更新对象的关联对象;所述发送模块,还用于向用户展示所述第一子图谱。
- 根据权利要求6至8任一项所述的系统,其特征在于,所述获取模块,还用于获取多个维度模型,其中,维度模型用于描述维度或维度间的关系;所述图谱管理模块,还用于从所述多个维度模型中抽取所述多个实体的实体信息及所述多个实体间的关系;根据所述多个实体的实体信息及所述多个实体间的关系,生成所述第一维度图谱,所述第一维度图谱是由多个节点和多条边构成的多层拓扑结构,所述第一维度图谱中的节点用于指示实体,所述第一维度图谱中的边用于指示位于边两端的节点所指示的实体间的关系。
- 根据权利要求9所述的系统,其特征在于,所述第一维度图谱所表示的实体还包括业务、业务使用的维表以及维表字段中的至少一个。
- 一种计算设备,其特征在于,所述计算设备包括处理器和存储器,所述存储器用于存储指令,所述处理器用于执行所述指令,以使得所述计算设备实现如权利要求1至5任一权利要求所述的方法。
- 一种计算机可读存储介质,其特征在于,包括计算机程序指令,当所述计算机程序指令由计算设备执行时,所述计算设备执行如权利要求1至5任一所述的方法的操作步骤。
- 一种包含指令的计算机程序产品,其特征在于,当所述指令被计算设备运行时,使得所述计算设备执行如权利要求的1至5任一所述的方法的操作步骤。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410764025.2A CN121144298A (zh) | 2024-06-13 | 2024-06-13 | 维度数据的管理方法及维度管理系统 |
| CN202410764025.2 | 2024-06-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2025256476A1 true WO2025256476A1 (zh) | 2025-12-18 |
Family
ID=97992516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2025/099648 Pending WO2025256476A1 (zh) | 2024-06-13 | 2025-06-06 | 维度数据的管理方法及维度管理系统 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN121144298A (zh) |
| WO (1) | WO2025256476A1 (zh) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110580293A (zh) * | 2019-09-03 | 2019-12-17 | 北京明略软件系统有限公司 | 实体关系的存储方法及装置 |
| KR20210040891A (ko) * | 2020-06-22 | 2021-04-14 | 베이징 바이두 넷컴 사이언스 테크놀로지 컴퍼니 리미티드 | 정보 추천 방법과 장치, 전자 장비, 컴퓨터 판독가능 저장 매체 및 컴퓨터 프로그램 |
| CN117112799A (zh) * | 2023-07-31 | 2023-11-24 | 中国电信股份有限公司技术创新中心 | 运维知识图谱的确定方法、装置、设备和存储介质 |
| US20240104401A1 (en) * | 2020-12-03 | 2024-03-28 | Riskthinking.Ai Inc. | Systems and methods with classification standard for computer models to measure and manage radical risk using machine learning and scenario generation |
| CN118069892A (zh) * | 2024-03-13 | 2024-05-24 | 新奥新智科技有限公司 | 思维图谱生成方法、装置、设备及介质 |
-
2024
- 2024-06-13 CN CN202410764025.2A patent/CN121144298A/zh active Pending
-
2025
- 2025-06-06 WO PCT/CN2025/099648 patent/WO2025256476A1/zh active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110580293A (zh) * | 2019-09-03 | 2019-12-17 | 北京明略软件系统有限公司 | 实体关系的存储方法及装置 |
| KR20210040891A (ko) * | 2020-06-22 | 2021-04-14 | 베이징 바이두 넷컴 사이언스 테크놀로지 컴퍼니 리미티드 | 정보 추천 방법과 장치, 전자 장비, 컴퓨터 판독가능 저장 매체 및 컴퓨터 프로그램 |
| US20240104401A1 (en) * | 2020-12-03 | 2024-03-28 | Riskthinking.Ai Inc. | Systems and methods with classification standard for computer models to measure and manage radical risk using machine learning and scenario generation |
| CN117112799A (zh) * | 2023-07-31 | 2023-11-24 | 中国电信股份有限公司技术创新中心 | 运维知识图谱的确定方法、装置、设备和存储介质 |
| CN118069892A (zh) * | 2024-03-13 | 2024-05-24 | 新奥新智科技有限公司 | 思维图谱生成方法、装置、设备及介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN121144298A (zh) | 2025-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110168523B (zh) | 改变监测跨图查询 | |
| US9418101B2 (en) | Query optimization | |
| US20190163824A1 (en) | Batch searches in a data fabric service system | |
| US20210382896A1 (en) | High Efficiency Data Querying | |
| US10013449B1 (en) | Validating and non-validating secondary indexes for a table in a non-relational data store | |
| US10146814B1 (en) | Recommending provisioned throughput capacity for generating a secondary index for an online table | |
| US20230060127A1 (en) | Techniques to generate and store graph models from structured and unstructured data in a cloud-based graph database system | |
| CN112528067B (zh) | 图数据库的存储方法、读取方法、装置及设备 | |
| US10193974B2 (en) | Managing collaboration of shared content using collaborator indexing | |
| US20140012988A1 (en) | Provisioning computer resources on a network | |
| US10348855B2 (en) | Integrating complex data structures in collaboration environments | |
| US11704345B2 (en) | Inferring location attributes from data entries | |
| US20250307453A1 (en) | Data Certification Process for Cloud Database Platform | |
| US20240037146A1 (en) | Efficient Storage and Query of Schemaless Data | |
| CN116561074A (zh) | 资源数据的分组方法、装置、计算机设备和存储介质 | |
| US11899806B1 (en) | Managing data permissions for disparate data sets | |
| WO2025256476A1 (zh) | 维度数据的管理方法及维度管理系统 | |
| US11210352B2 (en) | Automatic check of search configuration changes | |
| CN109933587B (zh) | 基于目录注册的数据处理方法、装置、系统及存储介质 | |
| CN117389960A (zh) | 文件解析方法、装置、设备、存储介质和程序产品 | |
| US12124483B2 (en) | Data segmentation using clustering | |
| CN119494121B (zh) | 一种数据权限的管理方法及相关设备 | |
| CN116540934A (zh) | 一种数据存储方法及装置 | |
| CN121051109A (zh) | 一种数据更新方法、系统及相关设备 | |
| CN121117022A (zh) | 一种sql语句优化方法、sql语句优化系统及相关设备 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 25821188 Country of ref document: EP Kind code of ref document: A1 |