WO2022062337A1 - Method and device for data aggregation, terminal device, and computer-readable storage medium - Google Patents

Abstract

A method and device for data aggregation, a terminal device, and a computer-readable storage medium. The method comprises: acquiring a service data aggregation parameter; the service data aggregation parameter comprising a service data identifier and target data fields; acquiring a directed acyclic graph model corresponding to the service data identifier, determining a starting node and a target node in the directed acyclic graph model on the basis of the service data identifier; selecting from the directed acyclic graph model the shortest path between the starting node and the target node on the basis of a dynamic path planning method; searching on the basis of the shortest path, looking for the target data fields in data fields of the target node; and combining the target data fields to produce aggregated data, the aggregated data being used for displaying on a page. The method allows, when a webpage needs to add a functional module, the functional module to be assembled quickly on the basis of a conversion scheme for service data, obviates the need to modify a large amount of data and program codes, thus increasing research and development efficiency.

Description

Data aggregation method, apparatus, terminal device and computer-readable storage medium

This application claims the priority of the Chinese patent application with the application number 202011041479.5 and the invention titled "Data Aggregation Method, Apparatus, Terminal Equipment and Computer-readable Storage Medium" filed with the China Patent Office on September 28, 2020, the entire content of which is Incorporated herein by reference.

technical field

The present application relates to the technical field of data processing, and in particular, to a data aggregation method, a terminal device and a computer-readable storage medium.

Background technique

For online platforms, web page display is very important. For example, pages such as the homepage of an e-commerce platform will display different content according to different festivals. In order to enrich the display capability and display form of the webpage, the webpage needs to be continuously updated. Among them, in the process of updating the webpage, it is usually necessary to continuously add more functional modules. However, even for the same type of functional modules (for example, modules that display products), the data fields displayed are not exactly the same, and the operation and configuration methods are also various. It can be seen that for the network platform, every time a functional module is added, all related data needs to be reconfigured; even if the original functional module is fine-tuned, a large amount of data and related program codes need to be modified, resulting in R&D. Low efficiency and high development cost.

SUMMARY OF THE INVENTION

In view of this, a data aggregation method, a terminal device and a computer-readable storage medium are provided in the embodiments of the present application, so as to overcome the need to reconfigure all data when adding functional modules in the page display in the prior art, resulting in low R&D efficiency and high development cost. The problem.

In a first aspect, an embodiment of the present application provides a data aggregation method, which includes:

Obtain business data aggregation parameters; the business data aggregation parameters include business data identifiers and target data fields;

Obtain a directed acyclic graph model corresponding to the business data identifier, and determine a start node and a target node in the directed acyclic graph model according to the business data identifier; wherein, the directed acyclic graph The model is constructed based on pre-acquired various business data and the conversion relationship between various business data;

Selecting the shortest path between the starting node and the target node from the directed acyclic graph model according to the dynamic path planning method;

Search according to the shortest path, and find the target data field in the data field of the target node;

The target data fields are combined to obtain aggregated data, and the aggregated data is used for display on the page.

In a second aspect, an embodiment of the present application provides a data aggregation apparatus, and the apparatus includes:

a parameter acquisition module for acquiring business data aggregation parameters; the business data aggregation parameters include a business data identifier and a target data field;

A graph model obtaining module, configured to obtain a directed acyclic graph model corresponding to the business data identifier;

A node determination module, configured to determine a start node and a target node in the directed acyclic graph model according to the service data identifier; wherein the directed acyclic graph model is based on pre-acquired multiple service data and The transformation relationship between various business data is constructed;

a shortest path selection module, configured to select the shortest path between the start node and the target node from the directed acyclic graph model according to a dynamic path planning method;

a target data field search module, configured to search according to the shortest path, and find the target data field in the data field of the target node;

The aggregated data obtaining module is used for combining the target data fields to obtain aggregated data, and the aggregated data is used for displaying on the page.

In a third aspect, embodiments of the present application provide a terminal device, including: a memory; one or more processors coupled to the memory; and one or more application programs, wherein the one or more application programs are stored In the memory and configured to be executed by one or more processors, one or more application programs are configured to execute the data aggregation method provided by the above first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the data aggregation method provided in the first aspect.

A data aggregation method, device, terminal device, and storage medium provided by the embodiments of the present application acquire service data aggregation parameters, wherein the service data aggregation parameters include a service data identifier and a target data field; and obtain corresponding directed data according to the service data identifier. Acyclic graph model, and the starting node and target node are determined in the directed acyclic graph model; the directed acyclic graph model is constructed according to the pre-acquired various business data and the conversion relationship between various business data; Then select the shortest path between the starting node and the target node from the directed acyclic graph model according to the dynamic path planning method; search according to the shortest path, and find the target data field in the data field of the target node; combined to get aggregated data. The directed acyclic graph model of the present application is used to represent the conversion relationship between various business data. As long as the starting node and the target node are known, it is possible to determine from one business data (that is, the business data corresponding to the starting node) to the starting node. All conversion methods of another business data (that is, the business data corresponding to the target node), and then select the best conversion method through the shortest path. According to the best conversion method, you can quickly find or search for another business data through the A target data field existing in business data, the target data fields are combined to obtain aggregated data, the aggregated data can be displayed on the page of the network platform to achieve a certain function, this method can be used when the webpage needs to add function modules. The conversion method of business data quickly combines the function module without modifying a large amount of data and related program codes, thereby improving R&D efficiency and greatly reducing development costs.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without any creative effort.

FIG. 1 is a schematic diagram of an application scenario of a data aggregation method provided by an embodiment of the present application;

2 is a schematic flowchart of a method provided by an embodiment of the present application;

3 is a schematic structural diagram of a directed acyclic graph model provided by an embodiment of the present application;

FIG. 4 is a visual interface diagram of aggregated data display provided by an embodiment of the present application;

5 is a schematic flowchart of a directed acyclic graph model construction process provided by an embodiment of the present application;

FIG. 6 is a visual interface diagram of a newly added node provided by an embodiment of the present application;

FIG. 7 is a visual interface diagram of a newly added edge provided by an embodiment of the present application;

FIG. 8 is a visual interface diagram of node management provided by an embodiment of the present application;

FIG. 9 is a visual interface diagram of edge management provided by an embodiment of the present application;

10 is a directed acyclic graph model visualization interface diagram provided by an embodiment of the application;

11 is a schematic diagram of combining sub-target data fields provided in an embodiment of the application;

12 is a schematic diagram of a process of providing business data aggregation in an embodiment of the application;

13 is a schematic structural diagram of a data aggregation apparatus provided in an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a terminal device provided in an embodiment of the application;

FIG. 15 is a schematic structural diagram of a computer-readable storage medium provided in an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In order to describe the present application in more detail, a data aggregation method, apparatus, terminal device and computer storage medium provided by the present application will be specifically described below with reference to the accompanying drawings.

Please refer to FIG. 1. FIG. 1 shows a schematic diagram of an application scenario of the data aggregation method provided by the embodiment of the present application. The application scenario includes the terminal device 100 provided by the embodiment of the present application. The terminal device 100 may be various types of display screens. Electronic equipment (such as the structural diagrams of 102, 104, 106 and 108), including but not limited to smart phones and computer equipment, wherein the computer equipment can be at least one of a desktop computer, a portable computer, a laptop computer, a tablet computer, etc. kind. The terminal device 100 can install and run related application programs for obtaining business data aggregation parameters. The user can set relevant information about business data aggregation, such as business data and other information, on the visual interface of the terminal device 100. The set information is obtained, and the business data aggregation parameters are obtained according to the obtained information. In addition, the user can set the relevant information of the service data and the conversion relationship between the service data on the visual interface of the terminal device 100, and the terminal device 100 can obtain the relevant information according to the user's setting of the service data and the conversion relationship between the service data The transformation relationship between business data (including business data identifiers, data types, data fields, etc.) and business data, and a directed acyclic graph model is constructed according to the business data and the transformation relationship between business data. In addition, the terminal device 100 may also be provided with at least one kind of database for storing the directed acyclic graph model and the like. The terminal device 100 may generally refer to one of multiple terminal devices, and this embodiment only takes the terminal device 100 as an example for illustration. Those skilled in the art may know that the number of the above-mentioned terminal devices may be more or less. For example, the above-mentioned terminal devices may be only a few, or the above-mentioned terminal devices may be dozens or hundreds, or more, and the embodiments of the present application do not limit the number and type of the terminal devices. The terminal device 100 may be used to execute a data aggregation method provided in this embodiment of the present application.

In an optional implementation manner, the application scenario includes, in addition to the terminal device 100 provided in the embodiment of the present application, a server, where a network is set between the server and the terminal device. A network is a medium used to provide a communication link between an end device and a server. A network may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

It should be understood that the numbers of terminal devices, networks and servers are merely exemplary. There can be any number of terminal devices, networks and servers according to implementation needs. For example, the server may be a server cluster composed of multiple servers. The terminal device interacts with the server through the network to receive or send messages and the like. The server may be a server that provides various services. The server may be used to perform steps other than visual display (eg, aggregated data display) in the data aggregation method provided in the embodiments of the present application. In addition, when the terminal device executes the data aggregation method provided in the embodiment of the present application, some steps may be executed on the terminal device, and some steps may be executed on the server, which is not limited herein.

For online platforms, web page display is very important. For example, pages such as the homepage of an e-commerce platform will display different content according to different festivals. In the development process of a homepage, in order to enrich the content and display style of the homepage, the product usually proposes more modules. But even in modules that need to display products, the data fields to be displayed by different modules are different. At the same time, there are various ways of operating the configuration. For ease of understanding, examples are given below for illustration.

For example, module A requires operators to configure through merchant ID, store ID or product SKU, and needs to display product name, product image, product market price, product discount price, product sales, and product inventory on the module. Then it needs to be implemented through the following data acquisition logic:

① Merchant ID-->list<Store ID>-->list<Product SKU>-->list<Product Details>-->list<View Data A>

②Store ID-->list<Product SKU>-->list<Product Details>-->list<View Data A>

③Product SKU-->Product Details-->View Data A

Module B requires operators to configure through merchant ID, store ID, and product SKU. It is necessary to display product pictures, product discount prices, and product rebate points on the module. Then it needs to be implemented through the following data acquisition logic:

① Merchant ID-->list<Store ID>-->list<Product SKU>-->list<Product Details>-->list<View Data B>

②Store ID-->list<Product SKU>-->list<Product Details>-->list<View Data B>

③Product SKU-->Product Details-->View Data B

Calling each of the above data acquisition and conversion links as a Fetcher, it can be seen that each time a module is added, a lot of Fetchers need to be added, and the logic of Fetchers is mostly similar, resulting in the need to write a lot of repetitive code for R&D, efficiency lower. Moreover, the data acquisition and conversion process in the real scene is far more complicated than the structure of the above case. Usually, not only the data of the product needs to be obtained, but also the information of the live broadcast room, store, distribution and so on. In this process, multiple RPC calls are required. , the redundant code will be more and the development is more difficult, among which RPC, that is, remote call, refers to the call across the process, or the call across the network.

Based on this, an embodiment of the present application provides a data aggregation method. Please refer to FIG. 2. FIG. 2 shows a schematic flowchart of a data aggregation method provided by an embodiment of the present application. The method is applied to the terminal device in FIG. 1 as an example for description, including the following steps:

Step S110, obtaining business data aggregation parameters; the business data aggregation parameters include business data identifiers and target data fields;

Generally speaking, different businesses have data matching the business, that is, different businesses have corresponding data types. For example, e-commerce business includes business data, store data, commodity data, sellerid, sku, etc.; live broadcast business Including live channel data, live room data, live content data, etc.; entertainment business includes program data, hosting data, etc.; entertainment business includes anchord, guildld, etc. In addition, data between different services may overlap, for example, merchant data, store data, commodity data, live broadcast channel data, live broadcast room data, and live broadcast content data can be stored between e-commerce business and live broadcast business. It can be seen that the business data can be various, and specifically can be set or customized by the business user or user according to actual needs.

Among them, the business data aggregation parameters refer to some information related to the business data aggregation process, which may include business data identifiers and target data fields; business data identifiers are information used to identify business data, and the identifiers are mainly used for indexing business data. , it is convenient to determine the directed acyclic graph model and the starting node and the target node, wherein the service data identifier is unique, that is, one service data identifier can only be used to identify one service data. The service data identification can be in various forms, and can be formed by a combination of characters (such as letters and numbers, etc.).

The target data field refers to the required field that needs to be obtained from the target node.

In an optional embodiment, before the step of acquiring service data aggregation parameters, the data aggregation method may further include receiving a service data aggregation request, where the service data aggregation request is mainly used to transmit service data aggregation parameters; the terminal device responds to the data Aggregate requests to obtain business data aggregation parameters. The sending object of the service data aggregation request may be a peripheral system or other devices.

In another optional implementation manner, the terminal device can obtain business data aggregation parameters through a visual interface. For example, the terminal device can obtain business data and required data during business data aggregation according to the newly added task interface. Fields, etc., to determine information such as business data identifiers, target data fields, and so on.

Among them, a data aggregation task can correspond to a new task interface, and the new task interface can provide multiple sub-interfaces based on different setting objects, which together constitute the new task interface, and the multiple sub-interfaces can be based on confirmation operations. of consecutive initiation relationships. That is, after the content of one sub-interface is set, the user can trigger the display of the next sub-interface through manual operation on the sub-interface. For example, the newly added task interface may include multiple sub-interfaces such as a business data setting interface, a required data field setting interface, etc. Optionally, the newly added task interface may also include other sub-interfaces, which are not limited in this embodiment of the present application . On the visual interface, users can fill in the business data and required data fields that the application needs to aggregate according to their needs. In addition, the newly added task interface may include a business data and required data field selection interface. There are many business data and required data field options on this interface. Users can directly select business data and required data fields on the corresponding interface.

Step S120, obtaining the directed acyclic graph model corresponding to the business data identifier, and determining the starting node and the target node in the directed acyclic graph model according to the business data identifier; It is constructed by different types of business data and the conversion relationship between various business data;

A directed acyclic graph is also known as a DAG graph. In graph theory, if a directed graph starts from any vertex and cannot return to the point through several edges, the graph is a DAG. Because a point in a directed graph goes through two routes to another point, it may not form a cycle, so a directed acyclic graph may not be converted into a tree, but any directed tree is a directed acyclic graph.

The directed acyclic graph model is a pre-built DAG graph structure. The DAG graph structure is constructed according to the related business data in a business and the conversion relationship between these business data; the construction process of the DAG graph structure can be expressed as It is a visual DAG. There are many paths in the DAG, each path has several nodes, each node corresponds to a kind of business data, and the edge between each node represents the business of the two nodes corresponding to the edge. Data can be transformed, that is, one kind of business data can be queried or searched for another kind of business data. A kind of business data includes business data name (corresponding to the node name in the node), business data identifier (corresponding to the node identifier in the node), data type (referring to the integer type, floating point type, etc., which are often said in computer languages, such as string , number, object, array, etc.), data fields (that is, what specific data is stored).

Among them, the conversion relationship can be a direct conversion relationship and an indirect conversion relationship; two nodes that are directly connected have a direct conversion relationship; two nodes are connected through other nodes, indicating that the two nodes have an indirect conversion relationship. A direct relationship indicates that the business data corresponding to the two nodes can be directly converted, and an indirect relationship indicates that the business data corresponding to the two nodes can be converted through an intermediate state. For example, if A is connected to B, it means that A and B have a direct connection relationship, and A can be directly converted into B; A is connected to B through C, which means that A and C have an indirect connection relationship, and A can be converted into C (intermediate state) first and then converted into B. In addition, the conversion relationship can be expressed as a conversion method or a conversion algorithm.

In order to facilitate understanding, a detailed example is given; in this example, an e-commerce business is used as an example to illustrate the construction process of a directed acyclic graph model, wherein the business data included in the e-commerce business includes uid, commodity, and horizontally drawn commodity. , selection, channel, double-column live room and Datong column live room; according to the conversion relationship between these business data, a directed acyclic graph model can be constructed. The result is shown in Figure 3, where each node in the figure corresponds to a kind of business data (for the convenience of display, the business data identifier is usually used to represent the node in the directed acyclic graph model). As can be seen from Figure 3, uid has a direct conversion relationship with channel number, commodity id, double-column live room, and Chase bar live room, and uid and Chase bar live room can also be indirectly connected, uid and horizontally drawn products and selections have both Indirect connection relationship. Among them, uid (user identification) user identity certificate, usually after the user registers the network platform, the system will automatically give the user a uid value.

It should be noted that the e-commerce business mentioned in the embodiment is only used to illustrate the structure of the DAG model, and the DAG model is not limited to only be used in e-commerce business, but also for entertainment. business, live broadcast, etc.

Since the DAG model is determined according to the conversion relationship between business data and business data, it can be seen that the DAG model is related to different businesses, that is, different businesses correspond to different DAG models. When data aggregation is required, the corresponding directed acyclic graph model can be determined according to the business data identifier. Since the nodes of the DAG model include business data identifiers, when data aggregation is required, the starting node and the target node are determined according to the business data identifiers of the business data aggregation parameters. The business data identifiers have at least Two, used to determine the starting node and the target node, respectively. Optionally, there may be multiple target nodes, and the number of service data identifiers should also be increased accordingly.

In addition, it should also be noted that the directed acyclic graph model only reflects the transformation relationship between nodes, and is used to represent the transformation relationship between various business data in a certain business. The actual conversion path can be determined only after the start node, end node, etc. are specified through service aggregation parameters in the specific service data aggregation process.

In addition, the DAG model can be stored in the storage device (such as a database) of the terminal device, or in the server or other devices; when the terminal device needs to use the DAG model, it can be stored from the storage device, obtained from a server or other device.

Step S130, selecting the shortest path between the start node and the target node from the directed acyclic graph model according to the dynamic path planning method;

Specifically, for any two nodes, there may be one or more paths between them, or there may be no path. If you want to convert the business data corresponding to one node to the business data corresponding to another node, you need to find a shortest path between the two nodes, and then execute the edge of the shortest path to complete the data conversion. In this embodiment, a dynamic path planning method is used to determine the shortest path. As shown in Figure 3, for example, to realize the conversion of uid to double-column live room, you can directly convert uid to double-column live room, or you can first convert uid to channel number and then convert to double-column live room according to the channel number , we need to determine which of the two conversion paths is the shortest path. If the path is the shortest path by calculating the conversion from uid to channel number and then according to the channel number to double-column live room, select the conversion method corresponding to the path to complete Business data conversion from uid to double-column live broadcast room.

Step S140, searching according to the shortest path, and finding the target data field in the data field of the target node;

After the shortest path is obtained, the conversion from the service data of the starting node to the service data of the target node can be realized according to the shortest path. Then search according to the conversion path to find the target data field in the data field of the target node.

Immediately after the above example, as shown in Figure 3, the path from uid to channel number and then converted to double-column live room according to the channel number is the shortest path, that is, the conversion path of uid-channel number-double-column live room is formed. When searching for the target data field, first find the channel number according to the uid, then find the double-column live room according to the channel number, and then find the target data field in the double-column live room. In a specific implementation process, it is usually necessary to assign a value to the uid to find the target data field, for example, uid=123456.

In step S150, the target data fields are combined to obtain aggregated data, and the aggregated data is used for display on the page.

Specifically, the target data field usually includes more than one data field, so these data fields need to be combined to obtain aggregated data. Among them, the aggregated data is used to display on the page to achieve a certain function. As shown in Figure 4, the store id in the store information, the target data field of the store level, and the live broadcast room number in the live broadcast room information can be used. , live broadcast title and broadcast cover and other target data fields are combined to obtain aggregated data and display it on the visualization page of the terminal device (as shown in the result preview in Figure 4).

In the data aggregation method provided by the embodiment of the present application, business data aggregation parameters are obtained, wherein the business data aggregation parameters include a business data identifier and a target data field; a corresponding directed acyclic graph model is obtained according to the business data identifier, and the In the ring graph model, the starting node and the target node are determined; the directed acyclic graph model is constructed according to the pre-acquired various business data and the conversion relationship between various business data, in which each of the directed acyclic graph model. The node represents a kind of business data, and the edge of the directed acyclic graph model represents the conversion relationship between the two connected business data; the business data includes business data identifiers and data fields; and then according to the dynamic path planning method from the directed acyclic graph The shortest path between the start node and the target node is selected in the model; the search is performed according to the shortest path, and the target data field is found in the data field of the target node; the target data fields are combined to obtain aggregated data. The directed acyclic graph model of the present application is used to represent the conversion relationship between different types of business data. As long as the starting node and the target node are known, it is possible to determine from one type of business data (that is, the business data corresponding to the starting node) to the starting node. All conversion methods of another business data (that is, the business data corresponding to the target node), and then select the best conversion method through the shortest path. According to the best conversion method, you can quickly find or search for another business data through the A target data field existing in business data, the target data fields are combined to obtain aggregated data, the aggregated data can be displayed on the page of the network platform to achieve a certain function, this method can be used when the webpage needs to add function modules. The conversion method of business data quickly combines the function module without modifying a large amount of data and related program codes, thereby improving R&D efficiency and greatly reducing development costs.

Referring to FIG. 5, in one embodiment, the method for constructing a directed acyclic graph model includes:

Step S210, acquiring a variety of business data and conversion relationships between various business data;

In an optional implementation manner, various service data and conversion relationships between various service data may be obtained from other peripheral systems or other devices; for example, service users use other peripheral systems or other devices to interact with terminal devices , which converts various business data and the conversion relationship between various business data. The type and quantity of business data and the conversion relationship between various business data can be preset or defined by business users. For example, business data related to e-commerce business can be selected, such as merchant id, store id, commodity sku, commodity Detailed information (product name, product image, product discount price, product sales, product inventory, etc.), and the conversion relationship between business data, such as merchant id-store id-product sku-product details, store id-product sku- Product details, etc. Among them, "-" indicates that the business data before and after has a conversion relationship.

Sku (stock keeping unit) is the basic unit of stock in and out measurement, which can be in pieces, boxes, trays, etc. sku This is a necessary method for the logistics management of the large chain supermarket DC (distribution center). At present, it has been extended to the abbreviation of the unified product number, and each product has a unique sku number. For e-commerce, sku refers to a product, and each product has a sku, which is convenient for e-commerce brands to identify the product.

In another optional implementation manner, the terminal device may acquire the service data and the conversion relationship between each service data through a visual interface. For example, it is used to log in to the visual interface of the terminal device, and add new nodes (as shown in Figure 6) and new edges (as shown in Figure 7) on the visual interface to obtain business data and the relationship between each business data. conversion relationship.

Referring to Figure 6, a new node can be added by entering the node name, node data type, node identifier, node data type and node data field on the new node interface, where the node data type can include string, number, object, etc. Filter from these types. The node data field includes field type and field content. Optionally, the content of the node data field (that is, which fields are included), the type of the field, and the restrictions of the field can be standardized through schema. In an optional implementation, the node data fields can be set via json schema.

In addition, the node data field can be empty, that is, not every node has a node data field; it can be determined according to the node data type, for example, when the node data type is string and number, the node data field can be empty; and the node data type is object, the node data field cannot be empty.

After entering the node name, node data type, node ID, node data type and node data fields on the visual interface, click the "Save" button to complete the addition of the current new node. Click "Cancel" to cancel the current new node. After a new node is added, the service data can be acquired according to the relevant information of the newly added node (ie, the node name, the node data type, the node identifier, and the node data field). By using the above methods and so on, the addition of all nodes can be completed, and corresponding business data can be obtained. In addition to completing all new nodes, you can also manage all new nodes. As shown in Figure 8, all new nodes can be displayed on one interface, and the relevant information of each node can be displayed. At this time, these new nodes can be edited, such as modifying the node name, node data type, node ID and Node data field; it can also be to delete a new node.

Referring to Figure 7, the new edge can be input by entering the relevant information of a node (usually represented by a node identifier, such as uid) at the starting point on the new connection interface, and inputting the relevant information identifier of another node at the end position (usually using Node identifiers, such as sid), weights and conversion rules, and then click the "Save" button to complete the addition of the current new edge, and click "Cancel" to cancel the current new edge. The newly added edge currently represents the conversion relationship between the node uid and the node sid, and the conversion relationship includes a conversion weight of 5 and a conversion rule.

The conversion weight can be a preset value, and business users can determine a reasonable value based on comprehensive consideration of parameters such as load and RPC time-consuming when different nodes are converted. Transformation weights can be positive or negative. The conversion weight between any two nodes can be a fixed value or a variable value; for example, the time consumed by the conversion between two nodes can be checked regularly, and then the conversion weight can be adjusted according to the length of time consumed. update etc. After a new edge is added, the conversion relationship between business data can be obtained according to the related information of the newly added edge (ie, the starting node, the target node, and the conversion relationship). By using the above methods and so on, the addition of all edges can be completed, and corresponding business data can be obtained. In addition to completing all new edges, you can also manage all new edges. As shown in Figure 9, all newly added edges can be displayed on one interface, and the relevant information of each edge can be displayed. At this time, these newly added edges can be edited, such as modifying the start node, target node, and conversion relationship; You can also delete new edges.

Step S220, taking each kind of business data as a node, and setting edges between corresponding nodes according to the conversion relationship between various business data;

Step S230, setting a weight for each edge to construct a directed acyclic graph model.

Specifically, after obtaining all the business data and the conversion relationship corresponding to the business data, take each business data as a node, set edges between the corresponding nodes based on the conversion relationship between the business data, and set corresponding edges to the edges. weights to build a directed acyclic graph model. The result is shown in Figure 10. The DAG model can be displayed on the visual interface of the terminal device, and business users can view the DAG model by logging into the corresponding APP or web page and entering the management platform (such as dashboard). In addition, after the DAG model is constructed, the DAG model can be modified and updated by adding new nodes, adding edges, node management and edge management, so as to facilitate the business data of a certain business. When changes occur, the directed acyclic graph model is adjusted in time to facilitate later data aggregation.

In some embodiments, the number of target nodes is multiple; the step of searching the target data field in the data field of the target node includes:

Select the shortest path between the starting node and each target node; use the path search method to search for each shortest path, and find the sub-target data field from the data field of each target node; combine the sub-target data fields, Get the target data field.

Specifically, when the number of target nodes is multiple, that is, one start node corresponds to multiple target nodes, and the target data field is among the multiple target nodes. At this time, it is necessary to first select the shortest path between the starting node and each target node, then search for each shortest path, and find the sub-target data field from the data field of each target node, so as to convert the sub-target data field into the sub-target data field. Combine to get the target data field. In addition, since the data field is usually in the form of key-value. In the process of extracting the data field, the value is extracted when the key is known, that is, the value is found by the key. At the same time, since the data field may be a structure with a multi-layer structure and multiple fields, an algorithm is required to traverse the structure to find the value corresponding to the key when extracting the field.

Among them, the path search method is a path parsing method, which parses the path into an abstract syntax tree, and then traverses the abstract syntax tree to obtain the required data fields. In this embodiment, the abstract syntax tree is obtained by using the path parsing method for each shortest path, and then the sub-target data field can be obtained by traversing the abstract syntax tree.

For ease of understanding, a detailed example is given. As shown in Figure 11, for example, take seller_id (where the value of seller_id is 1451672) as the starting node, and take store data, channel data and guild data as target nodes respectively; first find out the seller_id respectively to store data, channel data and The shortest paths between guild data, convert these shortest paths into abstract syntax trees, and then traverse the abstract syntax trees to find "store_name" from the store data: the sub-target data field of "Lucky Emerald"; search from the channel data Go to the sub-target data field of "channel_name: "Emerald Pickup"; find the sub-target data fields of "channel_id": "121594" and "guile_name: "Flying Dragon Guild" from the guile data. The sub-target data fields found are then combined to obtain the target data word.

In an optional embodiment, in the step of searching for each shortest path by using a path search method, and searching for a sub-target data field corresponding to the target data field from the data field of each target node, the steps include:

Use the jsonpath method to parse the data field of each target node to form a tree structure; traverse the tree structure to obtain sub-target data fields.

Specifically, Jsonpath is Stefan Goessner's path parsing method for json. Jsonpath can first parse this json data into an abstract syntax tree, and then perform traversal in the tree to obtain sub-target data fields. Jsonpathde parsing is very powerful, it can use a regular expression-like syntax to get the desired content, and it can support multiple coding languages, such as Javascript, python, PHP, and java.

In one embodiment, the service data identifiers include initial state service data identifiers and final state service data identifiers; the start node is the initial state node, and the target node is the final state node; the nodes of the directed acyclic graph model include the initial state node, Intermediate state nodes and final state nodes; one of the initial state nodes corresponds to one or more intermediate state nodes and one or more final state nodes;

According to the initial state business data identifier, find any node other than the most terminal node in the directed acyclic graph model, and determine it as the initial state node; The node that has a direct or indirect conversion relationship with the initial state node is determined as the final state node; the node that is located between the initial state node and the final state node, and has a direct or indirect conversion relationship with the initial state node or the final state node, is determined. is an intermediate node.

Specifically, the business data aggregation process is divided into two parts: data conversion and combination, as shown in Figure 12. The data conversion refers to converting one kind of business data into another kind of business data. If the initial service data before conversion is recorded as the initial state, and the target service data obtained after conversion is recorded as the final state (such as A, B, and C), then the services involved in the conversion process between the initial service data and the target service data are recorded as the final state. The data is in the intermediate state (eg A1, A2, B1). As shown in Figure 3, for example, the uid needs to be converted into a double-column live room, then the uid is the initial state, the double-column live room is the final state, and the channel number between the uid and the double-column live room is the intermediate state. For a business data, its form is not fixed. As shown in Figure 3, for example, if you want to convert the uid into a double-column live room item, then take the uid as the initial state, the double-column live room item as the final state, and the channel number between the uid and the double-column live room item as the intermediate state ; It can also take the uid as the initial state and the double-column live room as the final state. At this time, there is no intermediate state. In addition, if you want to convert the uid to the channel number, then use the uid as the initial state and the channel number as the final state.

If the process from the initial state to the intermediate state to the final state is called data conversion, then the process of combining the final state to obtain the final data is called data combination. For the directed acyclic graph model, since the nodes represent business data, the connection relationship (or transformation relationship) between nodes and nodes is the data transformation process. However, which nodes correspond to the initial state, which nodes correspond to the intermediate state, and which nodes correspond to the final state can only be determined when specific business data is aggregated, that is, it is usually determined according to business data aggregation parameters. The specific process is: determine the starting node, the target node and all paths from the starting node to the target node from the directed acyclic graph model according to the business data aggregation parameters; where the starting node corresponds to the initial state, the target node corresponds to the final state, and the starting node corresponds to the final state. Other nodes on all paths from the start node to the target node correspond to intermediate states. In addition, since the nodes in the DAG model represent business data, the nodes can also be divided into initial state nodes, intermediate state nodes and final state nodes. Among them, the initial state node, intermediate state node and final state node are not fixed; the number of initial state nodes is usually one, the initial state node corresponds to one or more intermediate state nodes, and the initial state node corresponds to one or more final state nodes state node.

The initial state node is any node other than the most terminal node in the directed acyclic graph model; the initial state node is determined according to the initial state service data identification; the final state node is located downstream of the initial state node and has a direct relationship with the initial state node. or indirect conversion relationship; the final state node is determined according to the final state service data identifier; the intermediate state node is located between the initial state node and the final state node, and has a direct or indirect conversion relationship with the initial state node or the final state node node.

In addition, the initial state node is usually the start node, and the final state node is usually the target node. The initial state node is any node other than the end node in the DAG model, that is, the initial state node cannot be the end node in the DAG model, because the end node cannot perform business data conversion. Except for the most terminal node, the initial state node can usually be any node of the directed acyclic graph model; and the final node can be any one or more nodes downstream of the initial state node; and between the initial state node and the final state node Nodes on all paths between are intermediate nodes.

In the actual data aggregation process, the initial state node and the final state node are determined according to the initial state service data identifier and the final state service data identifier.

In some embodiments, the step of selecting the shortest path between the start node and the target node from the directed acyclic graph model by the dynamic path planning method includes:

Screen out all paths between the initial state node and the final state node from the directed acyclic graph model according to the initial state node, intermediate state node and final state node; calculate the total weight value of each path according to the weight of each edge of the path , and select the path corresponding to the smallest total weight value as the shortest path.

Specifically, selecting the shortest path between the starting node and the target node is usually to calculate the total weight value of all paths between the initial state node and the final state node. The smaller the total weight value is, the shorter the path is. Find the total weight value. The path with the smallest value is the shortest path.

In one embodiment, the steps of calculating the total weight value of each path according to the weights of each edge of the path, and selecting the path corresponding to the smallest total weight value as the shortest path include:

Dijkstra's algorithm is used to select the shortest path by taking the weight of each edge as a positive value.

Specifically, the Dijkstra algorithm is a greedy algorithm, which is a method for quickly determining the shortest path. But in the process of using Dijkstra algorithm, its weight value cannot be negative. In this example, if the Dijkstra algorithm is used to calculate the total weight value of each path, the weight value of each edge can only take a positive value.

In an alternative embodiment, the Floyd algorithm may be used to select the shortest path.

Floyd's algorithm, Floyd's algorithm is an algorithm for solving the shortest path between any two points, which can correctly handle the shortest path problem of directed graph or negative weight.

It should be understood that the algorithm used in determining the shortest path is not limited to the Dijkstra algorithm or the Floyd algorithm mentioned in this embodiment. According to the technical teachings of the present invention, those skilled in the art can also use other algorithms for shortest path planning.

It should be understood that although the steps in the flowcharts of FIG. 2 and FIG. 5 are sequentially displayed according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIG. 2 and FIG. 5 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. These sub-steps or stages are completed. The order of execution of the steps is not necessarily sequential, but may be performed alternately or alternately with other steps or at least part of sub-steps or stages of other steps.

A data aggregation method is described in detail in the above-mentioned embodiments disclosed in this application. The above-mentioned method disclosed in this application can be implemented by various forms of equipment. Therefore, this application also discloses a data aggregation device corresponding to the above-mentioned method, which is given below. Specific embodiments are described in detail.

Please refer to FIG. 13 , which is a data aggregation apparatus disclosed in an embodiment of the present application, which mainly includes:

The parameter acquisition module 130 is used for acquiring service data aggregation parameters; the service data aggregation parameters include service data identifiers and target data fields;

The graph model obtaining module 132 is configured to obtain a directed acyclic graph model corresponding to the business data identifier;

The node determination module 134 is used to determine the starting node and the target node in the directed acyclic graph model according to the service data identification; wherein, the directed acyclic graph model is based on the pre-acquired multiple service data and various service data. The conversion relationship between them is constructed;

The shortest path selection module 136 is used for selecting the shortest path between the starting node and the target node from the directed acyclic graph model according to the dynamic path planning method;

The target data field search module 138 is used to search according to the shortest path, and find the target data field in the data field of the target node;

The aggregated data obtaining module 1310 is configured to combine target data fields to obtain aggregated data, and the aggregated data is used for display on the page.

In one embodiment, it also includes:

The data and relationship acquisition module is used to acquire various business data and conversion relationships between various business data;

The node and edge setting module is used to take each business data as a node, and set edges between corresponding nodes according to the conversion relationship between various business data;

The weight setting module is used to set the weight for each edge;

DAG model building block for building DAG models.

In one embodiment, the number of target nodes is multiple; the target data field search module further includes a sub-target data field search module and a sub-target data field combination module: the shortest path selection module is also used to select the starting node and each shortest path between target nodes;

The sub-target data field search module is used to search each shortest path by using the path search method, and find the sub-target data field from the data field of each target node;

The sub-target data field combining module is used for combining the sub-target data fields to obtain the target data fields.

In one embodiment, the sub-target data field search module also includes a path parsing module and a traversal module: a path parsing module, for using the jsonpath method to perform path parsing on the data fields of each of the target nodes to form a tree structure;

The traversal module is used for traversing the tree structure to obtain sub-target data fields.

In one embodiment, the service data identifiers include initial state service data identifiers and final state service data identifiers; the start node is the initial state node, and the target node is the final state node; the nodes of the directed acyclic graph model include the initial state node, Intermediate state nodes and final state nodes; one of the initial state nodes corresponds to one or more intermediate state nodes, and one or more final state nodes; also includes:

The initial state node determination module is used to find any node other than the most terminal node in the directed acyclic graph model according to the initial state service data identification, and determine it as the initial state node;

The final state node determination module is used to find the node in the directed acyclic graph model that is downstream of the initial state node and has a direct or indirect conversion relationship with the initial state node according to the final state node, and determines it as the final state node;

The intermediate state node determination module is used for determining a node located between the initial state node and the final state node and having a direct or indirect conversion relationship with the initial state node or the final state node as an intermediate state node.

In one embodiment, it also includes:

All path selection modules are used to filter out all paths between the initial state node and the final state node from the directed acyclic graph model according to the initial state node, the intermediate state node and the final state node;

The total weight calculation module is used to calculate the total weight value of each path according to the weight of each edge of the path;

The shortest path selection module is also used to select the path corresponding to the smallest total weight value as the shortest path.

In one embodiment, the shortest path selection module is further configured to adopt the Dijkstra algorithm to calculate the total weight value of each path by taking the weight of each edge as a positive value.

For the specific limitation of the data aggregation apparatus, reference may be made to the limitation on the method above, which will not be repeated here. Each module in the above apparatus may be implemented in whole or in part by software, hardware and combinations thereof. The above modules may be embedded in or independent of the processor in the terminal device in the form of hardware, or may be stored in the memory in the terminal device in the form of software, so that the processor can call and execute operations corresponding to the above modules.

Please refer to FIG. 14. FIG. 14 shows a structural block diagram of a terminal device provided by an embodiment of the present application. The terminal device 140 may be a computer device. The terminal device 140 in the present application may include one or more of the following components: a processor 142, a memory 144, and one or more application programs, wherein one or more application programs may be stored in the memory 142 and configured to be executed by a Executed by or multiple processors 144, one or more application programs are configured to execute the methods described in the foregoing method embodiments applied to terminal devices, and may also be configured to execute methods described in the foregoing method embodiments applied to data aggregation. method described.

Processor 144 may include one or more processing cores. The processor 144 uses various interfaces and lines to connect various parts of the entire terminal device 140, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 142, and calling the data stored in the memory 142. Various functions of the terminal device 140 and processing data. Optionally, the processor 144 may adopt at least one of a digital signal processing (Digital Signal Processing, DSP), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and a Programmable Logic Array (Programmable Logic Array, PLA). A form of hardware implementation. The processor 144 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a reporting verifier (Graphics Processing Unit, GPU) for buried point data, a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 134, and is implemented by a communication chip alone.

The memory 142 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 142 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 142 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data and the like created by the terminal device 140 in use.

Those skilled in the art can understand that the structure shown in FIG. 14 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the terminal equipment to which the solution of the present application is applied. The specific terminal device may be Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

To sum up, the terminal device provided in the embodiments of the present application is used to implement the corresponding data aggregation method in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

Please refer to FIG. 15 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable storage medium 150 stores program codes, and the program codes can be called by the processor to execute the methods described in the above data aggregation method embodiments, or can be invoked by the processor to execute the data aggregation method embodiments described above. Methods.

The computer readable storage medium 150 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 150 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 150 has storage space for program code 152 for performing any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. The program code 152 may be compressed, for example, in a suitable form.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, this application is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A data aggregation method, characterized in that the method comprises:

   Obtain business data aggregation parameters; the business data aggregation parameters include business data identifiers and target data fields;

   Obtain a directed acyclic graph model corresponding to the business data identifier, and determine a start node and a target node in the directed acyclic graph model according to the business data identifier; wherein, the directed acyclic graph The model is constructed based on pre-acquired various business data and the conversion relationship between various business data;

   Selecting the shortest path between the starting node and the target node from the directed acyclic graph model according to the dynamic path planning method;

   Search according to the shortest path, and find the target data field in the data field of the target node;

   The target data fields are combined to obtain aggregated data, and the aggregated data is used for display on the page.
2. The method according to claim 1, wherein the method for constructing the directed acyclic graph model comprises:

   Obtain a variety of business data and the conversion relationship between various business data;

   Taking each kind of the business data as a node, and setting edges between the corresponding nodes according to the conversion relationship between the various kinds of business data;

   A weight is set for each of the edges, and the directed acyclic graph model is constructed.
3. The method according to claim 1, wherein the number of the target nodes is multiple; the step of searching the target data field in the data field of the target node comprises:

   selecting the shortest path between the start node and each of the target nodes;

   Use the path search method to search each shortest path, and find the sub-target data field from the data field of each described target node;

   The sub-target data fields are combined to obtain the target data fields.
4. The method according to claim 3, wherein a path search method is used to search each shortest path, and the data fields of the sub-target data fields corresponding to the target data fields are searched from the data fields of each of the target nodes. steps, including:

   Use jsonpath method to perform path analysis on the data field of each described target node to form a tree structure;

   Traverse the tree structure to obtain sub-target data fields.
5. The method according to any one of claims 1 to 4, wherein the service data identifier includes an initial state service data identifier and a final state service data identifier; the starting node is an initial state node, and the target node is the final state node;

   The nodes of the directed acyclic graph model include an initial state node, an intermediate state node and a final state node; wherein one of the initial state nodes corresponds to one or more of the intermediate state nodes, and one or more of the final state nodes node;

   According to the initial state service data identifier, find any node other than the most terminal node in the directed acyclic graph model, and determine it as the initial state node;

   According to the final state node, find a node in the directed acyclic graph model that is located downstream of the initial state node and has a direct or indirect conversion relationship with the initial state node, and determines it as the final state node;

   A node located between the initial state node and the final state node and having a direct or indirect conversion relationship with the initial state node or the final state node is determined as the intermediate state node.
6. The method according to claim 5, wherein the step of selecting the shortest path between the start node and the target node from the directed acyclic graph model according to a dynamic path planning method comprises:

   Filter out all paths between the initial state node and the final state node from the directed acyclic graph model according to the initial state node, the intermediate state node and the final state node;

   The total weight value of each of the paths is calculated according to the weights of each edge of the path, and the path corresponding to the smallest total weight value is selected as the shortest path.
7. The method according to claim 6, wherein the total weight value of each path is calculated according to the weight of each edge of the path, and the path corresponding to the smallest total weight value is selected as the shortest path steps, including:

   Using Dijkstra's algorithm, the shortest path is selected by taking the weight of each of the edges as positive values.
8. A data aggregation device, characterized in that the device comprises:

   a parameter acquisition module for acquiring business data aggregation parameters; the business data aggregation parameters include a business data identifier and a target data field;

   A graph model obtaining module, configured to obtain a directed acyclic graph model corresponding to the business data identifier;

   A node determination module, configured to determine a start node and a target node in the directed acyclic graph model according to the service data identifier; wherein the directed acyclic graph model is based on pre-acquired multiple service data and The transformation relationship between various business data is constructed;

   a shortest path selection module, configured to select the shortest path between the start node and the target node from the directed acyclic graph model according to a dynamic path planning method;

   a target data field search module, configured to search according to the shortest path, and find the target data field in the data field of the target node;

   The aggregated data obtaining module is used for combining the target data fields to obtain aggregated data, and the aggregated data is used for displaying on the page.
9. A terminal device, characterized in that it includes:

   memory; one or more processors coupled to the memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, a or a plurality of application programs configured to perform the method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores program codes, and the program codes can be invoked by a processor to execute the method according to any one of claims 1-7.

Images