CN113392150A - Data table display method, device, equipment and medium based on service domain - Google Patents

Data table display method, device, equipment and medium based on service domain Download PDF

Info

Publication number
CN113392150A
CN113392150A CN202011238788.1A CN202011238788A CN113392150A CN 113392150 A CN113392150 A CN 113392150A CN 202011238788 A CN202011238788 A CN 202011238788A CN 113392150 A CN113392150 A CN 113392150A
Authority
CN
China
Prior art keywords
data
data table
metadata
node
service
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011238788.1A
Other languages
Chinese (zh)
Inventor
林岳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202011238788.1A priority Critical patent/CN113392150A/en
Publication of CN113392150A publication Critical patent/CN113392150A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/26Visual data mining; Browsing structured data

Abstract

The application discloses a data table display method based on a service domain, which comprises the following steps: acquiring M metadata from M data tables; determining the incidence relation among the M data tables according to the M metadata; constructing a target network according to the incidence relation among the M data tables, wherein the target network comprises M data nodes, the data nodes in the M data nodes and the data tables in the M data tables have corresponding relations, and each data node is used for storing one data table; carrying out area division processing on a target network to obtain at least one service domain; and when the operation aiming at the target viewing interface is acquired, displaying at least one data table corresponding to the target service domain through the interface of the terminal equipment. The embodiment of the application also provides a related device, equipment and a storage medium. According to the method and the device, the data sheet can be automatically integrated based on the service domain, so that developers can directly check the data sheet of a certain service domain conveniently, and time cost and labor cost are saved.

Description

Data table display method, device, equipment and medium based on service domain
Technical Field
The present application relates to the field of internet technologies, and in particular, to a method, an apparatus, a device, and a medium for displaying a data table based on a service domain.
Background
The data summarization refers to a process of performing aggregate summarization statistics on the service detail data stored in the database according to a specified dimension or a plurality of dimensions. In the process of software application, a large amount of data is generated, and usually, the data is stored in a database in the form of a data table so that a developer performs operations such as viewing or calling.
A large number of data tables are often stored in the database, and if a developer needs to obtain data for a certain service (or a certain service domain), data tables related to the service need to be found from the data tables, and then the related data tables are summarized, so that subsequent processing and analysis are performed.
However, since there are many types of service domains and different types of service domains may involve different data tables, developers need to not only know the data tables associated with a certain service (or a certain service domain), but also find out the associated data tables from a large number of data tables, resulting in high time cost and labor cost.
Disclosure of Invention
The embodiment of the application provides a data table display method based on a service domain, a related device, equipment and a storage medium, which can realize automatic integration of the data table based on the service domain, and are convenient for developers to directly view the data table corresponding to a certain service (or a certain service domain), thereby saving time cost and labor cost.
In view of this, an aspect of the present application provides a data table display method based on a service domain, including:
acquiring M metadata from M data tables, wherein the data tables in the M data tables have corresponding relations with the metadata in the M metadata, and M is an integer greater than or equal to 2;
determining the incidence relation among the M data tables according to the M metadata;
constructing a target network according to the incidence relation among the M data tables, wherein the target network comprises M data nodes, the data nodes in the M data nodes and the data tables in the M data tables have corresponding relations, and each data node is used for storing one data table;
performing area division processing on a target network to obtain at least one service domain, wherein each service domain comprises at least one data table;
when the operation aiming at the target viewing interface is obtained, at least one data table corresponding to the target service domain is displayed through the interface of the terminal equipment, wherein the target viewing interface is a viewing interface corresponding to the target service domain in at least one service domain.
Another aspect of the present application provides a data table display device, including:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring M metadata from M data tables, the data tables in the M data tables have corresponding relations with the metadata in the M metadata, and M is an integer greater than or equal to 2;
the determining module is used for determining the incidence relation among the M data tables according to the M metadata;
the system comprises a construction module, a data processing module and a data processing module, wherein the construction module is used for constructing a target network according to the incidence relation among M data tables, the target network comprises M data nodes, the data nodes in the M data nodes and the data tables in the M data tables have corresponding relations, and each data node is used for storing one data table;
the system comprises a dividing module, a sending module and a receiving module, wherein the dividing module is used for carrying out regional division processing on a target network to obtain at least one service domain, and each service domain comprises at least one data table;
and the display module is used for displaying at least one data table corresponding to the target service domain through an interface of the terminal equipment when the operation aiming at the target viewing interface is obtained, wherein the target viewing interface is a viewing interface corresponding to the target service domain in at least one service domain.
In one possible design, in another implementation manner of another aspect of the embodiment of the present application, the M data tables at least include a first data table and a second data table;
the acquisition module is specifically used for acquiring first metadata from a first data table;
obtaining second metadata from a second data table;
the determining module is specifically configured to determine a target association relationship between the first data table and the second data table according to the first metadata and the second metadata, where the target association relationship includes at least one of an edge connecting direction and an edge connecting weight between the first data node and the second data node, the first data node is used to store the first data table, and the second data node is used to store the second data table.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the determining module is specifically configured to determine, according to technical metadata included in the first metadata and technical metadata included in the second metadata, a direction of a connecting edge between the first data node and the second data node;
and determining the connection edge weight between the first data node and the second data node according to the service metadata included by the first metadata and the service metadata included by the second metadata.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the determining module is specifically configured to obtain a data blood margin corresponding to the first data table according to technical metadata included in the first metadata;
acquiring a data blood margin corresponding to the second data table according to the technical metadata included in the second metadata;
determining an upstream data table from the first data table and the second data table according to the data blood margin corresponding to the first data table and the data blood margin corresponding to the second data table, wherein the data blood margin belongs to the technical metadata;
if the upstream data table is the first data table, constructing a connecting edge from the first data node to the second data node;
and if the upstream data table is the second data table, constructing a connecting edge from the second data node to the first data node.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the determining module is specifically used for acquiring a service name corresponding to the first data table according to the service metadata included in the first metadata;
acquiring a service name corresponding to the second data table according to service metadata included in the second metadata;
determining the association degree between the first data table and the second data table according to the business name corresponding to the first data table and the business name corresponding to the second data table;
and determining the connection edge weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the determining module is specifically configured to obtain a service description corresponding to the first data table according to service metadata included in the first metadata;
acquiring a service description corresponding to the second data table according to service metadata included in the second metadata;
acquiring the association degree between the first data table and the second data table through a semantic matching model based on the service description corresponding to the first data table and the service description corresponding to the second data table;
and determining the connection edge weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the device comprises a dividing module, a judging module and a judging module, wherein the dividing module is specifically used for dividing data nodes in a target network to obtain N areas, and N is an integer which is greater than or equal to 1 and less than or equal to M;
at least one service domain is determined according to the N regions.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the dividing module is specifically used for acquiring data nodes to be divided from a target network;
acquiring a first data node and a second data node according to the data node to be divided, wherein the first data node and the second data node are both data nodes adjacent to the data node to be divided;
determining a first modularity according to the data node to be divided and the first data node;
determining a second modularity according to the data node to be divided and the second data node;
if the first modularity and the second modularity are both greater than 0 and the first modularity is greater than the second modularity, determining that the data node to be divided and the first data node belong to the same region of the N regions;
if the first modularity and the second modularity are both greater than 0 and the first modularity is smaller than the second modularity, determining that the data node to be divided and the second data node belong to the same region of the N regions;
and obtaining N areas until the algorithm termination condition is met.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the device comprises a dividing module, a first processing module and a second processing module, wherein the dividing module is specifically used for acquiring a first area and a second area from a target network;
acquiring a first gain value according to the first area and the second area, wherein the first gain value is the sum of the number of edges in the first area and the number of edges in the second area and the difference value of the number of edges between the first area and the second area;
acquiring a first data node in a first area and a second data node in a second area;
adding the second data node into the first area to obtain an updated first area, and adding the first data node into the second area to obtain an updated second area;
acquiring a second gain value according to the updated first region and the updated second region, wherein the second gain value is the sum of the updated number of edges in the first region and the updated number of edges in the second region and the difference between the updated number of edges in the first region and the updated number of edges in the second region;
determining a target gain value according to the first gain value and the second gain value;
if the target gain value is the maximum value of the P gain values, determining that the first data node belongs to the updated second region and the second data node belongs to the updated first region, wherein the P gain values comprise the gain values of every two data nodes between the first region and the second region, and P is an integer greater than or equal to 1;
and obtaining N areas until the algorithm termination condition is met.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the dividing module is specifically used for determining K edge betweenness according to the M data nodes and the K connecting edges in the target network, wherein the edge betweenness in the K edge betweenness has a corresponding relation with the connecting edges in the K connecting edges;
selecting a target edge betweenness from the K edge betweenness, wherein the target edge betweenness is the maximum value of the K edge betweenness;
deleting the connecting edges corresponding to the intermediate number of the target edge;
and obtaining N areas until the algorithm termination condition is met.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the device comprises a dividing module, a judging module and a judging module, wherein the dividing module is specifically used for acquiring Q data nodes included in an area to be identified from N areas, and Q is an integer greater than or equal to 1;
and determining a service domain corresponding to the region to be identified according to Q data tables corresponding to the Q data nodes, wherein the data nodes in the Q data nodes have a corresponding relation with the data tables in the Q data tables.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the display module is specifically used for displaying the business name and the viewing interface corresponding to each business domain;
and when the operation aiming at the target viewing interface is detected, displaying at least one data table corresponding to the target service domain through the interface of the terminal equipment.
In one possible design, in another implementation of another aspect of an embodiment of the present application,
the display module is specifically used for receiving a viewing instruction sent by the terminal equipment when the terminal equipment detects the operation aiming at the target viewing interface;
and sending at least one data table corresponding to the target service domain to the terminal equipment according to the viewing instruction so that the terminal equipment displays the at least one data table corresponding to the target service domain.
Another aspect of the present application provides a computer device, comprising: a memory, a processor, and a bus system;
wherein, the memory is used for storing programs;
a processor for executing the program in the memory, the processor for performing the above-described aspects of the method according to instructions in the program code;
the bus system is used for connecting the memory and the processor so as to enable the memory and the processor to communicate.
Another aspect of the present application provides a computer-readable storage medium having stored therein instructions, which when executed on a computer, cause the computer to perform the method of the above-described aspects.
In another aspect of the application, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided by the above aspects.
According to the technical scheme, the embodiment of the application has the following advantages:
the embodiment of the application provides a data table display method based on a service domain, which includes the steps of firstly obtaining M metadata from M data tables, wherein the data tables and the metadata have corresponding one-to-one correspondence, then determining an association relationship among the M data tables according to the M metadata, accordingly, constructing a target network according to the association relationship among the M data tables, wherein the target network includes M data nodes, each data node and the data tables also have one-to-one correspondence, each data node is used for storing one data table, finally, performing region division processing on the target network to obtain at least one service domain, each service domain includes at least one data table, and when the operation aiming at a target viewing interface is obtained, displaying the at least one data table corresponding to the target service domain through an interface of a terminal device. Through the method, the relation among the data tables can be determined by utilizing the metadata corresponding to each data table, a target network is formed based on the relation among the data tables, then the target network can be subjected to region division by adopting a community division algorithm, so that at least one region is obtained, each divided region can be regarded as a service domain, and each data node in the region is each data table included in the service domain, so that the automatic integration of the data tables can be realized based on the service domain, developers can conveniently and directly check the data tables of a certain service domain, and the time cost and the labor cost are saved.
Drawings
FIG. 1 is a schematic diagram of an architecture of a business processing system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an environment of a business processing system in an embodiment of the present application;
FIG. 3 is a flowchart illustrating a data table display method based on a service domain according to an embodiment of the present application;
FIG. 4 is a diagram illustrating a conversion from a data table to a data node according to an embodiment of the present application;
FIG. 5 is a schematic diagram of generating a service domain based on a target network in an embodiment of the present application;
FIG. 6 is a schematic diagram of an association relationship between data nodes in an embodiment of the present application;
FIG. 7 is another diagram illustrating an association relationship between data nodes in an embodiment of the present application;
FIG. 8 is a schematic diagram illustrating an embodiment of the present invention for determining the direction of a connecting edge between data nodes based on data blood relationship;
FIG. 9 is a schematic diagram of dividing regions based on the fast unfolding algorithm in the embodiment of the present application;
FIG. 10 is a schematic diagram of the region division based on Kernighan-Lin algorithm in the embodiment of the present application;
FIG. 11 is a schematic illustration of the partitioning of regions based on GN algorithm in an embodiment of the present application;
FIG. 12 is a schematic diagram of an interface showing business domains in an embodiment of the present application;
FIG. 13 is a schematic diagram of an interface showing a data table under a target service domain in an embodiment of the present application;
FIG. 14 is a schematic diagram of an embodiment of a data table showing device in the embodiment of the present application;
FIG. 15 is a schematic structural diagram of a server in an embodiment of the present application;
fig. 16 is a schematic structural diagram of a terminal device in the embodiment of the present application.
Detailed Description
The embodiment of the application provides a data table display method based on a service domain, a related device, equipment and a storage medium, which can realize automatic integration of the data table based on the service domain, and are convenient for developers to directly extract the data table corresponding to a certain service (or a certain service domain), thereby saving time cost and labor cost.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "corresponding" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
With the rapid development of industries such as social contact, e-commerce, finance, internet of things and the like, a huge relationship network is formed, and the relationship among data needing to be processed in the big data industry increases in a geometric exponential manner along with the data volume. The database is used as an important carrier for storing important and sensitive information of enterprises, bears more and more key business systems, becomes an important asset of the enterprises gradually, and is vital to development and maintenance of applications on how to effectively and timely carry out data asset combing work. In the process of application development, operation, maintenance, and the like, related personnel are usually required to look up a corresponding data table from a database for a specific service (or a specific service domain), and finally, the data tables are summarized for analysis and processing. Based on the data table display method based on the business domain, the problem of division of the business domain of the data assets can be solved systematically and automatically, the business domain division is carried out on the data assets through collection of business metadata and technical metadata and through a series of feature construction and transformation, effective operation and deposition of the data assets are achieved, relevant personnel can clearly know which data tables and valuable data assets exist in the current business domain, and accordingly the data assets are managed well.
It is to be understood that the data assets referred to herein include, but are not limited to, data assets of financial transactions, data assets of social transactions, data assets of search transactions, data assets of game transactions, data assets of video transactions, data assets of audio transactions, data assets of pay transactions, and data assets of subscription transactions. The related personnel involved in the present application may be internal personnel of an enterprise, such as operators, data analysts and product designers, external personnel of an enterprise, such as partners, or personnel of a data development team, such as development engineers, programmers, business analysts, etc., which are not limited herein.
For convenience of understanding, please refer to fig. 1, fig. 1 is a schematic diagram of an architecture of a business processing system in an embodiment of the present application, and as shown in the figure, the architecture of the business processing system may include five levels, which are a database, a data processing layer, a data management layer, a data service layer, and an application layer from bottom to top, and the contents of each level are introduced below.
A database is a repository that organizes, stores, and manages data according to a data structure, which is an organized, sharable, and uniformly managed collection of large amounts of data that is stored on a computer device for a long period of time. The data in the database may originate from internal systems as well as from external systems. The internal systems include, but are not limited to, an enterprise internal management system such as Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), and Software Configuration Management (SCM), a point of sale (POS) system of a retail channel, an enterprise-owned website, an Application (APP), an own e-commerce platform, an offline retail site, and a customer service center system. External systems include, but are not limited to, third party e-commerce platforms, search engines, Internet Service Provider (ISP) platforms, Digital Signal Processors (DSP) platforms, third party payment platforms, social media platforms, and third party data providers.
The data processing layer comprises data recognition, data cleaning and data fusion, wherein the data recognition can recognize different data types from the database, for example, the data types belong to user identification or gender and the like. Data cleansing deals with problems of data loss, out-of-bounds, inconsistent code, and duplicate data in terms of accuracy, integrity, consistency, uniqueness, timeliness, and validity of data. Data fusion can merge data under the same type.
The data management layer comprises data asset planning, data asset processing, data asset quality, data operation and maintenance, data asset safety and metadata management. The data asset planning specifically comprises data architecture management, data standardization, dimension table standardization, index standardization, data map planning and the like. The data asset processing comprises data flow design, data model design, data processing development, data application development, data testing, online and the like. The data operation and maintenance comprises operation monitoring, alarm management, data evaluation, data optimization, storage optimization, offline management and the like. The data asset quality comprises quality planning management, quality planning inspection, quality problem management and the like. The data asset security comprises security policy management, security vulnerability check, authority application distribution, security audit and the like. The metadata management comprises metadata collection, metadata classification, metadata verification, data relation analysis, field relation analysis, metadata service and the like.
The data service layer provides an Application Programming Interface (API), which is a predefined function for the related personnel to call or configure, so as to implement the processes of authority control, service call, interface configuration, and the like.
The application layer mainly refers to application of data assets, and specifically includes operation diagnosis of services, implementation of machine learning based on service data, analysis and mining of service data, and viewing of a data table under a certain service (or a certain service domain).
In order to facilitate that a relevant person can quickly query a data table under a certain service domain, the application provides a data table display method based on the service domain, the method is applied to the service processing system shown in fig. 2, and as shown in fig. 2, the service processing system includes a database, a server and a terminal device. The databases in the present application may be relational or non-relational databases. The server in the application can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, Network service, cloud communication, middleware service, domain name service, security service, Content Delivery Network (CDN), big data and an artificial intelligence platform. The terminal device in the present application may be a smart phone, a tablet computer, a notebook computer, a palm computer, a personal computer, a smart television, a smart watch, and the like, but is not limited thereto. The terminal device and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein. The number of servers and terminal devices is not limited.
In view of the fact that some terms are referred to in the embodiments of the present application, they will be described below.
1. Metadata (metadata): refers to data that describes relationships between data. Metadata in a data application system generally refers to data describing concepts (concepts), relationships (roles) between data and rules (rules) for data processing, and domain semantics (semantics) and knowledge (knowledge) also belong to the category of metadata.
2. Technical metadata: the technical statistical indexes generated in the data development process refer to data used by data warehouse design and management personnel for developing and daily managing the data warehouse. The technical metadata table comprises data source information, description of data conversion, definition of objects and data structures in a data warehouse, rules used in data cleaning and data updating, mapping of source data to target data, user access authority, data backup history, data import history, information release history and the like. Common technical metadata include data blood margin, fan-in number, fan-out number, field name, field length, and database table structure.
3. Service metadata: the information of business name, definition, description, etc. is used to represent various attributes and concepts in the enterprise environment, and to a certain extent, the business context behind all data can be regarded as business metadata, such as business name, business definition, business description, etc.
4. Data blood relationship: the method belongs to a concept in data management, finds the relation among related data in the process of data tracing, and belongs to a logic concept. The data's genetic relationship also includes some characteristic features, such as attribute, multi-source, traceability and hierarchy.
Specific data is attributed to a specific organization or individual, i.e., has attributes.
The same data can have multiple sources, one data can be generated by processing multiple data, and the processing process can be multiple, namely, multiple sources are provided.
The blood relationship of the data represents the life cycle of the data and represents the whole process from generation to extinction of the data, namely the traceability is realized.
The data is hierarchical in its relationship to blood. The classification, induction and summarization of the data, and the information describing the data forms new data, and the description information of different degrees forms the hierarchy of the data.
5. Complex networks: refers to a network with some or all of the properties of self-organization, self-similarity, attractor, worlds, unscaled. The research directions comprise key node discovery, community discovery and link prediction. Among them, the key node discovery is intended to discover nodes that play a key role in the structure and function of the network. Community discovery aims at discovering community structures in a complex network so as to reasonably divide the composition of network nodes. Link prediction aims to predict the likelihood of a link existing between any nodes in a complex network.
6. And (4) service domain: the method refers to the business range and the field of data and belongs to one of the key metadata. The business domain refers broadly to the subject matter of the data sheet, for example, for WeChat payments, the business domain may have red envelope, money transfer, financial products, or marketing products, etc.
With reference to the above description, the following description will describe a data table display method based on a service domain in the present application, and referring to fig. 3, an embodiment of the data table display method based on a service domain in the present application includes:
101. acquiring M metadata from M data tables, wherein the data tables in the M data tables have corresponding relations with the metadata in the M metadata, and M is an integer greater than or equal to 2;
in this embodiment, the data table display apparatus may obtain M data tables from the data, where each data table has one corresponding metadata. It should be noted that the data table display apparatus may be deployed in a computer device, where the computer device may be a server, or a terminal device, or a system composed of a server and a terminal device, and the present application is not limited thereto.
For ease of understanding, please refer to table 1, which is an example of a data table.
TABLE 1
Figure BDA0002767686060000081
As can be seen from Table 1, a data table may include table names, fields within the table, and records within the table, with the table names ensuring uniqueness and the table names being simple and intuitive to use. The field lengths in the tables are typically less than 64 characters, and the field names include letters, characters, numbers, spaces, and other characters. The records of the table are the specific parameters under the corresponding fields.
With reference to table 1, please refer to table 2, where table 2 is an example of metadata corresponding to a data table.
TABLE 2
Figure BDA0002767686060000082
Figure BDA0002767686060000091
It should be noted that, only part of the metadata is shown in table 2, and in an actual situation, the metadata may further include a fan-in number, a fan-out number, a database table structure, and the like, which is not limited herein.
102. Determining the incidence relation among the M data tables according to the M metadata;
in this embodiment, the data table showing device takes the metadata of each data table as the characteristics of the data table, and based on the metadata, the association relationship between the data tables can be constructed, for example, the upstream and downstream relationship between the data tables can be determined based on the data blood margin. For ease of understanding, please refer to fig. 4, in which fig. 4 is a schematic diagram illustrating the conversion from the data table to the data node according to the embodiment of the present application. As shown in fig. 4 (a), taking 10 data tables as an example, the association relationship between the data tables is determined according to the metadata of the 10 data tables, wherein the arrow points to the upstream data table which is the association relationship between the data tables, for example, "product specification table" is the "product detail table", and "product detail table" is the downstream data table of "product access information table" and "discount information table".
103. Constructing a target network according to the incidence relation among the M data tables, wherein the target network comprises M data nodes, the data nodes in the M data nodes and the data tables in the M data tables have corresponding relations, and each data node is used for storing one data table;
in this embodiment, the data table display apparatus may construct a target network based on an association relationship between M data tables, where the target network may be a complex network, the target network includes M data nodes, each data node is used to store one data table, and a feature of each data node is metadata of the data table.
For easy understanding, please refer to fig. 4 again, as can be seen from the diagram (a) in fig. 4, assuming that M is 10, that is, 10 data tables exist, a target network, that is, the target network shown in the diagram (B) in fig. 4, is constructed according to the association relationship between the 10 data tables, and includes 10 data nodes, where data node No. 1 is used to store "category specification table", data node No. 2 is used to store "commodity category table", data node No. 3 is used to store "discount information table", data node No. 4 is used to store "commodity list", data node No. 5 is used to store "commodity specification table", data node No. 6 is used to store "commodity access information table", data node No. 7 is used to store "user information table", data node No. 8 is used to store "order approval table", data node No. 9 is used to store "order history table", the data node number 10 is used for storing an order commodity table.
It should be noted that the number of data nodes and the number of data tables shown in fig. 4 are only one example, and should not be construed as limiting the present application.
104. And carrying out area division processing on the target network to obtain at least one service domain, wherein each service domain comprises at least one data table.
In this embodiment, the data table display apparatus divides the data nodes in the target network to obtain at least two areas (or communities), and then determines whether the divided areas can be used as service domains, and if the areas can be used as service domains, obtains the service domains. Since each area includes at least one data node, the data tables stored by these data nodes are taken as the data tables included in the service domain.
For convenience of explanation, please refer to fig. 5, where fig. 5 is a schematic diagram of generating a service domain based on a target network in the embodiment of the present application, and as shown in the diagram, it is assumed that three regions, namely, a region a, a region B, and a region C, are obtained after dividing the target network, where it is assumed that the region a, the region B, and the region C all satisfy a service domain determination condition, and then the service domain a, the service domain B, and the service domain C can be obtained. Since there are 10 data nodes in the area a, the service domain a includes 10 data tables. Since there are 7 data nodes in the area B, the service domain a includes 7 data tables. Since there are 14 data nodes in the area C, the service domain C includes 14 data tables.
105. When the operation aiming at the target viewing interface is obtained, at least one data table corresponding to the target service domain is displayed through the interface of the terminal equipment, wherein the target viewing interface is a viewing interface corresponding to the target service domain in at least one service domain.
In this embodiment, when the data table display device obtains an operation for a target viewing interface, a corresponding target service domain may be determined according to the target viewing interface, where the target service domain is one of at least one service domain. Therefore, the data table display device can display at least one data table corresponding to the target service domain through the interface of the terminal equipment.
The embodiment of the application provides a data table display method based on a service domain, which includes the steps of firstly obtaining M metadata from M data tables, wherein the data tables and the metadata have corresponding one-to-one correspondence, then determining an association relationship among the M data tables according to the M metadata, accordingly, constructing a target network according to the association relationship among the M data tables, wherein the target network includes M data nodes, each data node and the data tables also have one-to-one correspondence, each data node is used for storing one data table, finally, performing region division processing on the target network to obtain at least one service domain, each service domain includes at least one data table, and when the operation aiming at a target viewing interface is obtained, displaying the at least one data table corresponding to the target service domain through an interface of a terminal device. Through the method, the relation among the data tables can be determined by utilizing the metadata corresponding to each data table, a target network is formed based on the relation among the data tables, then the target network can be subjected to region division by adopting a community division algorithm, so that at least one region is obtained, each divided region can be regarded as a service domain, and each data node in the region is each data table included in the service domain, so that the automatic integration of the data tables can be realized based on the service domain, developers can conveniently and directly check the data tables of a certain service domain, and the time cost and the labor cost are saved.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in the embodiment of the present application, the M data tables at least include a first data table and a second data table;
acquiring M metadata from M data tables, specifically comprising the following steps:
obtaining first metadata derived from a first data table;
obtaining second metadata from a second data table;
determining the association relation among the M data tables according to the M metadata, which specifically comprises the following steps:
and determining a target association relation between the first data table and the second data table according to the first metadata and the second metadata, wherein the target association relation comprises at least one of an edge connecting direction and an edge connecting weight between the first data node and the second data node, the first data node is used for storing the first data table, and the second data node is used for storing the second data table.
In this embodiment, a method for constructing an association relationship between data tables is described. For convenience of description, a first data table and a second data table of the M data tables are taken as examples, and it is understood that, for other data tables of the M data tables, association relationships between the data tables may also be determined in a similar manner, and details are not repeated here.
Specifically, the data table display device first determines a first data table and a second data table, and then obtains first metadata corresponding to the first data table and second metadata of the second data table. And if the association relationship between the first metadata and the second metadata exists, determining that the first data table and the second data table also have the association. The first data table is stored in the first data node, the second data table is stored in the second data node, and the target association relationship between the first data node and the second data node is the target association relationship between the first data table and the second data table.
It should be noted that the target association relationship may include a direction of a connecting edge between the first data node and the second data node, or the target association relationship may include a weight of a connecting edge between the first data node and the second data node, or the target association relationship includes both the direction of a connecting edge between the first data node and the second data node and the weight of a connecting edge between the first data node and the second data node.
Secondly, in the embodiment of the application, a way of constructing the association between the data tables is provided, and through the way, for the two data tables, the association between the two data tables can be determined by using the corresponding metadata, so that a more accurate association is obtained, and the feasibility of the scheme is improved.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided by the embodiment of the present application, the determining the target association relationship between the first data table and the second data table according to the first metadata and the second metadata specifically includes the following steps:
determining a connecting edge direction between the first data node and the second data node according to the technical metadata included by the first metadata and the technical metadata included by the second metadata;
and determining the connection edge weight between the first data node and the second data node according to the service metadata included by the first metadata and the service metadata included by the second metadata.
In this embodiment, a method for determining a target association relationship according to metadata is described. The first metadata table comprises technical metadata and service metadata, the second metadata also comprises the technical metadata and the service metadata, and the technical metadata can reflect the source, the composition and the like of data. The edge connecting weight between the two data nodes can be determined based on the service metadata, wherein the edge connecting weight represents the relevance between the two data nodes, and the stronger the weight is, the stronger the relevance is, and the relevance between the data tables can be obtained.
The following describes the way of constructing the edge connecting direction and the edge connecting weight between the data nodes, respectively, with reference to two examples.
Firstly, constructing a connecting edge direction between data nodes;
specifically, referring to fig. 6, fig. 6 is a schematic diagram of an association relationship between data nodes in the embodiment of the present application, and as shown in (a) in fig. 6, taking an example that a target network includes 7 data nodes, a connecting edge between 7 data nodes represents an association between data nodes. Since each data node stores one data table, each data table corresponds to one metadata, a side connecting direction between the data nodes can be further constructed based on the metadata of 7 data nodes, that is, a side connecting direction shown in fig. 6 (B) is obtained. The two data nodes connected by the arrow have an upstream-downstream relationship, and the arrow points to the downstream data node, for example, the data node 1 is an upstream node of the data node 3, it is assumed that the data node 1 stores the data table 1, and the data node 3 stores the data table 3, that is, the data table 1 is an upstream data table of the data table 3.
Secondly, constructing the connection edge weight between the data nodes;
specifically, referring to fig. 7, fig. 7 is another schematic diagram of an association relationship between data nodes in the embodiment of the present application, as shown in (a) in fig. 7, taking an example that a target network includes 7 data nodes, a connecting edge between 7 data nodes represents an association between data nodes, and a connecting edge direction represents an upstream-downstream relationship between data nodes. Since each data node stores one data table, each data table corresponds to one metadata, the edge connection weight between the data nodes can be constructed based on the metadata of 7 data nodes. The thicker the connecting edge is, the larger the weight is, and conversely, the thinner the connecting edge is, the smaller the weight is. For example, the weight of the connecting edge between the data node No. 6 and the data node No. 7 is smaller, and it is assumed that the data node No. 6 stores the data table No. 6, and the data node No. 7 stores the data table No. 7, that is, the association between the data table No. 6 and the data table No. 7 is lower. For another example, the weight of the connecting edge between the data node No. 2 and the data node No. 5 is larger, and it is assumed that the data node No. 2 stores the data table No. 2 and the data node No. 5 stores the data table No. 5, that is, the association between the data table No. 2 and the data table No. 5 is higher.
In the embodiment of the present application, a method for determining a target association relationship according to metadata is provided, and through the above method, an association relationship between two data nodes can be constructed from different dimensions (i.e., an edge connecting direction and an edge connecting weight) by combining technical metadata and service metadata, and then an association relationship between data tables is determined, so that feasibility and operability of a scheme are improved.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided by the embodiment of the present application, the determining, according to the technical metadata included in the first metadata and the technical metadata included in the second metadata, the edge connecting direction between the first data node and the second data node specifically includes the following steps:
acquiring a data blood margin corresponding to the first data table according to the technical metadata included in the first metadata;
acquiring a data blood margin corresponding to the second data table according to the technical metadata included in the second metadata;
determining an upstream data table from the first data table and the second data table according to the data blood margin corresponding to the first data table and the data blood margin corresponding to the second data table, wherein the data blood margin belongs to the technical metadata;
if the upstream data table is the first data table, constructing a connecting edge from the first data node to the second data node;
and if the upstream data table is the second data table, constructing a connecting edge from the second data node to the first data node.
In this embodiment, a method for determining the direction of an edge between data nodes based on a data blood margin is described. The first metadata comprises technical metadata, and the second metadata also comprises the technical metadata, wherein the technical metadata can reflect the source, the composition and the like of data, and therefore, the connecting edge direction between two data nodes can be determined based on the technical metadata.
Specifically, the data table display device may determine the data blood relationship of the first data table and the blood relationship of the second data table according to the data blood relationship table, and if the data blood relationship table exists, the data table display device may directly extract the data blood relationship of the data table from the data blood relationship table. The data lineage relationship table is a table for storing data lineage relationships between data nodes, and for example, each data table may include a table name of a downstream data table and a table name of an upstream data table, and may record a manner in which the downstream data table is processed from the upstream data table. If the data lineage relationship table does not exist, the data table presentation device can periodically obtain technical metadata from Structured Query Language (SQL) code information and log information, and further extract data lineage from the technical metadata.
Since the data tables may be stored in the data nodes, the data bloodlines between the data tables may directly affect the upstream and downstream relationships between the data nodes. Taking the first data table and the second data table as an example, if the first data table is an upstream data table of the second data table, it indicates that the first data node is an upstream data node of the second data node, and therefore, the direction of the edge between the first data node and the second data node is from the first data node to the second data node. On the contrary, if the second data table is an upstream data table of the first data table, it indicates that the second data node is an upstream data node of the first data node, and therefore, the direction of the edge from the first data node to the second data node is from the second data node to the first data node.
For convenience of understanding, please refer to fig. 8, fig. 8 is a schematic diagram illustrating determining directions of connecting edges between data nodes based on data blood margins in the embodiment of the present application, as shown in the figure, 5 data nodes are taken as an example, and each data node stores a data table, where the data table No. 1 is an upstream data table of another 4 data packets, that is, the data node No. 1 is an upstream data node of another 4 data nodes, where W _1 and W _ 2 represent weights from the data node No. 1 to the data node No. 2, that is, represent an association between the data table No. 1 and the data table No. 2. W _1,3 represents the weight from data node No. 1 to data node No. 3, i.e. the association between data table No. 1 and data table No. 3. W _1,4 represents the weight from data node No. 1 to data node No. 4, i.e. the association between data table No. 1 and data table No. 4. W _1,5 represents the weight from data node No. 1 to data node No. 5, i.e. the association between data table No. 1 and data table No. 5.
Further, in the embodiment of the application, a mode for determining the direction of connecting edges between data nodes based on data blood margins is provided, and through the mode, the relationship between points and edges can be further described for all data tables based on the incidence relationship between the data tables described by the data blood margins, and then the community division and clustering are performed on the data assets through a data mining method introduced into a complex network, so that the service domain corresponding to the data assets is located, the data assets are subjected to fine management, and the effective operation and precipitation of the data assets are improved.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided by the embodiment of the present application, the determining the edge connecting weight between the first data node and the second data node according to the service metadata included in the first metadata and the service metadata included in the second metadata specifically includes the following steps:
acquiring a service name corresponding to the first data table according to service metadata included in the first metadata;
acquiring a service name corresponding to the second data table according to service metadata included in the second metadata;
determining the association degree between the first data table and the second data table according to the business name corresponding to the first data table and the business name corresponding to the second data table;
and determining the connection edge weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
In this embodiment, a method for determining a connection weight based on a service name is described. The first metadata comprises technical metadata, and the second metadata also comprises technical metadata, wherein the technical metadata can reflect the source, the composition and the like of data, and therefore, the connecting edge weight between two data nodes can be determined based on the technical metadata.
Specifically, for example, by determining the edge weight between the first data node and the second data node, the service name of the first data table and the service name of the second data table are first obtained, and then the similarity between the two service names may be determined based on a Natural Language Processing (NLP) technique. The text similarity can be measured mainly in the following three ways, which will be described separately below.
Firstly, matching method based on key words;
(1) defining the similarity of the service name based on an N-tuple language model (N-Gram), wherein the calculation of the N-Gram similarity refers to segmenting the service name according to the length N to obtain word segments, namely all substrings with the length N in the service name. For two business names, the association degree of the two business names can be defined from the number of common substrings.
(2) The Jaccard's (Jaccard) algorithm can compute the ratio of the intersection and union of sets of words between two business names. The larger the value is, the more similar the two service names are, and the method has certain advantages in efficiency when large-scale parallel operation is involved.
Secondly, a matching method based on vector space;
a vector of each service name is generated based on Word to vector (Word 2vec), and then the correlation between two service names may be calculated using euclidean distance, manhattan distance, cosine similarity distance, hamming distance, or pearson correlation coefficient, etc.
Thirdly, matching method based on deep learning;
the relevance between text names is predicted by using a pre-trained Semantic matching model, which includes, but is not limited to, Deep Structured Semantic Models (DSSMs), Convolutional Neural Networks (CNNs), Long Short Term Memory (LSTM) Networks, and tree LSTM Networks, and is not limited herein.
Further, in the embodiment of the present application, a method for determining a weight of an edge connection based on a business name is provided, and through the above method, an association between data tables can be determined through the business name, in general, the higher the degree of association of the business name is, the more compact the relationship between two data tables is, and the weight of the edge connection constructed by the method is more accurate.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided by the embodiment of the present application, the determining the edge connecting weight between the first data node and the second data node according to the service metadata included in the first metadata and the service metadata included in the second metadata specifically includes the following steps:
acquiring a service description corresponding to the first data table according to service metadata included in the first metadata;
acquiring a service description corresponding to the second data table according to service metadata included in the second metadata;
acquiring the association degree between the first data table and the second data table through a semantic matching model based on the service description corresponding to the first data table and the service description corresponding to the second data table;
and determining the connection edge weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
In this embodiment, a method for determining a connection weight based on a service description is introduced. The first metadata comprises technical metadata, and the second metadata also comprises technical metadata, wherein the technical metadata can reflect the source, the composition and the like of data, and therefore, the connecting edge weight between two data nodes can be determined based on the technical metadata.
Specifically, for example, determining the edge connecting weight between the first data node and the second data node, first obtaining the service description of the first data table and the service description of the second data table, then inputting the service descriptions into the semantic matching model, and outputting the association degree (i.e., the association degree score) through the semantic matching model, where the greater the association degree, the greater the edge connecting weight.
It should be noted that, in practical applications, the association between two service descriptions may also be determined based on a matching method of a keyword or a matching method of a vector space, which is not described herein again.
Further, in the embodiment of the present application, a method for determining a weight of a connection edge based on a service description is provided, and through the above method, since the service description often covers information related to a service, an association between data tables can be determined through the service description, in general, the higher the association degree of a service name is, the tighter the relationship between two data tables is, and the more accurate the weight of the connection edge constructed by the method is.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in the embodiment of the present application, performing area division processing on the target network to obtain at least one service domain, specifically includes the following steps:
dividing data nodes in a target network to obtain N areas, wherein N is an integer which is greater than or equal to 1 and less than or equal to M;
at least one service domain is determined according to the N regions.
In this embodiment, a method for performing area division on a target network is described. By utilizing a community discovery algorithm, data edge nodes in the target network can be divided, so that N divided regions are obtained, and at least one service domain is determined from the N regions.
Specifically, there are various types of community discovery algorithms, including an algorithm based on graph segmentation, a method based on hierarchical clustering, and a method based on modularity optimization. The algorithm based on graph segmentation may include a Kernighan-Lin (KL) algorithm, a spectrum bisection method, and the like. Hierarchical clustering-based methods may include the Greenman (GN) algorithm and the Newman (Newman) fast algorithm, among others. Methods based on modularity Optimization may include a fast unfolding (fast unfolding) algorithm, a greedy algorithm, a simulated annealing algorithm, a co-evolution (Memetic) algorithm, a Particle Swarm Optimization (PSO) algorithm, an evolutionary multi-objective Optimization algorithm, and the like.
Furthermore, in the embodiment of the present application, a method for performing area division on a target network is provided, and through the above method, automatic area division is achieved by using an algorithm, and different areas do not need to be manually divided, so that flexibility and convenience of a scheme are improved.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in the embodiment of the present application, the dividing processing is performed on the data nodes in the target network to obtain N areas, and specifically includes the following steps:
acquiring data nodes to be divided from a target network;
acquiring a first data node and a second data node according to the data node to be divided, wherein the first data node and the second data node are both data nodes adjacent to the data node to be divided;
determining a first modularity according to the data node to be divided and the first data node;
determining a second modularity according to the data node to be divided and the second data node;
if the first modularity and the second modularity are both greater than 0 and the first modularity is greater than the second modularity, determining that the data node to be divided and the first data node belong to the same region of the N regions;
if the first modularity and the second modularity are both greater than 0 and the first modularity is smaller than the second modularity, determining that the data node to be divided and the second data node belong to the same region of the N regions;
and obtaining N areas until the algorithm termination condition is met.
In this embodiment, a method for dividing an area by a fast unfolding algorithm is introduced. After the data nodes and the connecting edges are constructed, a fast unfolding algorithm based on the modularity Q can be adopted for dividing the areas. The modularity becomes an important standard for measuring the quality of community division, the larger the network modularity value after division is, the better the community division effect is, the fast underfolding algorithm is an algorithm for community division based on the modularity, the fast underfolding algorithm is an iterative algorithm, and the main objective is to continuously divide communities so that the modularity (modulation) of the whole divided network is continuously increased. The modularity refers to the proportion of edges connecting vertices inside the regional structure in the network, minus the expected value of the proportion of any two data nodes connected under the same regional structure.
It is understood that the modularity is defined as:
Figure BDA0002767686060000151
wherein, Q represents the modularity,
Figure BDA0002767686060000152
representing all weights in the target network, Ai,jRepresents the weight of the connecting edge between the data node i and the data node j, ki=∑j Ai,jRepresenting the weight of the connecting edge with the data node i, ciIndicates the region to which the vertex is assigned, δ (c)i,cj) And the judgment is carried out to judge whether the data node i and the data node j are divided into the same area, if so, the data node i and the data node j are 1, otherwise, the data node j is 0.
For convenience of understanding, please refer to fig. 9, where fig. 9 is a schematic diagram of partitioning regions based on a fast unfolding algorithm in the embodiment of the present application, and as shown in the figure, the fast unfolding algorithm includes two stages, where the first stage is modularity optimization, and mainly partitions each data node into a region where data nodes adjacent to the data node are located, so that a value of the modularity is continuously increased. The second stage is community aggregation, which is mainly to aggregate the regions divided in the first step into one point, i.e. to reconstruct the network according to the region structure generated in the previous step. Repeating the above processes until the algorithm termination condition is met, and obtaining N areas. The algorithm termination condition may be, for example, a preset partition threshold, and when the number of partitions reaches the partition threshold, the algorithm termination condition is satisfied, and for example, the algorithm termination condition may also be until the structure in the network is not changed any more.
Specifically, the flow of the fast unfolding algorithm includes the following steps:
in step 1, initialization, each data node is divided into different regions.
In step 2, for each data node, each data node is tried to be divided into areas where the data nodes adjacent to the data node are located, the modularity at the moment is calculated, and the difference value Δ Q of the modularity before and after division is judged. And judging whether the delta Q is a positive number, if so, accepting the current division, and if not, giving up the current division.
In step 3, the above process is repeated until the modularity can no longer be increased.
In step 4, a new map is constructed, representing each area scribed in step 3 for each sister in the new map, and steps 2 and 3 are continued until the structure of the area is no longer changed.
Still further, in the embodiment of the present application, a method for dividing a region by a fast unfolding algorithm is provided, and by the above method, region division can be achieved without supervision, the whole process is easy to implement, the algorithm speed is high, and the modularized gain is easy to calculate. The modular resolution limit problem can also be circumvented because the first stage of the algorithm is designed to shift a single data node from one region to another, and therefore, the algorithm has an inherent multi-level nature.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in the embodiment of the present application, the dividing processing is performed on the data nodes in the target network to obtain N areas, and specifically includes the following steps:
acquiring a first area and a second area from a target network;
acquiring a first gain value according to the first area and the second area, wherein the first gain value is the sum of the number of edges in the first area and the number of edges in the second area and the difference value of the number of edges between the first area and the second area;
acquiring a first data node in a first area and a second data node in a second area;
adding the second data node into the first area to obtain an updated first area, and adding the first data node into the second area to obtain an updated second area;
acquiring a second gain value according to the updated first region and the updated second region, wherein the second gain value is the sum of the updated number of edges in the first region and the updated number of edges in the second region and the difference between the updated number of edges in the first region and the updated number of edges in the second region;
determining a target gain value according to the first gain value and the second gain value;
if the target gain value is the maximum value of the P gain values, determining that the first data node belongs to the updated second region and the second data node belongs to the updated first region, wherein the P gain values comprise the gain values of every two data nodes between the first region and the second region, and P is an integer greater than or equal to 1;
and obtaining N areas until the algorithm termination condition is met.
In this embodiment, a method for partitioning regions by using the Kernighan-Lin algorithm is introduced. The Kernighan-Lin algorithm can divide M data nodes into two regions with specified scales, for any node pair (i, j) consisting of data nodes i and data nodes j belonging to different regions, the positions between the data nodes i and the data nodes j are exchanged, and then the gain value between the two regions before and after the exchange is calculated. And finding the maximum gain value in all node pairs, and exchanging the node pair corresponding to the maximum gain value. And repeating the process until N areas are obtained when the algorithm termination condition is met.
Specifically, for convenience of introduction, please refer to fig. 10, where fig. 10 is a schematic diagram of partitioning regions based on Kernighan-Lin algorithm in the embodiment of the present application, as shown in (a) of fig. 10, a data node 1, a data node 2, a data node 6, and a data node 7 belong to a first region, and a data node 3, a data node 4, a data node 5, and a data node 8 belong to a second region. Based on this, the sum of the number of sides in the first region and the number of sides in the second region is calculated, and taking the graph (a) in fig. 10 as an example, the number of sides in the first region is 1, the number of sides in the second region is 4, and the sum of the number of sides in the first region and the number of sides in the second region is 5. The sum of the number of continuous edges between the first region and the second region is calculated, and in the example of the graph (a) in fig. 10, the number of continuous edges between the first region and the second region is 7, and therefore, the first gain value is 5-7 — 2.
Then, a first data node (e.g., data node No. 7) is obtained from the first area, a second data node (e.g., data node No. 4) is obtained from the second area, and the first data node and the second data node are interacted to obtain an updated first area and an updated second area. As shown in fig. 10 (B), the data node 1, the data node 2, the data node 4, and the data node 6 belong to the updated first area, and the data node 3, the data node 5, the data node 7, and the data node 8 belong to the updated second area. Based on this, the sum of the updated number of edges in the first region and the updated number of edges in the second region is calculated, and in the example of the graph (B) in fig. 10, the updated number of edges in the first region is 4, the updated number of edges in the second region is 4, and the updated sum of the updated number of edges in the first region and the updated number of edges in the second region is 8. The sum of the number of continuous edges between the updated first region and the updated second region is calculated, and as shown in fig. 10 (B), the number of continuous edges between the updated first region and the updated second region is 4, and therefore the second gain value is 8-4 to 4.
After subtracting the first gain value from the second gain value, the absolute value is taken to obtain the target gain value, and in combination with the above example, the target gain value is obtained to obtain | -2-4| ═ 6. And if the target gain value is the maximum value of the P gain values, the first data node is used as the data node in the second area, and the second data node is used as the data node in the first area. And repeating the steps until the algorithm termination condition is met, and obtaining N areas.
For example, the algorithm terminating condition may be a preset swap threshold, and when the number of swaps reaches the iteration threshold, the algorithm terminating condition is satisfied, for example, the algorithm terminating condition may also be an evaluation parameter for calculating each region, and when the evaluation parameter is greater than the parameter threshold, the algorithm terminating condition is satisfied, and the evaluation parameter is calculated in the following manner:
Figure BDA0002767686060000171
wherein U represents an evaluation parameter, eiiRepresenting the ratio of all the number of connecting edges connecting each data node in a certain area, aiIndicating the occupation ratio of the edges connected by the data nodes in the ith area in all the connected edge numbers.
It should be noted that the evaluation parameter may also be calculated in other manners, which is only an illustration here and should not be construed as a limitation to the present application.
Still further, in the embodiment of the present application, a method for dividing regions by using the Kernighan-Lin algorithm is provided, and by using the above method, each pair of data nodes in a target network is considered, so that analysis is performed, the analysis difficulty is low, and the data nodes can be exchanged, so that a more accurate region division result is obtained.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in the embodiment of the present application, the dividing processing is performed on the data nodes in the target network to obtain N areas, and specifically includes the following steps:
determining K edge betweenness according to M data nodes and K connecting edges in the target network, wherein the edge betweenness in the K edge betweenness and the connecting edges in the K connecting edges have a corresponding relation;
selecting a target edge betweenness from the K edge betweenness, wherein the target edge betweenness is the maximum value of the K edge betweenness;
deleting the connecting edges corresponding to the intermediate number of the target edge;
and obtaining N areas until the algorithm termination condition is met.
In this embodiment, a method of dividing a region based on the GN algorithm is described. The GN algorithm is a community discovery algorithm, belongs to a split hierarchical clustering algorithm, and has the basic idea that the edge with the maximum edge Betweenness (Betwenness) relative to all the source nodes in the network is continuously deleted, then the edge Betweenness relative to all the source nodes of the rest edges in the network is recalculated, and the process is repeated until the termination condition of the algorithm is met.
Specifically, for convenience of introduction, please refer to fig. 11, fig. 11 is a schematic diagram of the partition of the area based on the GN algorithm in the embodiment of the present application, and as shown in (a) of fig. 11, it is assumed that the target network includes 7 data nodes and has 6 connected edges. The edge betweenness of each continuous edge can be calculated, for example, the edge betweenness between the data node No. 3 and the node No. 4 is 12, that is, there are 12 pairs of data nodes passing through the continuous edge between the data node No. 3 and the node No. 4. Based on this, the edge betweenness shown in fig. 11 (B) is obtained, where the edge betweenness between the data node No. 1 and the node No. 3 is 6, the edge betweenness between the data node No. 2 and the node No. 3 is 6, the edge betweenness between the data node No. 3 and the node No. 4 is 12, the edge betweenness between the data node No. 4 and the node No. 5 is 12, the edge betweenness between the data node No. 5 and the node No. 6 is 6, and the edge betweenness between the data node No. 5 and the node No. 7 is 6.
Therefore, the edge betweenness between the data node No. 3 and the node No. 4 and the edge betweenness between the data node No. 4 and the node No. 5 are determined to be the maximum values, then one edge betweenness is selected as a target edge betweenness, and then the connecting edge corresponding to the target edge betweenness is deleted, so that the two areas shown in the graph (C) in FIG. 11 are obtained. Similarly, the edge betweenness between the data nodes in each region is continuously calculated, and then the continuous edges corresponding to the maximum edge betweenness are continuously deleted, so that three regions shown in (D) of fig. 11 are obtained. And repeating the steps until the algorithm termination condition is met, and obtaining N areas.
The algorithm termination condition may be, for example, a preset iteration threshold, and when the number of divisions reaches the iteration threshold, the algorithm termination condition is satisfied, and for example, the algorithm termination condition may also be that an evaluation parameter of each region is calculated, and when the evaluation parameter is greater than the parameter threshold, the algorithm termination condition is satisfied.
Still further, in the embodiment of the present application, a method for partitioning a region based on a GN algorithm is provided, by which a global situation of a target network can be considered, the partitioned region has higher accuracy, and an algorithm termination condition can be defined in consideration of an end point of region partitioning, and once the algorithm termination condition is reached, the region is stopped to be continuously partitioned, so that while a region partitioning effect is considered, processing efficiency can be improved.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in the embodiment of the present application, the determining at least one service domain according to the N regions specifically includes the following steps:
obtaining Q data nodes included in the region to be identified from the N regions, wherein Q is an integer greater than or equal to 1;
and determining a service domain corresponding to the region to be identified according to Q data tables corresponding to the Q data nodes, wherein the data nodes in the Q data nodes have a corresponding relation with the data tables in the Q data tables.
In this embodiment, a method for interpreting the divided regions is described. As can be seen from the foregoing embodiment, the target network is divided into N regions, and then the region to which each data node belongs is obtained, for each region, it is necessary to determine whether the region meets a service domain determination condition, and if the condition is met, a corresponding service domain is obtained.
Specifically, taking the region to be identified as an example, the region to be identified is one of the N regions. And assuming that the area to be identified comprises Q data nodes, obtaining Q data tables, and determining the type of the area to be identified by combining service information and a manual sampling judgment form. For example, Q is 100, and 70 data tables of the Q data tables belong to the data table of the "red packet service", that is, the "red packet service" accounts for 70%. And if the service ratio is greater than or equal to the service ratio threshold, determining that the area to be identified meets the service domain determination condition, otherwise, not meeting the service domain determination condition.
For ease of description, please refer to table 3, where table 3 is an illustration of the relationship between the regions and the service domains.
TABLE 3
Region(s) Number of data nodes Type of service Service proportion Name of service Domain
Region A 100 Red packet service 70% Red packet service domain
Region B 50 Network car booking service 40% Is free of
Region C 60 Transfer service 75% Transfer service domain
Region D
150 Electric commerce 80% E-business domain
Taking the traffic ratio threshold as 60% as an example, in practical applications, the traffic ratio threshold may be other ratios, and is not limited herein. As can be seen from table 3, the traffic ratios of the data tables in the area a, the area C, and the area D are all greater than the traffic ratio threshold, and therefore, the traffic domain can be interpreted as the traffic type corresponding to the majority of the data tables. Based on the area B, the data table occupation ratio of the network car booking service is smaller than the service occupation ratio threshold value, so that the area does not belong to any service domain, namely, the area is interpreted as a non-service domain.
Still further, in the embodiment of the present application, a manner of interpreting the divided regions is provided, and by the manner, each divided region can be further interpreted, so as to determine the service domain to which the region belongs, and thus, a more reasonable service domain allocation result can be obtained.
Optionally, on the basis of the embodiment corresponding to fig. 3, in another optional embodiment provided in this application embodiment, when the operation for the target viewing interface is obtained, at least one data table corresponding to the target service domain is displayed through an interface of the terminal device, which specifically includes the following steps:
displaying the business name and the viewing interface corresponding to each business domain;
when the operation aiming at the target viewing interface is detected, at least one data table corresponding to the target service domain is displayed through the interface of the terminal equipment;
alternatively, the first and second electrodes may be,
when the operation aiming at the target viewing interface is acquired, at least one data table corresponding to the target service domain is displayed through the interface of the terminal equipment, and the method specifically comprises the following steps:
when the terminal equipment detects the operation aiming at the target viewing interface, receiving a viewing instruction sent by the terminal equipment;
and sending at least one data table corresponding to the target service domain to the terminal equipment according to the viewing instruction so that the terminal equipment displays the at least one data table corresponding to the target service domain.
In this embodiment, a method for displaying a business domain for related people is introduced. As can be seen from the foregoing embodiments, the data sheet presentation apparatus may be deployed in a server, or may be deployed in a terminal device, and based on this, the following description will be made with reference to a case where the data sheet presentation apparatus is deployed in different devices.
Firstly, a data table display device is deployed on terminal equipment;
after the terminal device obtains the service domains, the service names and the corresponding viewing interfaces of the service domains can be directly displayed. For convenience of understanding, referring to fig. 12, fig. 12 is an interface schematic diagram illustrating service domains in an embodiment of the present application, and as shown in the figure, different service domains, such as "red envelope service", "transfer service", "network appointment service", and "e-commerce service", are displayed on an interface of a data platform. When a user needs to check the data table under the red packet service, the user can click a check interface corresponding to the red packet service, and the check interface is a target check interface.
In response to the operation of the user on the target viewing interface, the terminal device may jump to the interface shown in fig. 13, please refer to fig. 13, where fig. 13 is an interface diagram showing data tables under the target service domain in the embodiment of the present application, as shown in the figure, the target service domain is taken as a "red packet service", and the target service domain includes 13 data tables, that is, identifiers of the 13 data tables are shown. If the user desires to view one or more of the data tables, the user may directly click on the options for the data tables, for example, select to view "data table 0156", "data table 3594", "data table 1072", "data table 4235", "data table 6569", and "data table 7711". After the selection is finished, the specific content in the data table can be displayed after clicking the 'view' button.
Secondly, the data table display device is deployed in the server;
after the server acquires the service domains, the server sends the service names corresponding to the service domains to the terminal equipment, so that the terminal equipment displays the service names of the service domains and the corresponding viewing interfaces. When the terminal device detects an operation aiming at the target viewing interface, a viewing instruction can be sent to the data table display device, and the viewing instruction carries the identifier of the target service domain, so that the server sends at least one data table corresponding to the target service domain to the terminal device according to the viewing instruction, and the terminal device displays the at least one data table corresponding to the target service domain.
It should be noted that the interface for the terminal device to display the service name and view the interface is similar to the interface shown in fig. 12, and the interface for the terminal device to display at least one data table in the target service domain is similar to the interface shown in fig. 13, so that the details are not repeated here.
Secondly, in the embodiment of the application, a mode for displaying the business domain facing the related personnel is provided, and through the mode, the divided business domains can be visually displayed, so that the related personnel can find the data table under a certain business domain more intuitively, the data table under a certain business domain does not need to be manually searched from a large number of data tables, the data searching efficiency is improved, and the flexibility and the operability of the scheme are improved.
Referring to fig. 14, fig. 14 is a schematic view of an embodiment of a data sheet display apparatus in an embodiment of the present application, and the data sheet display apparatus 20 includes:
an obtaining module 201, configured to obtain M pieces of metadata from M data tables, where a data table in the M data tables has a corresponding relationship with metadata in the M pieces of metadata, and M is an integer greater than or equal to 2;
a determining module 202, configured to determine, according to the M pieces of metadata, an association relationship between the M pieces of data tables;
the building module 203 is configured to build a target network according to an association relationship between the M data tables, where the target network includes M data nodes, a data node in the M data nodes has a corresponding relationship with a data table in the M data tables, and each data node is used to store one data table;
a dividing module 204, configured to perform area division processing on a target network to obtain at least one service domain, where each service domain includes at least one data table;
the displaying module 205 is configured to display, through an interface of the terminal device, at least one data table corresponding to a target service domain when an operation for the target viewing interface is obtained, where the target viewing interface is a viewing interface corresponding to the target service domain in the at least one service domain.
In the embodiment of the application, a data table display device is provided, and by adopting the device, the relation between data tables can be determined by using the metadata corresponding to each data table, a target network is formed based on the relation between the data tables, and then the target network can be subjected to region division by adopting a community division algorithm, so that at least one region is obtained, each divided region can be regarded as a service domain, and each data node in the region is each data table included in the service domain.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data table displaying apparatus 20 provided in the embodiment of the present application, the M data tables at least include a first data table and a second data table;
an obtaining module 201, specifically configured to obtain first metadata derived from a first data table;
obtaining second metadata from a second data table;
the determining module 202 is specifically configured to determine a target association relationship between the first data table and the second data table according to the first metadata and the second metadata, where the target association relationship includes at least one of an edge connecting direction and an edge connecting weight between a first data node and the second data node, where the first data node is used to store the first data table, and the second data node is used to store the second data table.
In the embodiment of the application, a data table display device is provided, and by adopting the device, for two data tables, the association relationship between the two data tables can be determined by using the corresponding metadata, so that a more accurate association relationship is obtained, and the feasibility of a scheme is improved.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a determining module 202, configured to determine, according to technical metadata included in the first metadata and technical metadata included in the second metadata, a direction of a connecting edge between the first data node and the second data node;
and determining the connection edge weight between the first data node and the second data node according to the service metadata included by the first metadata and the service metadata included by the second metadata.
In the embodiment of the application, a data table display device is provided, and by adopting the device, the incidence relation between two data nodes can be constructed from different dimensions (namely the edge connecting direction and the edge connecting weight) by combining technical metadata and service metadata, so that the incidence relation between the data tables is determined, and the feasibility and operability of the scheme are improved.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a determining module 202, configured to specifically obtain a data blood margin corresponding to the first data table according to technical metadata included in the first metadata;
acquiring a data blood margin corresponding to the second data table according to the technical metadata included in the second metadata;
determining an upstream data table from the first data table and the second data table according to the data blood margin corresponding to the first data table and the data blood margin corresponding to the second data table, wherein the data blood margin belongs to the technical metadata;
if the upstream data table is the first data table, constructing a connecting edge from the first data node to the second data node;
and if the upstream data table is the second data table, constructing a connecting edge from the second data node to the first data node.
In the embodiment of the application, the data table display device is provided, and by adopting the device, all data tables can be characterized into the relation between points and edges based on the incidence relation among the data tables characterized by the data bloodiness, and then the community division and clustering are carried out on the data assets through a data mining method introduced into a complex network, so that the service domains corresponding to the data assets are positioned, the refined management of the data assets is carried out, and the effective operation and precipitation of the data assets are promoted.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
the determining module 202 is specifically configured to obtain a service name corresponding to the first data table according to service metadata included in the first metadata;
acquiring a service name corresponding to the second data table according to service metadata included in the second metadata;
determining the association degree between the first data table and the second data table according to the business name corresponding to the first data table and the business name corresponding to the second data table;
and determining the connection edge weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
In the embodiment of the application, a data table display device is provided, and by using the device, the association between data tables can be determined through business names, and in general, the higher the association degree of the business names is, the tighter the relationship between the two data tables is, so that the built connection weight is more accurate.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a determining module 202, configured to specifically obtain, according to service metadata included in the first metadata, a service description corresponding to the first data table;
acquiring a service description corresponding to the second data table according to service metadata included in the second metadata;
acquiring the association degree between the first data table and the second data table through a semantic matching model based on the service description corresponding to the first data table and the service description corresponding to the second data table;
and determining the connection edge weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
In the embodiment of the application, a data table display device is provided, and by adopting the device, since the service description often covers information related to the service, the association between the data tables can be determined through the service description, in general, the higher the association degree of the service name is, the tighter the relationship between the two data tables is, and the more accurate the edge connection weight constructed by the method is.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a dividing module 204, configured to perform division processing on data nodes in a target network to obtain N regions, where N is an integer greater than or equal to 1 and less than or equal to M;
at least one service domain is determined according to the N regions.
In the embodiment of the application, a data table display device is provided, and by adopting the device, the automatic division of the areas is realized by utilizing the algorithm, and different areas do not need to be manually divided, so that the flexibility and the convenience of the scheme are improved.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
the partitioning module 204 is specifically configured to obtain data nodes to be partitioned from a target network;
acquiring a first data node and a second data node according to the data node to be divided, wherein the first data node and the second data node are both data nodes adjacent to the data node to be divided;
determining a first modularity according to the data node to be divided and the first data node;
determining a second modularity according to the data node to be divided and the second data node;
if the first modularity and the second modularity are both greater than 0 and the first modularity is greater than the second modularity, determining that the data node to be divided and the first data node belong to the same region of the N regions;
if the first modularity and the second modularity are both greater than 0 and the first modularity is smaller than the second modularity, determining that the data node to be divided and the second data node belong to the same region of the N regions;
and obtaining N areas until the algorithm termination condition is met.
In the embodiment of the application, the data table display device is provided, and by adopting the device, the region division can be realized under the unsupervised condition, the whole process is easy to realize, the algorithm speed is high, and the modularized gain is easy to calculate. The modular resolution limit problem can also be circumvented because the first stage of the algorithm is designed to shift a single data node from one region to another, and therefore, the algorithm has an inherent multi-level nature.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a dividing module 204, specifically configured to obtain a first area and a second area from a target network;
acquiring a first gain value according to the first area and the second area, wherein the first gain value is the sum of the number of edges in the first area and the number of edges in the second area and the difference value of the number of edges between the first area and the second area;
acquiring a first data node in a first area and a second data node in a second area;
adding the second data node into the first area to obtain an updated first area, and adding the first data node into the second area to obtain an updated second area;
acquiring a second gain value according to the updated first region and the updated second region, wherein the second gain value is the sum of the updated number of edges in the first region and the updated number of edges in the second region and the difference between the updated number of edges in the first region and the updated number of edges in the second region;
determining a target gain value according to the first gain value and the second gain value;
if the target gain value is the maximum value of the P gain values, determining that the first data node belongs to the updated second region and the second data node belongs to the updated first region, wherein the P gain values comprise the gain values of every two data nodes between the first region and the second region, and P is an integer greater than or equal to 1;
and obtaining N areas until the algorithm termination condition is met.
In the embodiment of the application, a data table display device is provided, and by adopting the device, each pair of data nodes in a target network is considered, so that analysis is performed, the data nodes can be exchanged, and a more accurate region division result is obtained.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
the dividing module 204 is specifically configured to determine K edge betweenness according to the M data nodes and the K connected edges in the target network, where an edge betweenness of the K edge betweenness and a connected edge of the K connected edges have a corresponding relationship;
selecting a target edge betweenness from the K edge betweenness, wherein the target edge betweenness is the maximum value of the K edge betweenness;
deleting the connecting edges corresponding to the intermediate number of the target edge;
and obtaining N areas until the algorithm termination condition is met.
In the embodiment of the application, a data table display device is provided, and by adopting the device, the global situation of a target network can be considered, the divided areas have higher accuracy, an algorithm termination condition can be defined by considering the end point of area division, and once the algorithm termination condition is reached, the area division is stopped to continue, so that the area division effect is considered, and the processing efficiency can be improved.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a dividing module 204, configured to specifically obtain Q data nodes included in the to-be-identified region from the N regions, where Q is an integer greater than or equal to 1;
and determining a service domain corresponding to the region to be identified according to Q data tables corresponding to the Q data nodes, wherein the data nodes in the Q data nodes have a corresponding relation with the data tables in the Q data tables.
In the embodiment of the application, a data table display device is provided, and by adopting the device, each divided area can be further interpreted, so that a service domain to which the area belongs is determined, and therefore, a more reasonable service domain distribution result can be obtained.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
a presentation module 205, configured to present a service name and a viewing interface corresponding to each service domain;
and when the operation aiming at the target viewing interface is detected, displaying at least one data table corresponding to the target service domain through the interface of the terminal equipment.
In the embodiment of the application, the data table display device is provided, and by adopting the device, the divided service domains can be visually displayed, so that related personnel can find the data table under a certain service domain more intuitively, the data table under a certain service domain does not need to be manually searched from a large number of data tables, the data searching efficiency is improved, and the flexibility and operability of the scheme are improved.
Optionally, on the basis of the embodiment corresponding to fig. 14, in another embodiment of the data sheet showing device 20 provided in the embodiment of the present application,
the presentation module 205 is specifically configured to receive a viewing instruction sent by the terminal device when the terminal device detects an operation directed to the target viewing interface;
and sending at least one data table corresponding to the target service domain to the terminal equipment according to the viewing instruction so that the terminal equipment displays the at least one data table corresponding to the target service domain.
In the embodiment of the application, the data table display device is provided, and by adopting the device, the divided service domains can be visually displayed, so that related personnel can find the data table under a certain service domain more intuitively, the data table under a certain service domain does not need to be manually searched from a large number of data tables, the data searching efficiency is improved, and the flexibility and operability of the scheme are improved.
Fig. 15 is a schematic diagram of a server 300 according to an embodiment of the present application, where the server 300 may have a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing applications 342 or data 344. Memory 332 and storage media 330 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 330 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, the central processor 322 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the server 300.
The Server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341, such as a Windows ServerTM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTMAnd so on.
The steps performed by the server in the above embodiment may be based on the server structure shown in fig. 15.
As shown in fig. 16, for convenience of description, only the portions related to the embodiments of the present application are shown, and details of the specific techniques are not disclosed, please refer to the method portion of the embodiments of the present application. The terminal device may be any terminal device including a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), a Point of Sales (POS), a vehicle-mounted computer, and the like, taking the terminal device as the mobile phone as an example:
fig. 16 is a block diagram illustrating a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Referring to fig. 16, the cellular phone includes: radio Frequency (RF) circuit 410, memory 420, input unit 430, display unit 440, sensor 450, audio circuit 460, wireless fidelity (WiFi) module 470, processor 480, and power supply 490. Those skilled in the art will appreciate that the handset configuration shown in fig. 16 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
The following describes each component of the mobile phone in detail with reference to fig. 16:
the RF circuit 410 may be used for receiving and transmitting signals during information transmission and reception or during a call, and in particular, receives downlink information of a base station and then processes the received downlink information to the processor 480; in addition, the data for designing uplink is transmitted to the base station. In general, the RF circuit 410 includes, but is not limited to, an antenna, at least one Amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry 410 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), and the like.
The memory 420 may be used to store software programs and modules, and the processor 480 executes various functional applications and data processing of the mobile phone by operating the software programs and modules stored in the memory 420. The memory 420 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The input unit 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the cellular phone. Specifically, the input unit 430 may include a touch panel 431 and other input devices 432. The touch panel 431, also called a touch screen, may collect touch operations of a user on or near the touch panel 431 (e.g., operations of the user on or near the touch panel 431 using any suitable object or accessory such as a finger or a stylus) and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 431 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 480, and receives and executes commands sent from the processor 480. In addition, the touch panel 431 may be implemented in various types, such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 430 may include other input devices 432 in addition to the touch panel 431. In particular, other input devices 432 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
The display unit 440 may be used to display information input by the user or information provided to the user and various menus of the cellular phone. The Display unit 440 may include a Display panel 441, and optionally, the Display panel 441 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 431 may cover the display panel 441, and when the touch panel 431 detects a touch operation on or near the touch panel 431, the touch panel is transmitted to the processor 480 to determine the type of the touch event, and then the processor 480 provides a corresponding visual output on the display panel 441 according to the type of the touch event. Although the touch panel 431 and the display panel 441 are shown in fig. 16 as two separate components to implement the input and output functions of the mobile phone, in some embodiments, the touch panel 431 and the display panel 441 may be integrated to implement the input and output functions of the mobile phone.
The handset may also include at least one sensor 450, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that adjusts the brightness of the display panel 441 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 441 and/or the backlight when the mobile phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the posture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, further description is omitted here.
Audio circuit 460, speaker 461, microphone 462 may provide an audio interface between the user and the cell phone. The audio circuit 460 may transmit the electrical signal converted from the received audio data to the speaker 461, and convert the electrical signal into a sound signal for output by the speaker 461; on the other hand, the microphone 462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 460 and converted into audio data, which is then processed by the audio data output processor 480 and then transmitted to, for example, another cellular phone via the RF circuit 410, or output to the memory 420 for further processing.
WiFi belongs to short-distance wireless transmission technology, and the mobile phone can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 470, and provides wireless broadband Internet access for the user. Although fig. 16 shows the WiFi module 470, it is understood that it does not belong to the essential constitution of the handset, and may be omitted entirely as needed within the scope not changing the essence of the invention.
The processor 480 is a control center of the mobile phone, connects various parts of the entire mobile phone by using various interfaces and lines, and performs various functions of the mobile phone and processes data by operating or executing software programs and/or modules stored in the memory 420 and calling data stored in the memory 420, thereby integrally monitoring the mobile phone. Optionally, processor 480 may include one or more processing units; optionally, the processor 480 may integrate an application processor and a modem processor, wherein the application processor mainly handles operating systems, user interfaces, application programs, and the like, and the modem processor mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 480.
The phone also includes a power supply 490 (e.g., a battery) for powering the various components, optionally, the power supply may be logically connected to the processor 480 through a power management system, so as to implement functions such as managing charging, discharging, and power consumption through the power management system.
Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which are not described herein.
The steps performed by the terminal device in the above-described embodiment may be based on the terminal device configuration shown in fig. 16.
Embodiments of the present application also provide a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the method described in the foregoing embodiments.
Embodiments of the present application also provide a computer program product including a program, which, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (15)

1. A data table display method based on a service domain is characterized by comprising the following steps:
acquiring M metadata derived from M data tables, wherein the data tables in the M data tables have corresponding relations with the metadata in the M metadata, and M is an integer greater than or equal to 2;
determining the incidence relation among the M data tables according to the M metadata;
constructing a target network according to the incidence relation among the M data tables, wherein the target network comprises M data nodes, the data nodes in the M data nodes and the data tables in the M data tables have a corresponding relation, and each data node is used for storing one data table;
performing area division processing on the target network to obtain at least one service domain, wherein each service domain comprises at least one data table;
when the operation aiming at the target viewing interface is obtained, at least one data table corresponding to the target service domain is displayed through an interface of the terminal equipment, wherein the target viewing interface is a viewing interface corresponding to the target service domain in the at least one service domain.
2. The method for displaying data table of claim 1, wherein the M data tables at least include a first data table and a second data table;
the obtaining M metadata derived from M data tables includes:
obtaining first metadata derived from the first data table;
obtaining second metadata derived from the second data table;
determining an association relationship between the M data tables according to the M metadata includes:
determining a target association relationship between the first data table and the second data table according to the first metadata and the second metadata, wherein the target association relationship comprises at least one of an edge connecting direction and an edge connecting weight between a first data node and a second data node, the first data node is used for storing the first data table, and the second data node is used for storing the second data table.
3. The method for displaying a data table according to claim 2, wherein the determining the target association relationship between the first data table and the second data table according to the first metadata and the second metadata comprises:
determining a connecting edge direction between the first data node and the second data node according to the technical metadata included in the first metadata and the technical metadata included in the second metadata;
and determining the connection edge weight between the first data node and the second data node according to the service metadata included by the first metadata and the service metadata included by the second metadata.
4. The method for displaying a data table according to claim 3, wherein the determining the direction of the connecting edge between the first data node and the second data node according to the technical metadata included in the first metadata and the technical metadata included in the second metadata comprises:
acquiring a data blood margin corresponding to the first data table according to the technical metadata included in the first metadata;
acquiring a data blood margin corresponding to the second data table according to the technical metadata included in the second metadata;
determining an upstream data table from the first data table and the second data table according to the data blood margin corresponding to the first data table and the data blood margin corresponding to the second data table, wherein the data blood margin belongs to the technical metadata;
if the upstream data table is the first data table, constructing a connecting edge from the first data node to the second data node;
and if the upstream data table is the second data table, constructing a connecting edge from the second data node to the first data node.
5. The method for displaying a data table according to claim 3, wherein the determining the edge connecting weight between the first data node and the second data node according to the service metadata included in the first metadata and the service metadata included in the second metadata comprises:
acquiring a service name corresponding to the first data table according to service metadata included in the first metadata;
acquiring a service name corresponding to the second data table according to service metadata included in the second metadata;
determining the association degree between the first data table and the second data table according to the business name corresponding to the first data table and the business name corresponding to the second data table;
and determining the edge connecting weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
6. The method for displaying a data table according to claim 3, wherein the determining the edge connecting weight between the first data node and the second data node according to the service metadata included in the first metadata and the service metadata included in the second metadata comprises:
acquiring a service description corresponding to the first data table according to service metadata included in the first metadata;
acquiring a service description corresponding to the second data table according to service metadata included in the second metadata;
acquiring the association degree between the first data table and the second data table through a semantic matching model based on the service description corresponding to the first data table and the service description corresponding to the second data table;
and determining the edge connecting weight between the first data node and the second data node according to the association degree between the first data table and the second data table.
7. The method for displaying a data table according to any one of claims 1 to 6, wherein the performing the area division processing on the target network to obtain at least one service domain comprises:
dividing data nodes in the target network to obtain N areas, wherein N is an integer which is greater than or equal to 1 and less than or equal to M;
and determining the at least one service domain according to the N areas.
8. The method for displaying a data table according to claim 7, wherein the dividing the data nodes in the target network to obtain N regions comprises:
acquiring data nodes to be divided from the target network;
acquiring a first data node and a second data node according to the data node to be divided, wherein the first data node and the second data node are both data nodes adjacent to the data node to be divided;
determining a first modularity according to the data node to be divided and the first data node;
determining a second modularity according to the data node to be divided and the second data node;
if the first modularity and the second modularity are both greater than 0 and the first modularity is greater than the second modularity, determining that the data node to be divided and the first data node belong to the same one of the N regions;
if the first modularity and the second modularity are both greater than 0 and the first modularity is less than the second modularity, determining that the data node to be divided and the second data node belong to the same one of the N regions;
and acquiring the N areas until an algorithm termination condition is met.
9. The method for displaying a data table according to claim 7, wherein the dividing the data nodes in the target network to obtain N regions comprises:
acquiring a first area and a second area from the target network;
acquiring a first gain value according to the first region and the second region, wherein the first gain value is a difference between the sum of the number of edges in the first region and the number of edges in the second region and the number of edges between the first region and the second region;
acquiring a first data node in the first area and a second data node in the second area;
adding the second data node to the first area to obtain an updated first area, and adding the first data node to the second area to obtain an updated second area;
acquiring a second gain value according to the updated first region and the updated second region, wherein the second gain value is the sum of the updated number of edges in the first region and the updated number of edges in the second region and the difference between the updated number of edges in the first region and the updated number of edges in the second region;
determining a target gain value according to the first gain value and the second gain value;
if the target gain value is the maximum value of P gain values, determining that the first data node belongs to the updated second region and the second data node belongs to the updated first region, wherein the P gain values comprise the gain values of every two data nodes between the first region and the second region, and P is an integer greater than or equal to 1;
and acquiring the N areas until an algorithm termination condition is met.
10. The method for displaying a data table according to claim 7, wherein the dividing the data nodes in the target network to obtain N regions comprises:
determining K edge betweenness numbers according to the M data nodes and K connecting edges in the target network, wherein the edge betweenness numbers in the K edge betweenness numbers have a corresponding relation with the connecting edges in the K connecting edges;
selecting a target edge betweenness from the K edge betweenness, wherein the target edge betweenness is the maximum value of the K edge betweenness;
deleting the connecting edges corresponding to the intermediate number of the target edges;
and acquiring the N areas until an algorithm termination condition is met.
11. The method of claim 7, wherein said determining the at least one service domain according to the N regions comprises:
obtaining Q data nodes included in the region to be identified from the N regions, wherein Q is an integer greater than or equal to 1;
and determining a service domain corresponding to the region to be identified according to Q data tables corresponding to the Q data nodes, wherein the data nodes in the Q data nodes have a corresponding relation with the data tables in the Q data tables.
12. The method according to claim 1, wherein when the operation for the target viewing interface is obtained, displaying at least one data table corresponding to the target service domain through an interface of the terminal device includes:
displaying the business name and the viewing interface corresponding to each business domain;
when the operation aiming at the target viewing interface is detected, displaying at least one data table corresponding to the target service domain through the interface of the terminal equipment;
alternatively, the first and second electrodes may be,
when the operation aiming at the target viewing interface is obtained, displaying at least one data table corresponding to the target service domain through the interface of the terminal equipment comprises the following steps:
when the terminal device detects the operation aiming at the target viewing interface, receiving a viewing instruction sent by the terminal device;
and sending at least one data table corresponding to the target service domain to the terminal equipment according to the viewing instruction so that the terminal equipment displays the at least one data table corresponding to the target service domain.
13. A spreadsheet display device, comprising:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring M metadata from M data tables, the data tables in the M data tables have corresponding relations with the metadata in the M metadata, and M is an integer greater than or equal to 2;
a determining module, configured to determine, according to the M pieces of metadata, an association relationship between the M pieces of data tables;
the building module is used for building a target network according to the incidence relation among the M data tables, wherein the target network comprises M data nodes, the data nodes in the M data nodes have a corresponding relation with the data tables in the M data tables, and each data node is used for storing one data table;
the division module is used for carrying out regional division processing on the target network to obtain at least one service domain, wherein each service domain comprises at least one data table;
and the display module is used for displaying at least one data table corresponding to the target service domain through an interface of the terminal equipment when the operation aiming at the target viewing interface is obtained, wherein the target viewing interface is a viewing interface corresponding to the target service domain in the at least one service domain.
14. A computer device, comprising: a memory, a processor, and a bus system;
wherein the memory is used for storing programs;
the processor is used for executing the program in the memory, and the processor is used for executing the data table showing method of any one of claims 1 to 12 according to the instructions in the program codes;
the bus system is used for connecting the memory and the processor so as to enable the memory and the processor to communicate.
15. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the data table presentation method of any one of claims 1 to 12.
CN202011238788.1A 2020-11-09 2020-11-09 Data table display method, device, equipment and medium based on service domain Pending CN113392150A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011238788.1A CN113392150A (en) 2020-11-09 2020-11-09 Data table display method, device, equipment and medium based on service domain

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011238788.1A CN113392150A (en) 2020-11-09 2020-11-09 Data table display method, device, equipment and medium based on service domain

Publications (1)

Publication Number Publication Date
CN113392150A true CN113392150A (en) 2021-09-14

Family

ID=77616533

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011238788.1A Pending CN113392150A (en) 2020-11-09 2020-11-09 Data table display method, device, equipment and medium based on service domain

Country Status (1)

Country Link
CN (1) CN113392150A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113918577A (en) * 2021-12-15 2022-01-11 北京新唐思创教育科技有限公司 Data table identification method and device, electronic equipment and storage medium
CN114521905A (en) * 2022-01-25 2022-05-24 中山大学 Electroencephalogram signal processing method and system based on synchronous connection characteristics
CN115185967A (en) * 2022-07-06 2022-10-14 北京字跳网络技术有限公司 Data processing method and device, electronic equipment and storage medium
WO2023246165A1 (en) * 2022-06-24 2023-12-28 华为云计算技术有限公司 Data asset display method and apparatus, and device and storage medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113918577A (en) * 2021-12-15 2022-01-11 北京新唐思创教育科技有限公司 Data table identification method and device, electronic equipment and storage medium
CN113918577B (en) * 2021-12-15 2022-03-11 北京新唐思创教育科技有限公司 Data table identification method and device, electronic equipment and storage medium
CN114521905A (en) * 2022-01-25 2022-05-24 中山大学 Electroencephalogram signal processing method and system based on synchronous connection characteristics
WO2023246165A1 (en) * 2022-06-24 2023-12-28 华为云计算技术有限公司 Data asset display method and apparatus, and device and storage medium
CN115185967A (en) * 2022-07-06 2022-10-14 北京字跳网络技术有限公司 Data processing method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN111310034B (en) Resource recommendation method and related equipment
CN109241431B (en) Resource recommendation method and device
CN109558951B (en) Method and device for detecting fraud account and storage medium thereof
CN113392150A (en) Data table display method, device, equipment and medium based on service domain
Kang et al. Cloudle: an ontology-enhanced cloud service search engine
WO2018040069A1 (en) Information recommendation system and method
CN111125269B (en) Data management method, blood relationship display method and related device
CN110019825B (en) Method and device for analyzing data semantics
CN111143697B (en) Content recommendation method and related device
CN110555172A (en) user relationship mining method and device, electronic equipment and storage medium
WO2021120875A1 (en) Search method and apparatus, terminal device and storage medium
CN111046237A (en) User behavior data processing method and device, electronic equipment and readable medium
CN113360580A (en) Abnormal event detection method, device, equipment and medium based on knowledge graph
WO2023217127A1 (en) Causation determination method and related device
US20170236226A1 (en) Computerized systems, processes, and user interfaces for globalized score for a set of real-estate assets
CN112818230A (en) Content recommendation method and device, electronic equipment and storage medium
CN112115710B (en) Industry information identification method and device
Subramanian et al. Systems dynamics-based modeling of data warehouse quality
KR102457359B1 (en) Marketing cost efficiency calculation method and electronic system for performing the method using neural networks
CN115392405A (en) Model training method, related device and storage medium
CN115619245A (en) Portrait construction and classification method and system based on data dimension reduction method
CN111475721B (en) Information pushing method, device, equipment and storage medium
CN115203194A (en) Metadata information generation method, related device, equipment and storage medium
CN111222918B (en) Keyword mining method and device, electronic equipment and storage medium
CN113722593A (en) Event data processing method and device, electronic equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40052242

Country of ref document: HK

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination