CN114385652A

CN114385652A - Data blood relationship construction method and system, electronic device and storage medium

Info

Publication number: CN114385652A
Application number: CN202111626866.XA
Authority: CN
Inventors: 蓝青; 刘高东; 徐忠; 田友丽
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-22

Abstract

The application discloses a data blood relationship construction method and system, electronic equipment and a storage medium, which are used for realizing construction of data blood relationship among multiple subsystems. The data blood relationship construction method provided by the embodiment of the application is applied to a service system comprising a plurality of subsystems, and comprises the following steps: acquiring metadata information of a data object contained in a service system; capturing metadata change information of each subsystem; obtaining a subsystem data blood relationship corresponding to each subsystem according to the metadata variable information; and obtaining the data blood relationship of the service system according to the subsystem data blood relationship of each subsystem included in the service system.

Description

Data blood relationship construction method and system, electronic device and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method and a system for constructing a data relationship, an electronic device, and a storage medium.

Background

With the rapid development of business innovation and a comprehensive process, the types of systems expand rapidly, the systems are mutually interwoven, and business systems become more and more complex. In addition, the business requirements in the internet era are constantly changing, and huge pressure and challenges are brought to system development, testing, operation and maintenance.

The system scale is enlarged along with the expansion of services, huge programs and software resources are formed, and even the system is divided into more subsystems and is developed and maintained by a plurality of development organizations. In the early stage, a core system framework oriented to a process design technology is adopted, the isolation and encapsulation of program modules are relatively poor, and the coupling degree between subsystems is high. Due to the fact that the number of the service changes is large, the implementation period is short, core codes of the service system are modified frequently, and the management difficulty of the single service system is greatly increased. In addition, the existing blood relationship management mainly depends on manual maintenance, and once the system code, script and model are changed, the manual maintenance is omitted or is not timely, and inaccurate relationship can be caused. Most of the current subsystem blood relationship management technologies are application level relationship management, the application level blood relationship is simple and easy to implement, the execution efficiency is high, the analysis precision is low, and enterprise-level multi-system and cross-system application is difficult to support.

Disclosure of Invention

The embodiment of the application provides a data blood relationship construction method and system, electronic equipment and a storage medium, which are used for realizing construction of data blood relationship among multiple subsystems.

The data blood relationship construction method provided by the embodiment of the application is applied to a service system comprising a plurality of subsystems, and comprises the following steps:

acquiring metadata information of a data object contained in a service system;

capturing metadata change information of each subsystem;

obtaining a subsystem data blood relationship corresponding to each subsystem according to the metadata variable information;

and obtaining the data blood relationship of the service system according to the subsystem data blood relationship of each subsystem included in the service system.

In some embodiments, obtaining metadata information of a data object included in a business system specifically includes:

integrating metadata for each data object into a business system;

defining metadata to obtain a metadata type, and creating a metadata entity according to the metadata type;

and defining a business label and a business classification for the metadata, and associating the business label and the business classification of the metadata object through the metadata.

In some embodiments, capturing metadata change information for each subsystem specifically includes:

capturing change information of a metadata entity of a subsystem;

and updating the metadata of the data object according to the change information.

In some embodiments, generating the subsystem data consanguinity relationship corresponding to each subsystem according to the metadata variation information of each subsystem specifically includes:

and generating a blood-margin dependence pedigree graph between the data objects of the subsystem according to the change information of each data object, wherein the blood-margin dependence pedigree graph is used as the subsystem data blood-margin relation of the subsystem.

In some embodiments, the method further comprises:

the business system establishes a blood relationship exchange space, generates a first public key and a first private key and stores the first public key and the first private key in the blood relationship exchange space;

the subsystem generates and stores a second public key and a second private key; the second public keys of different subsystems are different, and the second private keys of different subsystems are different;

the blood relationship exchange space sends a first public key to the subsystem, and the subsystem sends a second public key to the blood relationship exchange space;

obtaining the data consanguinity relationship of the service system according to the subsystem data consanguinity relationship of each subsystem included in the service system, specifically including:

each subsystem encrypts the blood relationship of the subsystem data through a second private key and sends the encrypted blood relationship of the subsystem data to a blood relationship exchange space;

and the blood relationship exchange space decrypts the blood relationship of the subsystem data according to the second public key, obtains the blood relationship among the subsystems according to the blood relationship of the subsystem data, and generates the blood relationship of the service system data.

In some embodiments, after obtaining the business system data relationship according to the subsystem data relationship of each subsystem included in the business system, the method further includes:

the blood relationship exchange space encrypts the blood relationship of the business system data through a first private key;

sending the encrypted data consanguinity relation of the service system to each subsystem;

and each subsystem decrypts the encrypted business system data blood relationship according to the first public key to obtain the decrypted business system data blood relationship.

In some embodiments, obtaining a blood relationship between a plurality of subsystems according to a blood relationship of data of each subsystem, and generating a blood relationship of data of a business system includes:

and analyzing, classifying, weighting and recursing the metadata information in the blood relationship of the subsystem data of each subsystem according to a preset rule to obtain the blood relationship among the subsystems and generate the blood relationship of the data of the service system.

In some embodiments, analyzing, classifying, weighting, and recursing metadata information in a blood relationship of subsystem data of each subsystem according to a preset rule to obtain the blood relationship among the subsystems specifically includes:

scanning data fields in the data consanguinity relationship of the subsystem; the data field includes: an input field and an output field;

defining the transmission relation and the operation relation of the data fields as the interactive relation of the input fields and the output fields;

generating a calling path according to the calling flow of the input field;

and (3) taking the output field as a starting point, combining a calling path, adopting a decision tree prediction model and a recursion algorithm, analyzing, classifying, weighting and recursing the data field, generating the blood relationship among a plurality of subsystems, and keeping the incidence relation between the input field and the output field.

The data blood relationship construction system provided by the embodiment of the application comprises:

the acquisition module is used for acquiring metadata information of a data object contained in the service system; the service system comprises a plurality of subsystems;

the capture module is used for capturing metadata change information of each subsystem;

the first data blood relationship construction module is used for acquiring the subsystem data blood relationship of each subsystem according to the metadata variable information;

and the second data consanguinity relationship construction module is used for obtaining the business system data consanguinity relationship according to the subsystem data consanguinity relationship of each subsystem included in the business system.

The embodiment of the application provides a computer device, which comprises:

one or more processors;

storage means for storing one or more programs;

when the one or more programs are executed by the one or more processors, the one or more processors implement the data context construction method provided by the embodiments of the present application.

A computer-readable storage medium is provided, on which a computer program is stored, where the computer program is executed by a processor to implement the data relationship construction method provided in the embodiments of the present application.

The computer program product provided by the embodiment of the present application includes a computer program, and when the computer program is executed by a processor, the method for constructing the data relationship provided by the embodiment of the present application is implemented.

The data blood relationship construction method and system, the electronic device and the storage medium provided by the embodiment of the application are applied to a business system comprising a plurality of subsystems, the subsystem data blood relationship corresponding to each subsystem is obtained based on metadata information of a data object, and the data blood relationship of the business system is obtained according to the subsystem data blood relationship corresponding to each subsystem, so that the enterprise-level data blood relationship of the cross-subsystem can be constructed. Through the data consanguinity relation of the business system, the functions of data management, system asset management and treatment can be conveniently realized; the system can provide a foundation for building an asset catalogue, an asset classification and an asset management of a business system, and simultaneously provide a cooperation function surrounding the data assets and the system assets for an operation and maintenance analyst or an management team.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flow chart of a data relationship construction method according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a decision tree provided in an embodiment of the present application;

FIG. 3 is a schematic flow chart illustrating another method for constructing data relationship according to an embodiment of the present disclosure;

FIG. 4 is a table-level data relationship diagram according to an embodiment of the present disclosure;

FIG. 5 is a diagram illustrating a relationship between field-level data and blood relationship according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a bleeding boundary relationship capture logic between subsystems according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a data relationship construction system according to an embodiment of the present application;

fig. 8 is a schematic diagram of another data relationship construction system according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings of the embodiments of the present application. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all embodiments. And the embodiments and features of the embodiments in the present application may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the application without any inventive step, are within the scope of protection of the application.

Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. As used in this application, the terms "first," "second," and the like do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items.

It should be noted that the sizes and shapes of the figures in the drawings are not to be considered true scale, but are merely intended to schematically illustrate the present disclosure. And the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

The embodiment of the present application provides a data blood relationship construction method provided by the embodiment of the present application, which is applied to a service system including a plurality of subsystems, and as shown in fig. 1, the method includes:

s101, acquiring metadata information of a data object contained in a service system;

s102, capturing metadata change information of each subsystem;

s103, obtaining a subsystem data blood relationship corresponding to each subsystem according to the metadata variable information of each subsystem;

and S104, obtaining the data blood relationship of the service system according to the subsystem data blood relationship of each subsystem included in the service system.

The data relationship construction method provided by the embodiment of the application is applied to a service system comprising a plurality of subsystems, the subsystem data relationship corresponding to each subsystem is obtained based on metadata information of a data object, and the data relationship of the service system is obtained according to the subsystem data relationship corresponding to each subsystem, so that the enterprise-level data relationship of the cross-subsystem can be constructed. Through the data consanguinity relation of the business system, the functions of data management, system asset management and treatment can be conveniently realized; the system can provide a foundation for building an asset catalogue, an asset classification and an asset management of a business system, and simultaneously provide a cooperation function surrounding the data assets and the system assets for an operation and maintenance analyst or an management team.

It should be noted that, in the data relationship construction method provided in the embodiment of the present application, the nature of data flow processing among multiple subsystems is used to record and process, and the data relationship is constructed. The transaction processing among subsystems is essentially a data collection and processing process, the data processing process is actually the transmission, operation deduction and filing activities of a program to data, and the filed database and files can influence the data genetic relationship construction result of the subsystems or are retransmitted to other subsystems. According to the data blood relationship construction method provided by the embodiment of the application, all subsystem software entities (including programs, databases, files, common components and communication interface structure definitions among all parts) included in the whole business system are used as analysis objects, and the continuity of data flow tracking can be ensured.

In specific implementation, the data lineage relationship construction method provided in the embodiment of the present application may be based on Structured Query Language (SQL), for example. It should be noted that SQL is a standard computer language for accessing and processing databases, and is a database query and programming language for accessing data and querying, updating, and managing relational database systems. SQL has the functions of data definition, data manipulation, and data control.

In specific implementation, in the whole data relationship construction process, the data objects are distributed in subsystems such as programs, communication interfaces, databases and files in the form of data fields. For the collection and storage of data field interaction information, the field level relationships include antecedent data field elements and postcedent data field elements. Each data field element contains 4-layer constraints: the version number, the application name, the entity name and the field name can be recorded and stored for the 4 layers of constraints respectively.

In some embodiments, the step S101 of obtaining metadata information of a data object included in the service system specifically includes:

integrating metadata for each data object into a business system;

It should be noted that the metadata of the data object is the basis for data relationship construction, and the metadata is integrated into the business system for subsequent data relationship construction.

It should be noted that the data objects related to the business system may include various types. The data objects comprise relational data and non-relational data; data objects include, for example: mysql, oracel, hive, hbase, and the like. In specific implementation, basic data can be provided for business system data relationship construction by manually registering metadata of a data object or directly importing the metadata (such as Hive metadata) of an existing data object.

In specific implementation, defining metadata to obtain a metadata type, and creating a metadata entity according to the metadata type, specifically including: modeling the metadata usage type, obtaining the metadata type, and representing the metadata type as an entity.

In specific implementation, the types are uniquely identified by "name", each type has a meta-type, and the meta-type includes: (1) primitive type: borolan, byte, short, int, long, float, double, bigineger, bigdecimal, string, date; (2) enumerating the element type; (3) the collection element type is as follows: array and mapping; (4) the compound element type: entities, structures, classifications, relationships.

In particular implementations, an entity is a specific value or a specific column of a type, such as a table, that is an entity. An entity is identified by a Unique Identifier (GUID). This GUID is generated by the server when defining the data object and remains unchanged throughout the life cycle of the entity. At any time, this particular entity may be accessed using its GUID.

In concrete implementation, the metadata definition mainly abstracts the metadata, so that various metadata sources of different types can be managed conveniently and uniformly. The definition of the identifier may guarantee the uniqueness of the metadata.

In particular, the service label and the service classification are defined for the metadata, that is, the service label and the service classification are allowed to be defined for the metadata by a user. The business tags and classes are associated by metadata to assets such as libraries, tables, columns, etc. at the time of implementation. Defining service tags and service classifications for metadata can achieve correlation of metadata by service tags and service classifications, correlation of metadata and data assets by service tags and service classifications, management of metadata by different service classifications, wider and finer expression of service ranges of metadata by service classification levels, and propagation of consanguinity dependency by service tags and service classifications.

capturing change information of a metadata entity of a subsystem;

and updating the metadata entity according to the change information.

In particular, the metadata entity of the data object may be updated through a message queue, and the change information may include, for example, entity creation information, entity update information, entity deletion information, field creation information, field update information, and field deletion information.

In particular implementations, change information for a metadata entity of a subsystem may be automatically captured by a capture hook. During the data object transmission process, operations of creation of metadata, updating of metadata, deletion of metadata and the like can be automatically captured through different types of capture hooks, and the metadata of the updated data object is notified through a message queue. The capture hook may capture the following operations: entity creation, entity update, entity deletion, field creation, field update, field deletion. For example, when the data field of the data object is queried and read, the query content of the data field is input, the value is written into the table, the values of the inclined table name and the column name are recorded after the query is completed, the table name and the column name are synchronously updated into the table, when the operation is executed in the data object transfer process, the capture hook is triggered, the changed output column and a group of input columns or input tables are received in the capture hook, and the changed output column and the group of input columns or input tables are the change information of the metadata entity.

In some embodiments, generating a subsystem data blood relationship corresponding to each subsystem according to the metadata variable information specifically includes:

In particular implementations, the subsystem data consanguinity relationship may be, for example, a subsystem consanguinity. The subsystem ontology comprises resources such as program instructions, databases, files, common components, and communication interface structure definitions among the above parts.

In a specific implementation, the generating a lineage diagram of dependency between data objects of the subsystem according to the change information of each data object specifically includes: analyzing the blood relationship of the data objects according to the change information to obtain the transmission relationship of each data object, and generating the blood relationship of each data object, namely a blood relationship dependence pedigree diagram.

In particular, the decision tree prediction model may be used to analyze the blood relationship of the data object. It should be noted that the decision tree prediction model is a mapping relationship between object attributes and object values. Each node in the decision tree represents an object, and each divergent path represents a possible attribute value, and each leaf node corresponds to the value of the object represented by the path traversed from the root node to the leaf node. The decision tree has only a single output, and if there are plural outputs, independent decision trees can be established to process different outputs. As shown in FIG. 2, a decision tree contains three types of nodes: decision nodes, typically represented by rectangular boxes; opportunity nodes, typically represented by circles; the termination points are typically represented by triangles.

In specific implementation, the blood relationship of the data object can be analyzed by a process-oriented program by using a decision tree prediction model. Specifically, according to the change information, a complete logical processing segment is identified by using the procedure-oriented program, for example, a complete logical processing segment is identified by main to end, and a logical processing segment is divided into more smaller logical processing segments by instructions such as select, branch, jump, and the like. And refining the selection and branch conditions of all the logic processing sections layer by layer, thereby simplifying the selection and branch conditions into a tree-shaped graph consisting of a plurality of logic processing sections as a data blood-margin dependence pedigree graph.

In particular implementations, the data lineage graph is generated by associating an output column with a set of input columns or tables, for example, when the change information includes an output column and a set of input columns or tables.

In practice, the dependency types of the blood-related dependency pedigree map are mainly 3 types as follows:

simple dependence: the output column has the same value as the input.

Expression: the output columns are transformed at runtime by some expression (e.g., Hive SQL expression) on the input columns.

Script: the output column is the script transformation provided by the user.

In specific implementation, metadata entity change is triggered through sql processing, change information is captured by a capture hook, and a data lineage chart is constructed. That is, the metadata alteration consanguinity action can be triggered by executing the sql statement, and the capture hook is automatically used to start capturing the consanguinity.

In a specific implementation, after generating the lineage diagram of the relationship dependence between the data objects of the subsystem, the method further includes: and storing the blood relationship dependence pedigree graph to a graph database, and generating an index to be stored to a search engine.

The following is an example of a process for generating a lineage-dependent lineage diagram, as shown in fig. 3, which includes:

s201, acquiring metadata of a data object;

s202, defining the acquired metadata;

s203, creating a metadata entity based on the defined metadata;

s204, defining a service label and a service classification for the metadata;

s205, capturing the change information of the metadata entity of the subsystem, and updating the metadata entity according to the change information;

s206, analyzing the blood relationship of the data object according to the change information to generate a blood relationship dependence pedigree diagram;

s207, storing the blood relationship dependence pedigree map into a map database;

and S208, generating an index of the blood-margin dependent pedigree graph and storing the index into a search engine.

In some embodiments, the data consanguinity relationship construction method further comprises:

generating a calling path according to the calling flow of the input field;

In the specific implementation, the association between subsystems is divided into two levels, namely an entity level and a data field level. The entity-level relevance refers to the coupling and calling relationship among the subsystems, and the data field-level relevance refers to the coupling relationship of the data field of any subsystem and the data fields of other subsystems, such as mapping, calculation, transmission and the like.

According to the data consanguinity relationship construction method provided by the embodiment of the application, mapping and transfer relationships among data fields of related subsystems are identified through semantic analysis of program instructions, the analysis precision is high, an influence analysis view and a business logic view of an enterprise big data consanguinity tree are created according to analysis results, the whole life cycle of a data processing process is recorded, problem details and influence ranges among systems are conveniently evaluated, and the method has important significance in aspects of problem prediction, quality control, data analysis, usability and the like. The enterprise can efficiently and effectively meet the requirement of data compliance in the enterprise through blood margin treatment, and can be conveniently integrated with the whole enterprise ecosystem.

In particular implementations, static analysis and listing of paths that may be involved in the program logic of a business system may be performed. Specifically, depth-first traversal can be performed on a program associated with the system model, and the dependency relationship between the models is judged by writing in and reading out the system model id. And judging that the current system model is dependent operation when reading operation is met, and judging that the current system model is target operation when writing operation is met. All system model ids and associated system model programs are recursively operated. And if a clause is encountered, pushing the current processing to process the clause. And after the clauses are processed, popping the stack. In the process of processing the words, the information of the current sub-query is stored when the sub-query is encountered, the relation with the father query is judged, and finally a tree structure is formed.

In the implementation, the table-level data relationship is shown in fig. 4, the field-level data relationship is shown in fig. 5, and in fig. 4, the system a, the system B, the system C, and the system D represent two different subsystems, respectively. The logic diagram for capturing the relationship between the blood relationship between the subsystems is shown in fig. 6, a system a and a system B represent two different subsystems respectively, when the system a performs a table creation action and the system B performs a table deletion action, a capture hook is triggered, an output column and a group of input columns or input tables which are changed are received in the capture hook, metadata of the system a and the system B is updated, and the relationship between the system a and the system B is analyzed to be a dependency relationship according to change information of the two subsystems.

According to the data blood margin construction method, the inter-subsystem interactive execution process can be automatically captured to obtain the data blood margin relation among the subsystems. And establishing a data blood relationship among the cross-subsystem, and identifying missing values, abnormal values and other data abnormalities through deep mining analysis and automatic data quality analysis. It is also possible to reveal, through deep mining analysis, how the data evolves during its lifecycle, where it came from, and foresee ancestry and impact analysis of assets that will be affected by future changes. Each table or column that is derived from a sensitive column is guaranteed to inherit the same classification and security controls.

In addition, the data consanguinity construction method provided in the embodiment of the present application not only reports the subsystem data consanguinity relationship to the consanguinity relationship exchange space and obtains the consanguinity relationship among the multiple subsystems, that is, obtains the business system data consanguinity relationship, but also synchronously issues the obtained business system data consanguinity relationship to each subsystem, so that, for a business system including the multiple subsystems, a complete closed loop can be formed in which the business system data consanguinity relationship is obtained from data acquisition, subsystem data consanguinity relationship construction, cross-subsystem data consanguinity relationship construction, and the business system data consanguinity relationship is issued to each subsystem. The continuous development of a business system is supported, and the construction of the blood relationship of a mass data model can be realized. Data traceability in the data management process can be realized by constructing the data blood relationship of the service system, data fusion is guaranteed by analyzing the blood relationship of the data, and traceability of data fusion processing can be realized by analyzing the blood relationship of the data.

According to the data consanguinity construction method provided by the embodiment of the application, the metadata defines the service labels and the service classification, for example, a table and a field can be labeled, and through consanguinity analysis, the labels, such as service data, advertisements and order classes, are labeled in the whole data circulation process. The corresponding label can be marked according to the priority importance degree.

In specific implementation, the data blood relationship of the business system can be obtained by using a data blood relationship construction method to evaluate the data, and evaluation analysis can be performed from multiple angles such as concentration, distribution and redundancy of the data through data blood relationship analysis, so that the value of the data is judged. And the data quality detection and processing can be ensured in each link according to the data blood relationship, so that the quality of the final data is improved. The data warehouse can be optimized by analyzing the data of the blood margin of the table and the field, finding out more data to use, and analyzing whether repeated calculation exists or not and wasting resources.

Based on the same inventive concept, an embodiment of the present application further provides a data blood relationship construction system, as shown in fig. 7, including:

a first obtaining module 101, configured to obtain metadata information of a data object included in a service system; the service system comprises a plurality of subsystems;

a capture module 102, configured to capture metadata change information of each subsystem;

the first data blood relationship construction module 103 is configured to obtain a subsystem data blood relationship corresponding to each subsystem according to the metadata variable information of each subsystem;

and a second data relationship construction module 104, configured to obtain the business system data relationship according to the subsystem data relationship of each subsystem included in the business system.

In some embodiments, the data consanguinity construction system further comprises: a metadata integration module for integrating metadata of each data object into the business system;

the first obtaining module is configured to obtain metadata information of a data object included in a service system, and specifically includes:

In some embodiments, the capturing module is configured to capture metadata change information of each subsystem, and specifically includes:

capturing change information of a metadata entity of a subsystem;

and updating the metadata entity according to the change information.

In particular implementations, the capture module includes a capture hook, i.e., change information of a metadata entity of the subsystem can be automatically captured by the capture hook. During the data object transmission process, operations of creation of metadata, updating of metadata, deletion of metadata and the like can be automatically captured through different types of capture hooks, and the metadata of the updated data object is notified through a message queue. The capture hook may capture the following operations: entity creation, entity update, entity deletion, field creation, field update, field deletion. For example, when the data field of the data object is queried and read, the query content of the data field is input, the value is written into the table, the values of the inclined table name and the column name are recorded after the query is completed, the table name and the column name are synchronously updated into the table, when the operation is executed in the data object transfer process, the capture hook is triggered, the changed output column and a group of input columns or input tables are received in the capture hook, and the changed output column and the group of input columns or input tables are the change information of the metadata entity.

In some embodiments, the first data consanguinity relationship building module is configured to generate a subsystem data consanguinity relationship of the subsystem according to the metadata variable information, and specifically includes:

In specific implementation, the blood relationship of the data object can be analyzed by a process-oriented program by using a decision tree prediction model.

In specific implementation, the data blood relationship construction system further includes:

a graph database for storing the lineage graph and generating an index;

a search engine to store the index.

In some embodiments, the data consanguinity construction system further comprises:

the system comprises a blood relationship exchange space establishing module, a blood relationship exchange space establishing module and a blood relationship exchange space establishing module, wherein the blood relationship exchange space establishing module is used for establishing a blood relationship exchange space, generating a first public key and a first private key and storing the first public key and the first private key into the blood relationship exchange space;

each subsystem also comprises a key pair generation module used for generating and storing a second public key and a second private key; the second public keys of different subsystems are different, and the second private keys of different subsystems are different;

in some embodiments, the kindred relationship exchange space includes a second data kindred relationship construction module;

the kindred relationship exchange space further comprises:

the first sending and receiving module is used for sending the first public key to the subsystem and receiving the second public key sent by the subsystem;

each subsystem further comprises:

the second sending and receiving module is used for encrypting the blood relationship of the subsystem data through a second private key and sending the encrypted blood relationship of the subsystem data to a blood relationship exchange space;

the second data consanguinity relationship construction module is configured to obtain a business system data consanguinity relationship according to a subsystem data consanguinity relationship of each subsystem included in the business system, and specifically includes:

and decrypting the blood relationship of the subsystem data according to the second public key, obtaining the blood relationship among the subsystems according to the blood relationship of the subsystem data, and generating the blood relationship of the service system data.

In some embodiments, the first transceiver module is further configured to: encrypting the data consanguinity relation of the business system by a first private key; sending the encrypted data consanguinity relation of the service system to each subsystem;

the second sending and receiving module is further configured to: and decrypting the encrypted business system data blood relationship according to the first public key to obtain the decrypted business system data blood relationship.

In some embodiments, the second data consanguinity relationship building module is configured to obtain consanguinity relationships among the multiple subsystems according to the data consanguinity relationships of each subsystem, and generate the data consanguinity relationships of the business system, and specifically includes:

generating a calling path according to the calling flow of the input field;

Based on the same inventive concept, the embodiment of the present application further provides a computer device, which includes:

one or more processors;

storage means for storing one or more programs;

one or more processors;

storage means for storing one or more programs;

The electronic device may be a desktop computer, a portable computer, a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a server, and the like. In some embodiments, the storage device is, for example, a memory, and as shown in fig. 8, the electronic device may include a processor 201 and a memory 202.

The Processor 201 may be a general-purpose Processor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic blocks disclosed in the embodiments of the present Application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

Memory 202, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charged Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. The memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 202 in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; the computer storage media may be any available media or data storage device that can be accessed by a computer, including but not limited to: various media that can store program codes include a removable Memory device, a Random Access Memory (RAM), a magnetic Memory (e.g., a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical Memory (e.g., a CD, a DVD, a BD, an HVD, etc.), and a semiconductor Memory (e.g., a ROM, an EPROM, an EEPROM, a nonvolatile Memory (NAND FLASH), a Solid State Disk (SSD)).

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media that can store program codes include a removable Memory device, a Random Access Memory (RAM), a magnetic Memory (e.g., a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical Memory (e.g., a CD, a DVD, a BD, an HVD, etc.), and a semiconductor Memory (e.g., a ROM, an EPROM, an EEPROM, a nonvolatile Memory (NAND FLASH), a Solid State Disk (SSD)).

In some possible embodiments, various aspects of the methods provided by the present disclosure may also be implemented in a form of a program product including program code for causing a computer device to perform the steps of the methods according to various exemplary embodiments of the present disclosure described above in this specification when the program product is run on the computer device, for example, the computer device may perform the data relationship construction method described in the embodiments of the present disclosure. The program product may employ any combination of one or more readable media.

To sum up, the data relationship construction method and system, the electronic device, and the storage medium provided in the embodiments of the present application are applied to a business system including a plurality of subsystems, and the data relationship of the business system is obtained based on metadata information of a data object and the data relationship of the subsystem corresponding to each subsystem, so as to construct an enterprise-level data relationship across the subsystems. Through the data consanguinity relation of the business system, the functions of data management, system asset management and treatment can be conveniently realized; the system can provide a foundation for building an asset catalogue, an asset classification and an asset management of a business system, and simultaneously provide a cooperation function surrounding the data assets and the system assets for an operation and maintenance analyst or an management team.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A data blood relationship construction method is applied to a business system comprising a plurality of subsystems, and is characterized by comprising the following steps:

acquiring metadata information of a data object contained in the service system;

capturing metadata change information for each of the subsystems;

obtaining a subsystem data blood relationship corresponding to each subsystem according to the metadata variable information of each subsystem;

and obtaining the data blood relationship of the service system according to the data blood relationship of the subsystem of each subsystem included in the service system.

2. The method according to claim 1, wherein the obtaining metadata information of the data object included in the service system specifically includes:

integrating metadata for each of the data objects into the business system;

defining the metadata to obtain a metadata type, and creating a metadata entity according to the metadata type;

3. The method of claim 2, wherein the capturing metadata change information for each of the subsystems comprises:

capturing change information for the metadata entity of the subsystem;

and updating the metadata entity according to the change information.

4. The method according to claim 3, wherein the generating a subsystem data consanguinity relationship corresponding to each of the subsystems according to the metadata variant information of each of the subsystems specifically comprises:

and generating a lineage graph of dependency between the data objects of the subsystem according to the change information of each data object, wherein the lineage graph is used as a subsystem data lineage relation of the subsystem.

5. The method of claim 4, further comprising:

the blood relationship exchange space sends the first public key to the subsystem, and the subsystem sends the second public key to the blood relationship exchange space;

obtaining a data consanguinity relationship of the service system according to the subsystem data consanguinity relationship of each subsystem included in the service system, specifically including:

each subsystem encrypts the subsystem data consanguinity relationship through the second private key and sends the encrypted subsystem data consanguinity relationship to the consanguinity relationship exchange space;

and the blood relationship exchange space decrypts the blood relationship of the subsystem data according to the second public key, obtains the blood relationship among a plurality of subsystems according to the blood relationship of each subsystem data, and generates the blood relationship of the service system data.

6. The method of claim 5, wherein after obtaining business system data context relationships from the subsystem data context relationships of each of the subsystems comprised by a business system, further comprising:

the blood relationship exchange space encrypts the blood relationship of the business system data through the first private key;

7. The method of claim 6, wherein the obtaining of the blood-related relationships between the plurality of subsystems according to the blood-related relationships of each subsystem data to generate the data-related relationships of the business system comprises:

and analyzing, classifying, weighting and recursing the metadata information in the subsystem data blood relationship of each subsystem according to a preset rule to obtain the blood relationship among the subsystems and generate the business system data blood relationship.

8. The method according to claim 7, wherein analyzing, classifying, weighting, and recursion the metadata information in the subsystem data blood relationship of each of the subsystems according to a preset rule to obtain blood relationship between the subsystems specifically comprises:

scanning data fields in the subsystem data consanguinity relationship; the data field includes: an input field and an output field;

defining the transmission relation and the operation relation of the data fields as the interactive relation between the input fields and the output fields;

generating a calling path according to the calling flow of the input field;

and taking the output field as a starting point, combining the calling path, adopting a decision tree prediction model and a recursion algorithm, analyzing, classifying, weighting and recursing the data field, generating the blood relationship among a plurality of subsystems, and keeping the incidence relation between the input field and the output field.

9. A data relationship construction system, comprising:

the acquisition module is used for acquiring metadata information of a data object contained in the service system; the business system comprises a plurality of subsystems;

the first data blood relationship construction module is used for acquiring the subsystem data blood relationship corresponding to each subsystem according to the metadata variable information of each subsystem;

and the second data blood relationship construction module is used for obtaining the data blood relationship of the service system according to the subsystem data blood relationship of each subsystem included in the service system.

10. A computer device, the device comprising:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the data lineage construction method according to any one of claims 1-8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the data relationship construction method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the data lineage relationship construction method of any one of claims 1 to 8.