CN115511434A - Enterprise data management method and system based on business metadata drive - Google Patents

Abstract

The invention discloses an enterprise data management method and system based on business metadata drive, belonging to the enterprise field and used for solving the problem of separation of an enterprise data management platform and enterprise business operation, comprising a data management interface, a data model analysis subsystem and a data management execution subsystem, wherein the data management subsystem comprises a data management workflow, a data cleaning platform, a data exchange platform and a scheduling and monitoring system for data quality analysis.

Description

Enterprise data management method and system based on business metadata drive

Technical Field

The invention belongs to the field of enterprises, relates to a data management technology, and particularly relates to an enterprise data management method and system based on business metadata driving.

Background

At present, almost all enterprises are installed or built with own informatization systems, such as financial systems, customer relationship management systems, supply chain management systems and the like. With the continuous improvement of enterprise informatization degree, enterprises deploy more and more business applications and deposit more and more data, and the following problem is how to effectively and uniformly manage the data of each business system, so that 1) data islands of different business systems are opened to enable better collaboration among businesses, 2) during cross-system business operation, because the business data have inconsistent explanations in different business departments, the business departments have to maintain offline forms to support the collaboration of cross-business applications, 3) the data of each business system are synthesized to establish an enterprise global data view, and cross-business, multi-angle and multi-level comprehensive analysis is carried out on each business line from a global perspective, 4) the application system cannot timely follow the changes of business rules and management rules in enterprise development, and after the rules change, the business departments usually also maintain the enterprise data in an offline form mode, and the scattered data cannot be timely managed and normalized.

Currently, the mainstream data management is mostly implemented in a project system, and a general project is divided into a data management consultation part and a data management platform implementation part.

The data management consultation project is mainly used for carrying out data management theoretical training on an enterprise, researching the current business situation, the current data situation and the business requirements in a certain time of the enterprise, formulating a data model and a data standard for the enterprise according to the business model of the enterprise, establishing a data management organization, a management flow and a management system, and providing an enterprise data management planning scheme.

The enterprise data management platform implementation project mainly comprises the steps of cleaning historical data according to an enterprise data model and a data standard, standardizing and arranging data exchange among current application systems of an enterprise, establishing a data mapping relation, deploying a data quality monitoring system, and monitoring and tracking the data quality of the enterprise according to the current data model and the standard of the enterprise.

The data governance scheme of current project system can only guarantee to solve the current data governance problem of enterprise when the project is implemented and is accomplished, because the continuous development and the change of enterprise's business, can't also bring extra management burden for the continuous governance of guarantee enterprise data simultaneously. The main points are as follows:

1. The problem of enterprise business management and data management 'two-layer skin' is not solved. The core of enterprise operation is business management, and a data management organization established by a data management project is separated from a business management flow and gradually flows into forms along with the time.

2. The data standard established by the project is established only from the data management perspective and is separated from the service scene. Under different business scenes, different business processes and business rules exist, and therefore, different data standards exist. From the perspective of data standards, if service scene data cannot be supported, effective support for existing and future services cannot be formed.

3. The data integration scheme formulated by the project is only an integration scheme oriented to the current business application system, and cannot support implementation and integration of a subsequent newly added application system of an enterprise.

4. The data governance established by the project does not settle data management capacity for the enterprise, and the enterprise lacks effective tools to configure and update the existing data governance system when the business mode and the business flow are changed.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide an enterprise data governance method and system based on business metadata driving.

The invention solves the technical problem that an enterprise data management platform is separated from the operation of enterprise business, introduces enterprise data management driven by business metadata, automatically generates an enterprise data model matched with a business model through a classified and layered business analysis model according to the business metadata generated by enterprise business management, and then analyzes the business metadata to a data standard model, a data integration model, a data quality model and a data monitoring model of the enterprise data management platform through the data model, thereby realizing the intelligent adaptation of the enterprise data management platform to the enterprise business, improving the quality of enterprise data management and reducing the management burden.

The purpose of the invention can be realized by the following technical scheme:

the enterprise data governance system based on the business metadata drive comprises a data governance management interface, a data model analysis subsystem and a data governance execution subsystem;

the data management interface is used for providing corresponding operation interfaces for a service architect, a data management engineer and a system integration engineer, providing functions of user login, service model calling, service model rule configuration and the like for the service architect, providing functions of data model management, data standard verification and release, data quality management, analysis result tracking and the like for the data management engineer, and providing functions of service and application system relationship configuration and application system data distribution viewing for the system integration engineer;

The data model analysis subsystem is used for providing analysis from a business model based on business metadata to the data model to generate an enterprise data model, and then generating a data standard, a data integration model and a data quality model facing enterprise data management through the business model and the data model, wherein the data standard, the data integration model and the data quality model comprise an API (application program interface) of data integration and a data quality analysis task in system integration, and further control the data management execution system to execute a specific data management task;

the data management execution subsystem comprises a data management workflow, a data cleaning platform, a data exchange platform and a scheduling and monitoring system for data quality analysis.

Further, the data governance system management interface unit comprises:

a user login module: the system is used for verifying the identity of the user, ensuring that the authorized user can log in the system and executing the corresponding operation of the authorized execution interface after logging in the system;

a business model import module: the system is used for supporting a business architect to import a business architecture model output according to a specified format into the system by a visual interface;

a business rule configuration module: the system is used for supporting a business architect to configure business rules for the business scenes of all the business process sections by using a visual interface;

A data model management module: the system is used for supporting a data administration engineer to manage the data model generated by the system by using a visual interface;

and (3) data standard auditing and publishing: the system is used for auditing and releasing the specified data standard of a data management engineer and managing a data standard reference library by using a visual interface;

the data quality management and analysis result tracking module: the system is used for supporting a data management engineer to configure and schedule a data quality analysis task by a visual interface, checking a system data quality analysis result and tracking a data quality improvement task;

a service and application system relation configuration module: the system is used for supporting the cooperation of a system integration engineer and a business architect by a visual interface, configuring the corresponding relation between a business module and an application system and applying an interactive interface between the systems;

the application system data distribution viewing module: the system integration engineer is supported by a visual interface to check the life cycle of the data in each application system and the data interaction condition among the application systems.

Further, the data model parsing subsystem comprises:

a service model analysis algorithm unit: the system comprises a business model analysis algorithm program, a data model generation algorithm program, rules for supporting the operation of the algorithm and a storage module;

A data standard generation algorithm unit: the method comprises a data standard generation algorithm program, and a rule and data standard reference library for supporting the operation of an algorithm;

a data integration analysis algorithm unit: the system comprises an integrated API generation algorithm program for application system integration, an application system interface configuration library for supporting algorithm operation, and an API testing and storing module;

data quality analysis algorithm unit: the method comprises a data quality model generation algorithm facing different data types and quality judgment rule configuration supporting the operation of the algorithm.

Further, the data governance execution subsystem comprises:

data management workflow unit: task circulation of processes such as application, audit, change and the like in the data management execution process is included;

a data cleaning platform: the method comprises the steps of including a tool kit required by historical data cleaning in the data governance execution process;

a data exchange platform: the method comprises the steps of data exchange among application systems in the data governance execution process, including data acquisition and distribution;

scheduling a data quality monitoring task: the method comprises the steps of data quality monitoring and task scheduling in the data management execution process;

a data quality analysis platform: the method comprises the automatic execution of a data quality analysis task in the data governance execution process.

The enterprise data management method based on the business metadata drive specifically comprises the following steps:

step S101, a system configuration sub-process;

step S102, analyzing a sub-process by a service model;

step S103, a data model analysis sub-process;

step S104, the data model application sub-process.

Further, in step S101, the system configuration sub-process specifically includes:

user login: a user logs in the system through a user name and a password;

importing a business model: a service architect imports a service model into the system on an interface, and the system stores the service model into a system database for a service model analysis algorithm to use;

configuring a business rule: a service architect configures service rules on an interface, and the system stores the service rules into a system database; the business rules are configured on the nodes of the layered business model and are divided into flow rules and data rules;

configuring a business model analysis rule;

importing a main data model: the data management engineer imports the main data and the basic data model into the system on an interface, and the system stores the main data and the basic data model into a system database for a service model analysis algorithm unit to use;

Configuring data standard model rules: a data standard model is configured on an interface by a data management engineer, and the data standard model and matching rules thereof are stored in a system database by the system for a data standard generation algorithm unit to use;

and auditing and issuing data standards: a data management engineer initiates a process of auditing and releasing data standards on an interface, and corresponding managers perform auditing and releasing on the interface;

configuring a data quality model rule: a data quality model is configured on an interface by a data management engineer, and the data quality model and the matching rules thereof are stored in a system database by the system for a data quality judgment algorithm unit to use;

and (3) configuring data quality analysis and scheduling: a data quality engineer configures parameters of a data quality analysis task on an interface, directly starts the data quality analysis task through the interface or sets task starting time;

configuring the relation between the service and the application system: a system integration engineer configures the corresponding relation between the business layering model and the application system on an interface, and the system analyzes input information to form a business application system relation model for a data integration analysis algorithm;

configuring an application system data exchange interface: and configuring a data exchange interface supported by the application system on the interface by a system integration engineer, wherein the data exchange interface comprises an HTTP API, a database connection and a third party definition interface, configuring corresponding parameters for each interface mode, and storing the interface capability of the application system in a database for a data integration analysis algorithm.

Further, the business model includes:

a service layering model: taking the enterprise value chain as a root node, and describing by adopting a tree structure, wherein each node of the tree structure describes a business object corresponding to an enterprise process;

wherein, the information transmission rule in the hierarchical model is as follows:

step A: in the hierarchical model, all information flows from left to right;

and B, step B: the same-layer nodes under the same father node with service connection directly transmit information through connection;

and C: the nodes of different father nodes gather information to upper nodes step by step and upwards, and information transmission across the father nodes is completed through information gathering transmission of the upper nodes;

step D: the information is generated at the child node at the bottom layer, and the father node undertakes information aggregation and transmission;

service scenario connection model: describing the connection relation among the nodes, including the triggering condition of the node for transmitting information and the content of the transmitted information;

service post mapping model: describing post information for executing corresponding business processes in an enterprise, wherein the post information comprises organization departments and employee information to which posts belong;

service application system relationship model: the application system where the corresponding business process is executed is described, including data content and storage mode recorded for the business process in the business system.

Further, in step S102, the business model parsing sub-process includes:

loading a main data model: loading a main data model imported by a data management engineer when a subsystem request is configured;

analyzing the service hierarchical model: loading a business hierarchical model, performing depth-first traversal on a hierarchical model tree according to the later sequence, and constructing a business object tree from bottom to top; traversing each node, reading a corresponding node business model, calling a business model analysis rule, and creating a business object and an attribute set of the node; for each attribute of the business object, if the attribute is generated for the business object, generating an attribute field name, a type and a unit, if the attribute is from the calling of main data, judging which main data information is quoted by the business object, storing an index corresponding to the main data object, if the attribute is from the quoting of other business objects, judging the quoting type and the quoting mode, then judging the data connection between the front business object and the back business object, and creating a connection object;

analyzing a service scene connection model: loading a service connection scene model, performing depth-first traversal on a service connection creation model tree according to a preamble, and constructing a service object scene graph based on a service object tree from top to bottom; traversing each node in the service scene connection model, reading a service scene rule of the corresponding node, analyzing the rule to construct a service scene judgment expression, binding the service object attribute and the scene judgment expression, and establishing indirect data connection between the service objects;

Analyzing the service post mapping: loading a business post mapping relation, traversing a business scene connection graph for each post, and finding out a business object set related to the post and information output by the business objects; establishing association between organizations and business objects according to the posts and the hierarchical association of the organizations, establishing business objects associated with business processes under each organization, and identifying business objects across a plurality of organizations and business objects bearing information transmission among the organizations for a data quality model generation algorithm to use;

analyzing the relation of the service application system: loading a mapping relation between the service and the application systems, and finding out a service object set related to each application system; identifying business objects which span a plurality of application systems and business objects which are used for bearing information transfer between the application systems and used by a data integration model and a data quality model generation algorithm;

generating a data model: generating a system data model, the data model comprising four modules: defining business objects, drawing relation among the business objects, distributing diagram of data among organizations and distributing diagram of data among application systems; based on the analysis of the layering model and the connection model, the construction of business objects is completed, and based on the information transmission and reference relationship among the objects, the construction of a relational graph among the business objects is completed; based on the analysis of the mapping relation of the service posts, completing the construction of a distribution diagram of the data among the organizations; and completing construction of a distribution diagram of the data between the application systems based on the relation analysis of the service application systems.

Further, in step S103, the data model parsing sub-process includes:

loading a main data and business data model: loading main data and a basic data model configured by a data management engineer and loading a data model generated in a business model analysis sub-process in response to a configuration subsystem request;

generating a data standard model: the data standard model generation algorithm is divided into two aspects, namely, generating a main data standard and generating a service data standard;

the data standard generation flow of the main data is as follows: loading a business object connection graph, traversing the business object, calling the main data according to the business object, and associating the business rule with the corresponding main data attribute if the business object attribute has a bound business rule; traversing all the main data, sequentially generating a data standard of each attribute of the main data, and if the attribute is associated with the service rule, generating the data standard according to the configured service rule; if the attribute is a coding attribute, generating a data standard according to coding rule configuration, otherwise generating the data standard according to a data standard reference library;

the data standard generation process of the service data comprises the following steps: loading a business object connection graph, traversing all business objects, and for each attribute of the business objects:

If the data is newly generated in the business process: if the data standard is associated with the business rule, generating the data standard according to the configured business rule, otherwise, generating the data standard according to a data standard reference library;

if the attribute is the attribute of the quoted main data object, the data standard is the attribute value under the unique identifier of the bound main data object;

if the attribute is the attribute of the other referenced business objects: if the referenced business object is not changed along with other business objects, the data standard is the data standard of the attribute of the referenced business object; if the quote is changed with other business objects, the data standard is the attribute value under the unique identification of the bound business object;

generating a data integration model: the algorithm generates a corresponding data integration model for each application system, and the flow specifically comprises the following steps: loading service objects which are identified by a service model analysis algorithm and cross the application systems and are used for bearing information transmission between the application systems; loading an inter-application system data distribution map in the data model; traversing each application system configured by a system integration engineer;

generating a data quality model: the data quality model generation algorithm is also divided into two aspects, namely, a main data quality model and a service data quality model are generated.

Further, in step S104, the data model application sub-process specifically includes the following steps:

formulating a data cleaning rule: the data cleaning is to complement and correct the historical data aiming at the data quality problem in the historical data;

configuring a data exchange platform: according to the data integration model generated in the data model analysis sub-process, configuring the data exchange platform;

configuring data quality monitoring: and analyzing the data quality model generated in the sub-process according to the data model, and configuring the data quality monitoring platform.

Compared with the prior art, the invention has the beneficial effects that:

1. the invention sets up a data management system of an enterprise from a business source of data generation, unifies business management and data management, can radically solve the problem of 'two-layer skin' of business management and data management of the enterprise, promotes the data circulation of the enterprise to be more suitable for business operation of the enterprise, and enables the enterprise to better utilize digital technology to improve enterprise operation.

2. The invention identifies the data of the cross-service department through the service model, and gets through the different interpretations of the same data due to different services through the analysis of the reference relationship of the data among the service processes, so that the data can better realize the cross-department circulation.

3. The invention identifies the data of the cross-application system through the business model, and accesses different coding and storing modes of different application systems to the same data through identifying the data of the cross-application system in the business angle, thereby realizing the unification of enterprise data views.

4. The invention can better adapt to the development of enterprise business and the change of business management rules. The new business mode and management rule of the enterprise can be configured by a business architect and timely updated to a data management system, and a new data standard, data integration and data quality management mode is automatically generated through an intelligent business and data model analysis algorithm, so that the data management work of the enterprise is greatly reduced.

5. According to the configured data quality rule, the influence of historical data on the current business operation can be analyzed in real time, the historical data is cleaned according to the data value, the low-value historical data is removed, and the work of cleaning the historical data is greatly reduced.

Drawings

To facilitate understanding for those skilled in the art, the present invention will be further described with reference to the accompanying drawings.

FIG. 1 is a block diagram of the system of the present invention;

FIG. 2 is a flow chart of a data governance process in the present invention;

FIG. 3 is a flow diagram of a business model parsing sub-process in the present invention;

FIG. 4 is a flow chart of a data model parsing sub-process in the present invention.

Detailed Description

The technical solutions of the present invention will be described below clearly and completely in conjunction with the embodiments, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In one embodiment, as shown in fig. 1, an enterprise data governance system driven based on business metadata is proposed, and the system is composed of a data governance management interface, a data model analysis subsystem and a data governance execution subsystem:

1. the data management interface is a visual interface of the system, provides corresponding operation interfaces for a service architect, a data management engineer and a system integration engineer, provides functions of user login, service model calling, service model rule configuration and the like for the service architect, provides functions of data model management, data standard verification and release, data quality management, analysis result tracking and the like for the data management engineer, and provides functions of service and application system relationship configuration, application system data distribution viewing and the like for the system integration engineer;

2. And the data model analysis subsystem is used for providing analysis from a business model based on business metadata to the data model to generate an enterprise data model, and then generating a data standard, a data integration model and a data quality model facing enterprise data management through the business model and the data model, wherein the data standard, the data integration model and the data quality model comprise an API (application program interface) for data integration and a data quality analysis task in system integration, and further control the data management execution system to execute a specific data management task.

3. The data governance execution subsystem (the subsystem can be used as a single system, and the data governance system in the industry is mainly concentrated on the subsystem at present) comprises a data management workflow, a data cleaning platform, a data exchange platform and a scheduling and monitoring system for data quality analysis.

The core component units of the system are as follows:

1. management interface unit of data management system

1.1, a user login module: verifying the identity of the user, ensuring that the authorized user can log in the system, and only executing the corresponding operation of the authorized execution interface after logging in the system;

1.2, a business model importing module: a visual interface supports a business architect to import a business architecture model output according to a specified format into a system;

1.3, a business rule configuration module: supporting a service architect to configure service rules for the service scenes of each service flow section by using a visual interface;

1.4, a data model management module: a visual interface is used for supporting a data management engineer to manage a data model generated by the system;

1.5, data standard auditing and publishing: the visual interface supports the verification and release of the specified data standard of the data management engineer and the management of a data standard reference library;

1.6, a data quality management and analysis result tracking module: supporting a data management engineer to configure and schedule a data quality analysis task by a visual interface, checking a system data quality analysis result, and tracking a data quality improvement task;

1.7, a service and application system relation configuration module: supporting the cooperation of a system integration engineer and a service architect by a visual interface, configuring the corresponding relation between a service module and an application system, and applying an interactive interface between the systems;

1.8, an application system data distribution viewing module: and a visual interface supports a system integration engineer to check the life cycle of the data in each application system and the data interaction condition among the application systems.

2. Data model parsing subsystem

2.1, a service model analysis algorithm unit: the system mainly comprises a service model analysis algorithm program, a data model generation algorithm program, rules for supporting algorithm operation and a storage module;

2.2, a data standard generation algorithm unit: the method mainly comprises a data standard generation algorithm program, and a rule and data standard reference library for supporting the operation of an algorithm;

2.3, a data integration analysis algorithm unit: the system mainly comprises an integrated API generation algorithm program for application system integration, an application system interface configuration library for supporting algorithm operation, and an API testing and storing module;

2.4, a data quality analysis algorithm unit: the method mainly comprises a data quality model generation algorithm oriented to different data types and quality judgment rule configuration supporting the operation of the algorithm.

3. Data governance execution subsystem

3.1, a data management workflow unit: the method mainly comprises the steps of task circulation of processes such as application, audit, change and the like in the data management execution process;

3.2, a data cleaning platform: the method mainly comprises a tool kit required by historical data cleaning in the data governance execution process;

3.3, a data exchange platform: the method mainly comprises the steps of data exchange among application systems in the data governance execution process, including data acquisition and distribution;

3.4, scheduling a data quality monitoring task: the method mainly comprises the steps of data quality monitoring and task scheduling in the data management execution process;

3.5, a data quality analysis platform: the method mainly comprises the automatic execution of a data quality analysis task in the data governance execution process.

The core system of the invention completes the communication between the business model and the enterprise data management in the data model analysis subsystem. In the data governance execution subsystem, the main work is to realize the model-driven enterprise data governance, and the specific functional modules in the subsystem are similar to the industry data governance execution system;

in this embodiment, the data management system based on the service metadata sets up a data management system of an enterprise from a service source of data generation, unifies service management and data management, and can fundamentally solve the problem of 'two-layer skin' of enterprise service management and data management, so that enterprise data flow is more suitable for enterprise service operation, and enterprises can better utilize a digital technology to improve enterprise operation;

the data management system based on the service metadata framework identifies data of cross-service departments through a service model, and gets through different interpretations of the same data due to different services through analysis of reference relations of the data among service flows, so that the data can better realize cross-department circulation;

Based on data governance driven by service metadata, identifying data of cross-application systems through a service model, identifying the data of the cross-application systems from a service angle, and communicating different coding and storage modes of different application systems for the same data to realize the unification of enterprise data views;

the data management based on the service metadata drive can better adapt to the development of enterprise services and the change of service management rules. The new business mode and management rule of the enterprise can be configured by a business architect, and can be timely updated to a data management system, and a new data standard, data integration and data quality management mode is automatically generated through an intelligent business and data model analysis algorithm, so that the data management work of the enterprise is greatly reduced;

based on the data management driven by the service metadata, according to the configured data quality rule, the influence of historical data on the current service operation can be analyzed in real time, the historical data is cleaned according to the data value, the low-value historical data is removed, and the work of cleaning the historical data is greatly reduced.

In another embodiment, as shown in fig. 2 to 4, an enterprise data governance method based on business metadata driving is proposed, and a data governance process of the present invention is composed of 3 sub-processes, which respectively are: the system configuration sub-process, the business model analysis sub-process, the data model analysis sub-process and the data model application sub-process. The method comprises the following specific steps:

Step S101, a system configuration sub-process:

the method comprises the steps of user login, business model importing, business rule configuring, business model analysis rule configuring, main data model importing, main data analysis rule configuring, data standard model rule configuring, data standard auditing and releasing, data quality model rule configuring, data quality analysis and scheduling configuring, business and application system relation configuring and system data exchange interface configuring, and specifically comprises the following steps:

user login: the user logs into the system using a username and password.

Importing a business model: and the service architect imports the service model into the system on the interface, and the system stores the service model into a system database for a service model analysis algorithm to use. The business model comprises a business layering model, a business scene connection model, a business post mapping model and a business application system relation model. The business hierarchical model takes the enterprise value chain as a root node and adopts tree structure description, and each node of the tree structure describes a business object corresponding to an enterprise process. The information transfer rules in the hierarchical model are as follows:

A. in the hierarchical model, all information flows from left to right (information flow of the reverse business process is implemented in the business scenario connection model);

B. The same-layer nodes under the same father node with service connection directly transmit information through connection;

D. the nodes of different father nodes gather information to upper nodes step by step and upwards, and information transmission across the father nodes is completed through information gathering transmission of the upper nodes;

D. the information is generated at the bottom-level child node, and the father node undertakes information aggregation and transmission.

Specifically, the service scene connection model describes a connection relationship between nodes, including a trigger condition for the nodes to transmit information and the content of the transmitted information; the business post mapping model describes post information for executing corresponding business processes in an enterprise, wherein the post information comprises organization departments and employee information to which posts belong; the business application system relation model describes an application system where a corresponding business process is executed, wherein the application system comprises data content and a storage mode recorded for the business process in the business system.

Configuring a business rule: and the service architect configures service rules on the interface, and the system stores the service rules into a system database. The business rules are configured on the nodes of the layered business model and are divided into flow rules and data rules. The process rule sets the conditions of the node connection relationship, for example, a contract exceeding a specified amount must be approved by a certain management level, and the connection relationship with the rule condition is established between a contract generation process node and a contract review process node. The data rule sets the necessary conditions of the node information, for example, the client registration needs to fill in the information of client contacts, the necessary conditions of the corresponding node are checked after the client registration is completed, and the information can be transmitted to the subsequent node only after the conditions are met.

Configuring a business model analysis rule:

importing a main data model: and the data management engineer imports the main data and the basic data model into the system on an interface, and the system stores the main data and the basic data model into a system database for a service model analysis algorithm unit to use.

Configuring data standard model rules: and the data standard model is configured on the interface by a data management engineer, and the data standard model and the matching rule thereof are stored in a system database by the system for the data standard generating algorithm unit to use.

And (3) auditing and issuing data standards: and the data management engineer initiates a process of auditing and issuing the data standard on the interface, and corresponding managers perform auditing and issuing on the interface.

Configuring a data quality model rule: and the data quality model is configured on the interface by a data management engineer, and the data quality model and the matching rule thereof are stored in a system database by the system for the data quality judgment algorithm unit to use.

And (3) configuring data quality analysis and scheduling: the data quality engineer configures parameters of a data quality analysis task (including configuring a data source, selecting a quality analysis model, a data quality grade and task scheduling time) on the interface, and can directly start the data quality analysis task or set the task starting time through the interface.

Configuring service and application system relationship: and a system integration engineer configures the corresponding relation between the business layering model and the application system on an interface, and the system analyzes the input information to form a business application system relation model for a data integration analysis algorithm.

Configuring an application system data exchange interface: and configuring a data exchange interface supported by the application system on the interface by a system integration engineer, wherein the data exchange interface comprises an HTTP API, a database connection and a third party definition interface, and configuring corresponding parameters for each interface mode. The system saves the interface capability of the application system to a database for the data integration analysis algorithm to use.

Step S102, a business model analysis sub-process:

the business model analysis sub-process of the invention is shown in fig. 3, and comprises 6 steps of loading a main data model, analyzing a business layering model, analyzing business scene connection, analyzing business post mapping, analyzing business application system relationship, generating a data model and the like, and specifically comprises the following steps:

loading a main data model: and loading the main data model imported by the data management engineer according to the request of the configuration subsystem.

Analyzing the business layering model: the algorithm loads a business hierarchical model at first, performs depth-first traversal on a hierarchical model tree according to the sequence, and constructs a business object tree from bottom to top. And traversing each node, reading the service model of the corresponding node, calling a service model analysis rule, and creating a service object and an attribute set of the node. For each attribute of the business object, if the attribute is generated for the business object, generating an attribute field name, a type and a unit; if the attribute comes from the calling of the main data, the service object is judged which main data information is quoted, the index of the corresponding main data object is stored, if the attribute comes from the quoting of other service objects, the quoting type and the quoting mode are judged, then the data connection between the front and the back service objects is judged, and a connection object is created.

Analyzing a service scene connection model: and the algorithm loads a service connection scene model, performs depth-first traversal on the service connection creation model tree according to the preamble, and constructs a service object scene graph based on the service object tree from top to bottom. Traversing each node in the service scene connection model, reading the service scene rule of the corresponding node, analyzing the rule to construct a service scene judgment expression, binding the service object attribute and the scene judgment expression, and establishing indirect data connection between the service objects.

Analyzing the service post mapping: the algorithm loads the mapping relation of the service posts, traverses the service scene connection diagram for each post, and finds out the service object set related to the post and the information (service connection object) output by the service objects. And then, according to the posts and the hierarchical association of the organization, the association of the organization and the business objects, the business object associated with the business process under each organization are constructed, and the business objects which cross a plurality of organizations and the business objects which undertake information transmission among the organizations are identified for the use of a data quality model generation algorithm.

Analyzing the relation of the service application system: and (4) loading the mapping relation between the service and the application system by the algorithm, and finding out a service object set related to each application system. Then, business objects across multiple application systems and business objects that host information transfer between application systems are identified for use by the data integration model and the data quality model generation algorithm.

Generating a data model: the algorithm generates a system data model. The data model comprises four modules, business object definition, a relation graph among business objects, a distribution graph of data among organizations and a distribution graph of data among application systems. And completing construction of business objects based on the analysis of the layering model and the connection model, and completing construction of a relation graph between the business objects based on information transmission and reference relation between the objects. And (4) completing construction of a distribution graph of the data among the organizations based on analysis of the business post mapping relation. And completing construction of a distribution diagram of the data among the application systems based on the relation analysis of the service application system.

Step S103, data model analysis sub-process:

the data model analysis sub-process of the present invention is shown in fig. 4, and includes 4 steps of loading a main data and service data model, generating a data standard model, generating a data integration model, and generating a data quality model, specifically:

loading a main data and business data model: and loading the main data and the basic data model configured by the data management engineer and loading the data model generated in the business model analysis sub-process according to the request of the configuration subsystem.

Generating a data standard model: the data standard model generation algorithm is divided into two aspects, namely main data standard generation and service data standard generation. The data standard generation flow of the main data is as follows:

1) Loading a business object connection graph, traversing the business objects, calling the main data according to the business objects, and associating the business rules with the corresponding main data attributes if the business object attributes have bound business rules;

2) Traversing all the main data, sequentially generating a data standard of each attribute of the main data, and if the attribute is associated with the service rule, generating the data standard according to the configured service rule; and if the attribute is the encoding attribute, generating the data standard according to the encoding rule configuration, otherwise generating the data standard according to the data standard reference library.

The data standard generation process of the service data comprises the following steps: loading a business object connection graph, traversing all business objects, and for each attribute of the business objects:

1) If the data is newly generated for the business process:

1.1 If associated with a business rule, generating its data standard according to the configured business rule;

1.2 Otherwise, generating the data standard according to the data standard reference library;

2) If the attribute is the attribute of the quoted main data object, the data standard is the attribute value under the unique identifier of the main data object bound by the data standard;

3) If the service object is the attribute of the other referenced service object;

3.1 Data criteria are data criteria for attributes of the referenced business object if the reference is not followed by a change in other business objects;

3.2 Data criteria is the value of an attribute under the unique identification of the business object to which it is bound if the reference is followed by changes to other business objects.

Generating a data integration model: the algorithm generates a corresponding data integration model for each application system. The process is as follows:

1) Loading service objects which are identified by a service model analysis algorithm and cross the application systems and are used for bearing information transmission between the application systems;

2) Loading an inter-application system data distribution map in the data model;

3) Traversing each application system configured by the system integration engineer, for each application system:

3.1 Finding out the business object associated with the current application system in the step 1), and if the data source of the business object attribute is not in the application system, finding out the dependent data source through a distribution diagram;

3.2 Collecting and merging according to the data sources, and confirming the data needing to be integrated from each data source;

3.3 According to the service flow type of the cross-application system, determining a data integration triggering mode, if the periodic service flow adopts a timing integration mode, and if the operation type service flow adopts an immediate on-demand integration mode according to operation;

3.4 ) generating a data integration interface according to an integration mode supported by the data source application system configured by the system integration engineer.

Generating a data quality model: the data quality model generation algorithm is also divided into two aspects, namely, a main data quality model and a service data quality model are generated.

The generation algorithm flow of the main data quality model is as follows:

1) Loading a master data object standard model;

2) Traversing all the main data objects;

2.1 For each attribute of the object, adding an attribute validity rule according to a data standard corresponding to the attribute;

2.2 For a primary data object, adding an object completeness rule according to a corresponding business rule;

2.3 The question level of the quality rules is set according to the data quality model rules configured by the data governance engineers.

3) And adding the quality rules to the data quality analysis task according to the data objects.

The generation algorithm flow of the quality model of the service data is as follows:

1) Loading data standards of all service data;

2) Loading a relation graph between business objects in a data model;

3) Traversing all business data objects:

3.1 For each attribute of the object;

3.1.1 If the data is generated, adding an attribute validity rule according to the data standard of the attribute;

3.1.2 If the data is the referenced data, adding an attribute consistency rule according to the reference type;

3.1.3 According to the relation graph among the business objects, confirming the dependency of other business objects on the attribute, and adding an attribute completeness rule;

3.2 Setting a problem level of a quality rule of a business object according to a data quality model rule configured by a data governance engineer;

4) Quality rules are added to the data quality analysis task by data object.

Step S104, a data model application sub-process:

the data model application sub-process of the invention is shown as the fourth sub-process in fig. 2, and comprises 13 steps of loading a data standard model, formulating a data cleaning rule, loading a data quality model, analyzing historical data quality, formulating a historical data cleaning range, configuring a data cleaning platform, loading a data integration model, configuring a data exchange platform, integrating system data, loading a data quality model, monitoring configured data quality, analyzing data quality, tracking data quality analysis results and the like, wherein the core comprises formulating a data cleaning rule, configuring a data exchange platform and monitoring data quality. The method specifically comprises the following steps:

formulating a data cleaning rule: the data cleaning mainly aims at the data quality problem in the historical data to complement and correct the historical data, but different historical data have different business values for enterprise management, so the data cleaning rule cleans according to the business values of the data

The data cleaning rule is formulated based on a business object, and the process comprises the following steps:

1) Loading a relation graph between business objects in a data model;

2) Analyzing business object data in the historical data;

2.1 If all the current active business objects have no correlation with the historical business object data, cleaning is not needed, otherwise, the business objects are classified according to the data quality grade;

3) According to the data quality grade, historical data cleaning is carried out from high to low, and the historical data with low grade does not need cleaning.

Configuring a data exchange platform: and configuring the data exchange platform according to the data integration model generated in the data model analysis sub-process.

Monitoring the quality of the configuration data: and analyzing the data quality model generated in the sub-process according to the data model, and configuring the data quality monitoring platform.

The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand the invention for and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims (10)

1. The enterprise data management system based on the service metadata drive is characterized by comprising a data management interface, a data model analysis subsystem and a data management execution subsystem;

the data management interface is used for providing corresponding operation interfaces for a service architect, a data management engineer and a system integration engineer, providing functions such as user login, service model call-in, service model rule configuration and the like for the service architect, providing functions such as data model management, data standard verification and release, data quality management, analysis result tracking and the like for the data management engineer, and providing functions such as service and application system relationship configuration and application system data distribution viewing for the system integration engineer;

the data model analysis subsystem is used for providing analysis from a business model based on business metadata to the data model, generating an enterprise data model, generating a data standard, a data integration model and a data quality model facing enterprise data management through the business model and the data model, and controlling the data management execution system to execute a specific data management task, wherein the data standard, the data integration model and the data quality model comprise an API (application program interface) for data integration and a data quality analysis task in system integration;

The data management execution subsystem comprises a data management workflow, a data cleaning platform, a data exchange platform and a scheduling and monitoring system for data quality analysis.

2. The business metadata driven-based enterprise data governance system according to claim 1, wherein the data governance system management interface unit comprises:

a user login module: the system is used for verifying the identity of the user, ensuring that the authorized user can log in the system and executing the corresponding operation of the authorized execution interface after logging in the system;

a business model importing module: the system is used for supporting a business architect to import a business architecture model output according to a specified format into the system by a visual interface;

a business rule configuration module: the system is used for supporting a business architect to configure business rules for the business scenes of all the business process sections by using a visual interface;

a data model management module: the system is used for supporting a data administration engineer to manage the data model generated by the system by using a visual interface;

and (3) data standard auditing and publishing: the system is used for auditing and releasing the specified data standard of a data administration engineer and managing a data standard reference library by using a visual interface;

The data quality management and analysis result tracking module: the system is used for supporting a data management engineer to configure and schedule a data quality analysis task by a visual interface, checking a system data quality analysis result and tracking a data quality improvement task;

a service and application system relation configuration module: the system is used for supporting the cooperation of a system integration engineer and a business architect by a visual interface, configuring the corresponding relation between a business module and an application system and applying an interactive interface between the systems;

the application system data distribution viewing module: the visual interface is used for supporting a system integration engineer to view the life cycle of data in each application system and the data interaction condition among the application systems.

3. The business metadata driven-based enterprise data governance system of claim 1, wherein the data model parsing subsystem comprises:

a service model analysis algorithm unit: the system comprises a business model analysis algorithm program, a data model generation algorithm program, rules for supporting algorithm operation and a storage module;

a data standard generation algorithm unit: the method comprises a data standard generation algorithm program, and a rule and data standard reference library for supporting the operation of an algorithm;

A data integration analysis algorithm unit: the system comprises an integrated API generation algorithm program for application system integration, an application system interface configuration library for supporting algorithm operation, and an API testing and storing module;

data quality analysis algorithm unit: the method comprises a data quality model generation algorithm facing different data types and quality judgment rule configuration supporting the operation of the algorithm.

4. The business metadata driven-based enterprise data governance system of claim 1, wherein the data governance execution subsystem comprises:

data management workflow unit: the method comprises the steps of task circulation of application, audit and change in the data management execution process;

a data cleaning platform: the method comprises the steps of including a tool kit required by historical data cleaning in the data governance execution process;

a data exchange platform: the method comprises the steps of data exchange among application systems in the data governance execution process, including data acquisition and distribution;

scheduling a data quality monitoring task: the method comprises the steps of data quality monitoring and task scheduling in the data management execution process;

a data quality analysis platform: the method comprises the automatic execution of a data quality analysis task in the data governance execution process.

5. The enterprise data management method based on the business metadata drive is characterized by comprising the following steps of:

Step S101, a system configures a sub-process;

step S102, analyzing a sub-process by a service model;

step S103, a data model analysis sub-process;

step S104, the data model application sub-process.

6. The method for governing enterprise data based on business metadata driving of claim 5, wherein in step S101, the system configuration sub-process specifically comprises:

user login: a user logs in the system through a user name and a password;

importing a business model: a service architect imports a service model into the system on an interface, and the system stores the service model into a system database for a service model analysis algorithm to use;

configuring a business rule: a service architect configures service rules on an interface, and the system stores the service rules into a system database; the business rules are configured on the nodes of the layered business model and are divided into flow rules and data rules;

configuring a business model analysis rule;

importing a main data model: the data management engineer imports the main data and the basic data model into the system on an interface, and the system stores the main data and the basic data model into a system database for a service model analysis algorithm unit to use;

Configuring data standard model rules: a data standard model is configured on an interface by a data management engineer, and the data standard model and the matching rules thereof are stored in a system database by the system for a data standard generation algorithm unit to use;

and (3) auditing and issuing data standards: a data management engineer initiates a process of auditing and releasing data standards on an interface, and corresponding managers perform auditing and releasing on the interface;

configuring a data quality model rule: a data quality model is configured on an interface by a data management engineer, and the data quality model and the matching rules thereof are stored in a system database by the system for a data quality judgment algorithm unit to use;

and (3) configuring data quality analysis and scheduling: a data quality engineer configures parameters of a data quality analysis task on an interface, or directly starts the data quality analysis task through the interface or sets the starting time of the task;

configuring the relation between the service and the application system: a system integration engineer configures the corresponding relation between the business layering model and the application system on an interface, and the system analyzes input information to form a business application system relation model for a data integration analysis algorithm;

configuring an application system data exchange interface: and configuring a data exchange interface supported by the application system on the interface by a system integration engineer, wherein the data exchange interface comprises an HTTP API, a database connection and a third party definition interface, configuring corresponding parameters for each interface mode, and storing the interface capability of the application system in a database for a data integration analysis algorithm.

7. The business metadata driven-based enterprise data governance method according to claim 6, wherein the business model comprises:

a business layering model: taking the enterprise value chain as a root node, and describing by adopting a tree structure, wherein each node of the tree structure describes a business object corresponding to an enterprise process;

wherein, the information transmission rule in the hierarchical model is as follows:

step A: in the hierarchical model, all information flows from left to right;

and B: the same-layer nodes under the same father node with service connection directly transmit information through connection;

and C: the nodes of different father nodes gather information to upper nodes step by step and upwards, and information transmission across the father nodes is completed through information gathering transmission of the upper nodes;

step D: the information is generated at the child node at the bottom layer, and the father node undertakes information aggregation and transmission;

service scenario connection model: describing the connection relation among the nodes, including the triggering condition of the node for transmitting information and the content of the transmitted information;

the business position mapping model is as follows: describing post information for executing corresponding business processes in an enterprise, wherein the post information comprises organization departments and employee information to which posts belong;

service application system relationship model: the application system where the corresponding business process is executed is described, including data content and storage mode recorded for the business process in the business system.

8. The business metadata driven-based enterprise data governance method according to claim 5, wherein in step S102, the business model parsing sub-process comprises:

loading a main data model: loading a main data model imported by a data management engineer according to a configuration subsystem request;

analyzing the service hierarchical model: loading a business hierarchical model, performing depth-first traversal on a hierarchical model tree according to the later sequence, and constructing a business object tree from bottom to top; traversing each node, reading a corresponding node business model, calling a business model analysis rule, and creating a business object and an attribute set of the node; for each attribute of the business object, if the attribute is generated for the business object, generating an attribute field name, a type and a unit, if the attribute is from the calling of main data, judging which main data information is quoted by the business object, storing an index of the corresponding main data object, if the attribute is from the quoting of other business objects, judging the quoting type and the quoting mode, then judging the data connection between the front and back business objects, and creating a connection object;

analyzing a service scene connection model: loading a service connection scene model, performing depth-first traversal on a service connection creation model tree according to a preamble, and constructing a service object scene graph based on a service object tree from top to bottom; traversing each node in the service scene connection model, reading a service scene rule of the corresponding node, analyzing the rule to construct a service scene judgment expression, binding the service object attribute and the scene judgment expression, and establishing indirect data connection between the service objects;

Analyzing the service post mapping: loading a business post mapping relation, traversing a business scene connection graph for each post, and finding out a business object set related to the post and information output by the business objects; establishing association between organizations and business objects according to the posts and the hierarchical association of the organizations, establishing business objects associated with business processes under each organization, and identifying business objects across a plurality of organizations and business objects bearing information transmission among the organizations for a data quality model generation algorithm to use;

analyzing the relation of the service application system: loading a mapping relation between the service and the application systems, and finding out a service object set related to each application system; identifying business objects which span a plurality of application systems and business objects which are used for bearing information transfer between the application systems and used by a data integration model and a data quality model generation algorithm;

generating a data model: generating a system data model, the data model comprising four modules: defining business objects, drawing relation among the business objects, drawing a distribution diagram of data among organizations and a distribution diagram of the data among application systems; based on the analysis of the layering model and the connection model, the construction of business objects is completed, and based on the information transmission and reference relationship among the objects, the construction of a relationship graph among the business objects is completed; based on the analysis of the mapping relation of the service posts, completing the construction of a distribution diagram of the data among the organizations; and completing construction of a distribution diagram of the data among the application systems based on the relation analysis of the service application system.

9. The business metadata driven-based enterprise data governance method according to claim 5, wherein in step S103, the data model parsing sub-process comprises:

loading a main data and business data model: loading main data and a basic data model configured by a data management engineer and loading a data model generated in a business model analysis sub-process when a subsystem configuration request is required;

generating a data standard model: the data standard model generation algorithm is divided into two aspects, namely, generating a main data standard and generating a service data standard;

the data standard generation flow of the main data is as follows: loading a business object connection graph, traversing the business object, calling the main data according to the business object, and associating the business rule with the corresponding main data attribute if the business object attribute has a bound business rule; traversing all the main data, sequentially generating a data standard of each attribute of the main data, and if the attribute is associated with a service rule, generating the data standard according to the configuration service rule; if the attribute is a coding attribute, generating a data standard according to coding rule configuration, otherwise generating the data standard according to a data standard reference library;

the data standard generation process of the service data comprises the following steps: loading a business object connection graph, traversing all business objects, and for each attribute of the business objects:

If the data is newly generated in the business process: if the data standard is associated with the business rule, generating the data standard according to the configured business rule, otherwise, generating the data standard according to a data standard reference library;

if the attribute is the attribute of the quoted main data object, the data standard is the attribute value under the unique identifier of the bound main data object;

if the attributes are the attributes of other referenced business objects: if the referenced business object is not changed along with other business objects, the data standard is the data standard of the attribute of the referenced business object; if the reference is followed by the change of other business objects, the data standard is the attribute value under the unique identification of the bound business object;

generating a data integration model: the algorithm generates a corresponding data integration model for each application system, and the flow specifically comprises the following steps: loading service objects which are identified by a service model analysis algorithm and cross the application systems and are used for bearing information transmission between the application systems; loading an inter-application system data distribution map in the data model; traversing each application system configured by the system integration engineer;

generating a data quality model: the data quality model generation algorithm is also divided into two aspects, namely, a main data quality model and a service data quality model are generated.

10. The method for governing enterprise data based on business metadata driving of claim 5, wherein in step S104, the data model application sub-process is specifically as follows:

formulating a data cleaning rule: the data cleaning is to complement and correct the historical data aiming at the data quality problem in the historical data;

configuring a data exchange platform: according to the data integration model generated in the data model analysis sub-process, configuring the data exchange platform;

monitoring the quality of the configuration data: and analyzing the data quality model generated in the sub-process according to the data model, and configuring the data quality monitoring platform.

Images