Background
The enterprise owner data management is mainly used for managing metadata of various data in each business system in an enterprise, and generally includes storage locations of the data, adopted storage technologies and storage specifications, such as: tables, fields, and description information in the database, etc. Because various business data exist in an enterprise and are scattered in each business system, if a uniform standard management and control platform is not provided, the enterprise cannot conveniently search the required business data. When data is needed, the data is often sent to each service system, and the related data or related interfaces are obtained through the help of the service system implementation manufacturer. Because the maturity of each manufacturer is different and the adopted specific technology is different, the data in the enterprise is difficult to be effectively searched and used.
Aiming at the problems, the existing solutions mainly support the unified management and control of various heterogeneous data through methods such as an enterprise service bus, a data service bus and the like, and the main ideas are divided into two types: one is to store various heterogeneous service data on a main data platform after cleaning and converting the data, and then provide data service to the outside, which is called a data warehouse mode, and the other is a mode in which the main data platform does not store data and mediates a data user and a source system where the data is located by a mediation conversion mode, which is called a data hosting mode.
However, the above method focuses more on protocol conversion in the technical level, specification access of data and improvement of performance, and can play a good role in the technical level, and still has some disadvantages:
1. the data specification is still in the technical layer, and data has no more service attributes, such as data tables, fields, access modes and the like, which are all concepts and languages in the technical aspect, have certain technical requirements on users and need to have deeper understanding on the source service system.
2. There is no explicit concept of data objects and data directories or the concept is relatively simple, much in the face of interactions between systems, and no more business concepts assigned to it.
3. Although there are many additional control functions, such as security, the definition of rules is often cumbersome and requires translation of business rules into technical rules.
4. Under the scene that a national power supply large data information resource platform is used as a main data provider and source, along with the type and the quantity of main data, the related service fields are more and more, the traditional main data management method is more and more complex, and the manageability is more and more poor.
Disclosure of Invention
Aiming at the defects in the prior art, the business-oriented main data management method and the business-oriented main data management system are used for enhancing a data bridge between a connection business and a specific IT system, so that data in a main data platform can be more directly business-oriented, and mapping and translation of various rules between the systems are facilitated.
A main data management system facing to business comprises a domestic appliance big data information resource platform, a main data using end and a main data management platform,
the national appliance big data information resource platform is used for registering metadata to the main data management platform in an object registration mode;
the main data management platform is used for defining a business rule and managing a data object;
and the main data using end is used for acquiring and using data according to the service rule defined by the main data management platform.
Furthermore, the main data management platform is connected with the main data using end and the domestic appliance big data information resource platform through a data access module, the data access module is respectively connected with a data access statistical information module and a uniform data cache module,
the data access module is used for providing a uniform data format, a standard data interface and an access mechanism between the main data management platform and the main data using end and between the main data management platform and the domestic appliance big data information resource platform;
the data access statistical information module is used for storing statistical information of data access;
the unified data cache module is used for temporarily storing the data objects and setting storage time according to the levels of the data objects.
Further, the master data management platform comprises a data catalog management service module and a core module,
the core module is used for managing the data object according to a preset service rule and forming data directory information;
and the data directory management service module is used for packaging the data directory information formed by the core module into a data directory management service interface for business personnel to browse and use.
Further, the core module comprises a service attribute management unit, a data object association unit, a data object management unit, a data access mapping unit, a data authority management unit, a service dimension management unit and a service label management unit,
the service attribute management unit is used for defining service attributes and service attribute association rules;
the data object management unit is used for defining a multi-level data directory of the data object, managing the data object and defining a control rule of the data object according to the service attribute;
the service dimension management unit is used for defining and managing a plurality of service dimensions and maintaining the relationship between the data object and the service dimensions;
the service label management unit is used for defining and managing a service label and maintaining the relation between the data object and the service label;
the data authority management unit is used for defining a data authority control strategy, performing data access control according to a label on a data object and performing authority control according to a data directory and a data object level;
the data object association unit is used for defining association relations among the data objects and analyzing potential association relations among the data objects according to the service attributes and the labels on the data objects;
and the data object access mapping unit is used for completing the mapping between the metadata of the domestic appliance big data information resource platform and the service attribute of the data object.
A service-oriented master data management method comprises the following steps:
s01: defining a business concept in a master data management platform, the business concept comprising: five fields of a service space, a service attribute, a service code, a service dimension and a service label;
s02: registering metadata in the national power supply big data information resource platform into a main data management platform in an object data registration mode;
s03: the main data management platform acquires metadata of the domestic appliance big data information resource platform through the data access module;
s04: the main data management platform performs service attribute mapping between metadata and data objects through the data object access mapping module, and performs service rule association on the data objects;
s05: and the service personnel or the main data using end manages and operates the data object according to the data directory.
Further, the step S04 includes the following steps:
s0401: performing association through the service space and the service classification of the metadata and the data object to complete the service classification directory association between the metadata and the data object;
s0402: carrying out normalized coding on the metadata and the data directory where the data object is located;
s0403: the business attribute association between the metadata and the data object is completed by associating the metadata and the business field of the data object;
s0404: the association of the metadata and the data object business label is completed by printing the business label on the metadata and the data object;
s0405: the business dimension association between the metadata and the data object is accomplished by specifying the business dimension of the metadata and the data object.
According to the technical scheme, the invention has the beneficial effects that: a set of standard rules for hosting the main data and a standard service concept of the data object are defined through the main data management platform, and the data object is added to serve as a data bridge among the main data management platform, a main data using end and a national power supply large data information resource platform, so that data in the main data platform can be more directly oriented to services, and mapping and translation of various rules among systems are facilitated.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
Referring to fig. 1, the main data management system for business facing provided by this embodiment includes a domestic appliance big data information resource platform, a main data using end and a main data management platform,
the domestic appliance data information resource platform is used for registering metadata to a main data management platform in a registration mode, and comprises a plurality of types of main data providing systems, for example: data warehouses that register metadata to the master data management platform through services provided by the master data management platform;
the main data management platform is used for defining a business rule and managing a data object, wherein the business rule comprises a data object space, a business attribute, a business code, a business dimension and business label defining rule, a data object access rule and a data object association rule;
and the main data using end is used for acquiring and using data according to the service rule defined by the main data management platform.
In actual use, a main data management system facing to services is based on a data storage mode, a set of standard rules for storing main data and a standard service concept of a data object are defined through a main data management platform, and the data object is increased to serve as a data bridge among the main data management platform, a main data using end and a national power supply large data information resource platform, so that data in the main data platform can be more directly facing to services, and mapping and translation of various rules among the systems are facilitated.
In this embodiment, the main data management platform is connected to the main data user and the domestic appliance big data information resource platform through a data access module, the data access module is respectively connected to the data access statistical information module and the unified data cache module,
the data access module is used for providing a uniform data format and a standard data interface between the main data management platform and the main data using end and between the main data management platform and the national appliance big data information resource platform, updating the interface according to the service attribute of a data object, defining a data access mechanism, and supporting various storage engines in the national appliance big data information resource platform, such as a common relational database storage engine: MySQL, Oracle, the large capacity data warehouse storage engine of the national electric power delivery big data information resource platform: hive, HBase, sequential library storage engines Influxdb, PI, etc.
The data access statistical information module is used for storing statistical information of data access to form an access record library, so that a manager can conveniently perform hot spot analysis, correlation analysis and other operations.
The unified data cache module is used for temporarily storing the data objects, and setting storage time according to the levels of the data objects, so that the access rate of the data is improved.
In this embodiment, the master data management platform includes a data directory management service module and a core module,
the core module is used for managing the data objects and the value association relation between the data objects according to a preset service rule and forming data directory information, so that the authority of the data objects can be conveniently controlled, and a user can quickly position according to the data directory.
And the data directory management service module is used for packaging the data directory information formed by the core module into a data directory management service interface for business personnel to browse and use.
In this embodiment, the core module includes a service attribute management unit, a data object association unit, a data object management unit, a data access mapping unit, a data authority management unit, a service dimension management unit, and a service tag management unit,
the service attribute management unit is used for defining service attributes and service attribute association rules, so that the positioning, the query and the association analysis between the data objects and the service attributes are facilitated;
the data object management unit is used for defining a multi-level data directory of the data object, managing the data object, defining a control rule of the data object according to the service attribute and supporting the multi-level classified data directory.
The service dimension management unit is used for defining and managing a plurality of service dimensions and maintaining the relationship between the data object and the service dimensions;
the service label management unit is used for defining and managing a service label and maintaining the relation between the data object and the service label;
the data authority management unit is used for defining a data authority control strategy, performing data access control according to a label on a data object and performing authority control according to a data directory and a data object level;
the data object association unit is used for defining association relations among the data objects and analyzing potential association relations among the data objects according to the service attributes and the labels on the data objects;
and the data object access mapping unit is used for completing the mapping between the metadata of the domestic appliance big data information resource platform and the service attribute of the data object.
In practical use, based on the standard business concept, various data control units can control data authority according to the language definition rule of the business concept, such as authority control according to the hierarchy of business codes; various control modules can define rules according to the language of the service, such as controlling data authority according to the service space, controlling authority according to the segmentation of service codes, and the like; through unified business attribute management, a user can quickly know related business concepts of data, confusion is avoided, the actual meaning of the data represented by the data can be quickly known, the presentation form of the data is standardized, and potential association between data objects in a data directory is quickly mined; through standard business dimensions and business labels, data in enterprises are displayed, classified and aggregated, so that the data analysis method can play an important role in data analysis.
Referring to fig. 2, a service-oriented main data management method, which is developed and designed based on Java language and encapsulated based on a universal X86 server, a J2EE programming framework, and Restful service, includes the following steps:
s01: defining a business concept in a master data management platform, the business concept comprising: the method comprises the following steps that five fields including a service space, a service attribute, a service code, a service dimension and a service label are included, wherein the service space classifies data directory data objects from the aspect of service by defining a logic container; the service attribute has uniqueness as a target of authority control and must be Chinese; the business code has a unique identifier and is advanced according to the hierarchical structure of the data directory in form, the code of the superior directory is used as the prefix of the code of the inferior directory, and the directory code is used as the prefix of the code of the data object under the directory, so that the searching and the analysis of the data object on the potential business incidence relation are facilitated; the business dimension divides the business field into a plurality of dimensions, so that the data objects can be associated on the plurality of dimensions conveniently; the business label is used for marking the business attribute of the data object, so that the data object can participate in various loose aggregations conveniently.
S02: metadata in the national power supply large data information resource platform is registered in the main data management platform in a data registration mode, so that the main data management platform can conveniently access data of the national power supply large data information resource platform, for example, information such as an address, a database type, a user account number and a password of a national power supply large data information resource platform database is obtained.
S03: the main data management platform acquires metadata of the domestic appliance big data information resource platform through the data access module;
s04: the main data management platform performs business attribute mapping between the metadata and the data objects through the data object access mapping module, and performs business rule association on the data objects, so that specific rules of the data objects are conveniently and definitely implemented, and the flexibility and the convenience of the data object management method are enhanced.
S05: the service personnel or the main data using end manages and operates the data object according to the data directory, the main data platform processes and converts according to the configured service rule, for example, desensitizes the data, converts the format and precision of the data, and the data obtained by the service user is more standard and consistent.
Referring to fig. 3, the step S04 includes the following steps:
s0401: performing association through the service space and the service classification of the metadata and the data object to complete the service classification directory association between the metadata and the data object;
s0402: carrying out normalized coding on the metadata and the data directory where the data object is located;
s0403: the business attribute association between the metadata and the data object is completed by associating the metadata and the business field of the data object, so that business personnel can directly retrieve the data object through the business attribute.
S0404: the association of the metadata and the data object business label is completed by printing the business label on the metadata and the data object;
s0405: the business dimension association between the metadata and the data object is accomplished by specifying the business dimension of the metadata and the data object.
In actual use, the data objects are accessed, retrieved and queried conveniently from the business layer by associating the business classification, the business attribute, the business label and the business dimension between the metadata and the data objects and supplementing the business specification and the association relation of the data objects.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.