CN111382205A

CN111382205A - CMDB-based external data reconciliation method

Info

Publication number: CN111382205A
Application number: CN202010198613.6A
Authority: CN
Inventors: 刘向东
Original assignee: Guangzhou Ziyuan Information Technology Co ltd
Current assignee: Guangzhou Ziyuan Information Technology Co ltd
Priority date: 2020-03-20
Filing date: 2020-03-20
Publication date: 2020-07-07

Abstract

The invention belongs to the technical field of CMDB, and relates to synchronous management of CMDB external data; the CMDB external data is filtered and identified by using a CMDB data identification technology, and the identified data updates the external data to the CMDB according to a data reconciliation rule, a priority rule and a renewal rule. The invention can intensively configure the data identification rules of different data sources, and ensure the uniqueness of CMDB instance data through the processing of the data identification logic of the unified reconciliation engine service; the method can realize centralized configuration of data reconciliation rules of different data sources, and ensure the accuracy of CMDB instance data through the processing of data reconciliation logic of unified reconciliation engine service; and authorizing the updating of the null value attributes in different data source reconciliation rules, and ensuring the integrity of the CMDB instance data through authorizing the updating of the null value attributes of the data reconciliation logic served by the uniform reconciliation engine.

Description

CMDB-based external data reconciliation method

Technical Field

The invention belongs to the technical field of CMDB, and relates to synchronous management of CMDB external data. The CMDB data recognition technology is used for realizing filtering recognition of CMDB external data, and the recognized data updates the external data to the CMDB according to a data reconciliation rule, a priority rule and a renewal rule, in particular to a CMDB-based external data reconciliation method.

Background

An Information Technology Infrastructure Library (ITIL) describes a set of integrated management processes and best practices for the IT service management field, and with the development and perfection of many years, the ITIL becomes the actual industry standard in the IT service management field. And the Configuration Management Database (CMDB) is used as a basis for constructing an ITIL process and best practices, and stores and manages various configuration information (CI instances) of the equipment in the IT architecture of the enterprise and the incidence relation (CI instance relation) among the configurations. As a data provider of IT business service, the construction quality of the CMDB directly influences the success or failure of enterprise IT service management;

with the intensive degree and complexity of enterprise IT and the continuous development of the Internet of things and cloud technology, the enterprise IT scale is larger and larger, the flexibility is higher and higher, and the operation and maintenance difficulty is higher and higher. The traditional passive operation and maintenance is gradually changed to the active automatic operation and maintenance, IT does not serve business but drives the development of business, and CMDB also becomes the foundation for the automatic operation and maintenance in the digital IT service age. How to keep the CMDB highly consistent with the IT environment and how to ensure the uniqueness, accuracy and integrity of the CMDB configuration information brings great challenges to enterprise IT service management.

(1) Uniqueness of configuration information

The data source of the CMDB configuration information can be automatically discovered by an automatic discovery tool, the data synchronization of a third-party system, the import of external data and the manual creation in the system. In order to ensure the uniqueness of CI instance data, the automatic discovery tool generates an ID of the automatic discovery data when storing the CI instance data, and updates the data by taking the ID as a unique identifier; the third-party system data synchronization and external data import are implemented by performing CI instance query in the CMDB according to preset identifiable and unique field column data in a synchronization and import program module, and updating the CI instance data by external data if the CI instance data is queried; if the CMDB can not be matched with the data, newly building external data into the CMDB; the CI instance is manually created in the system, generally, only when the CI instance is saved, whether the current saved field value exists is checked according to the field set by saving, if the current saved field value does not pass the field set by saving, a prompt is given, and when the current saved field value passes the field set by saving, the newly created CI instance data is saved.

(2) Accuracy of configuration information

Due to the diversity of data sources of configuration information, for a piece of CI instance information, different data sources may all hold data related to the CI instance, and the trusted data of the CI instance attribute may come from different data sources respectively. In this case, a common solution is to determine the field attributes that can be put in storage from different data sources by an offline method, and then put in storage in respective storage programs. Or directly taking the source data which is finally put in storage as the latest data of the CI example.

(3) Integrity of configuration information

In the CMDB configuration model (CI model) level, different enterprises have inconsistent requirements on CI class management granularity, and the customization of the CMDB configuration model realizes the customized extension of the configuration model and the CI attributes. In the CI instance data plane, the null value of the attribute field directly affects the data integrity of the configuration information of the current CI. The data integrity is usually determined by determining whether the current attribute value is allowed to be empty according to convention when storing the CI instance data.

The prior art solutions have the following disadvantages in terms of ensuring uniqueness, accuracy, integrity of the CI instance data of the CMDB. The concrete points are as follows:

(1) aspect of uniqueness

Multiple data sources need to respectively set a field with unique identification, and the identification fields need to be configured or cannot be flexibly configured through respective functional modules;

if the data source relates to a plurality of different manufacturers, the determination of the unique field needs to be agreed by the plurality of manufacturers, and the coordination and communication cost is higher;

when a new data source is added, the expansibility is poor, and the logic processing of data uniqueness needs to be redesigned independently according to the added data source or the realization of recoding;

(2) aspect of accuracy

The credible data source attribute is determined and adjusted inflexibly or cannot be adjusted, and is limited by the design of a data synchronization function of a data source;

(3) aspect of integrity

The control mode of data integrity is not flexible and is limited by the design of data synchronization function of a data source side.

The proposal aims to provide a data mediation technology, a unified CMDB data warehousing interface is provided through mediation engine service, CMDB data warehousing processing logic is isolated from the outside through unified identification and mediation configuration, a priority rule and a renewal rule are combined, external data only need to be transmitted into original data, and the only accuracy and integrity of CMDB data are ensured by mediation engine service.

Disclosure of Invention

The invention aims at the problems and provides a CMDB-based external data reconciliation method.

In order to achieve the aim, the invention adopts the technical proposal that,

a CMDB-based external data reconciliation method sequentially comprises two steps of data identification and data reconciliation,

the data identification comprises the configuration of identification rules and data identification flow processing;

the identification rule is configured to configure unique identification attributes of CI instances for different data sources aiming at a specified CI model, the identification attributes are multi-attribute combinations, multiple identification attribute combinations can be configured for the same data source of the current configuration model, the priority needs to be configured for the combination of the multiple identification attributes, and the CI instances can be matched according to the attribute combination with high priority in the data identification process processing logic;

the data identification flow processing steps are as follows:

1) the data identification processing flow firstly carries out compliance verification on the received current data, and error-reporting prompt or direct discarding is carried out on the source data which is not qualified in verification;

2) after the data verification is passed, matching whether a corresponding identification rule exists according to the received data source and the CI class, when the current CI class is not configured with the identification rule, inheriting the identification rule of a parent class of the CI class for matching, and if the CI class inherits no identification rule, directly storing the source data into a CMDB;

3) according to configured identification rules and priorities, from high priority to low priority, carrying out one-by-one matching of CI instances in a CMDB by using received source original data, if the CI instances cannot be matched, newly adding the current source data into a warehouse, if a plurality of source original data are matched, prompting that the CI instances are not unique, and if the unique CI instances are matched, carrying out identification;

4) according to the recognized data condition, the recognized CI classes can be self classes, father classes and brother classes, and the system enters a logic part of a reconciliation flow after recognizing the identification;

the data reconciliation comprises the configuration of reconciliation rules and the processing of data reconciliation flow;

the configuration reconciliation rule of the reconciliation rule is configured with a data source component which is authorized to update the CI attribute list for different data sources aiming at the specified CI model, and is authorized for the attribute which is specified to allow the updating of the null value, the reconciliation rule configured with a plurality of data sources also creates a priority rule of the reconciliation data, and can also define the frequency of updating the CI attribute of the specified data source by configuring a renewal rule;

the data reconciliation flow process is to start the reconciliation update flow by the reconciliation engine service according to the source data which needs to be updated and identified by the identification rule.

Preferably, the data reconciliation flow process includes the following steps:

1) firstly, matching whether the attribute configures a reconciliation rule in the reconciliation rule one by one according to the attribute of the identified source data, and if the attribute does not configure the reconciliation rule, updating the current attribute value of the current source data into a CMDB library;

2) the current process processing node firstly searches a data source with updated attribute according to the attribute of the identified source data;

3) the current process processing node matches the priority of the data source updated last time according to the attribute of the identified source data, and if the priority of the data source updated last time is lower than or the same as the priority of the source at this time, the attribute value of the source data at this time is allowed to be updated into the CMDB;

4) when CMDB data is updated, the reconciliation engine service can use null value update according to field authorization which is configured in the reconciliation rule and specifies that the null value is allowed to be updated, when the source data attribute value is empty, the null value is adopted for updating, the field which is not specified to allow the null value to be updated is not specified, and when the source data attribute value is empty, the null value is not updated.

Compared with the prior art, the invention has the advantages and positive effects that,

1. by the method, the data identification rules of different data sources can be configured in a centralized manner, and the uniqueness of the CMDB instance data is guaranteed through the processing of the data identification logic of the unified reconciliation engine service;

2. by the method, the centralized configuration of the data reconciliation rules of different data sources can be realized, and the accuracy of the CMDB instance data can be ensured by processing the data reconciliation logic of the unified reconciliation engine service;

3. by the method, updating authorization of null value attributes in different data source reconciliation rules can be realized, and the integrity of CMDB instance data can be ensured by authorizing updating the null value attributes of data reconciliation logic of a uniform reconciliation engine service.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.

FIG. 1 is a flow chart of a data identification process in a CMDB-based method of reconciling external data;

FIG. 2 is a flow chart of a reconciliation update process for data in a CMDB-based method for external data reconciliation.

Detailed Description

In order that the above objects, features and advantages of the present invention can be more clearly understood, the present invention will be further described with reference to the accompanying drawings and examples. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and thus the present invention is not limited to the specific embodiments of the present disclosure.

Embodiment 1, as shown in fig. 1 and fig. 2, the present invention provides a CMDB-based method for performing external data reconciliation, which sequentially includes two steps of data identification and data reconciliation.

The following two steps are specifically described:

1. data identification

Data identification is the key processing logic to ensure the uniqueness of CMDB data. The data identification service is divided into the configuration of identification rules and the data identification flow processing.

(1) The configuration of the rules is identified.

The identification rule is a key attribute for configuring the uniqueness of the CI instance for different data sources according to a specified CI model. The identification attribute can be a combination of multiple attributes, multiple identification attribute combinations can be configured for the same data source of the current configuration model, priorities need to be configured for the multiple identification attribute combinations, and CI instance matching can be performed according to the attribute combination with the high priority in the data identification flow processing logic. The identification rules support the inheritance of model identification rules, i.e. the identification rules of a sub-class model automatically inherit the identification rules of a parent class.

(2) Data identification flow processing

According to the definition of the unified warehousing interface, the data received by the reconciliation engine service needs to include: data source information, configuration class names, each attribute value in storage and the like. When the reconciliation engine service receives the raw data through the unified warehousing interface, it starts the data identification process (as shown in fig. 1).

1) The data identification processing flow firstly carries out compliance verification on the received current data. Carrying out error prompt or directly discarding for the source data which is checked to be unqualified;

2) after the data verification is passed, whether a corresponding identification rule exists is matched according to the received data source and the CI class, and when the identification rule is not configured in the current CI class, the identification rule of the parent class is inherited to carry out matching. If no identification rule is configured on the CI class inheritance tree, the source data can be directly put into the CMDB.

3) And according to the configured identification rule and priority, carrying out one-by-one matching of CI instances in the CMDB base by using the received source raw data from high priority to low priority. And if the CI instances are not matched, newly adding the current source data into a warehouse, and if a plurality of pieces of source data are matched, prompting that the CI instances are not unique. And if the unique CI instance is matched, performing identification.

4) According to the recognized data condition, the recognized CI classes can be self classes, father classes and brother classes, and after the system carries out recognition identification, the logic processing of the reconciliation flow is carried out.

2 data reconciliation

Data reconciliation is the key processing logic to ensure the accuracy and integrity of CMDB data. The data reconciliation service is divided into the configuration of reconciliation rule and the data reconciliation flow process.

(1) And (4) configuration of the reconciliation rule.

The reconciliation rule is configured with the data source component for different data sources for the specified CI model, with the authority to update the CI attribute list, and with the authority to specify attributes that allow null updates. Reconciliation rules configured with multiple data sources will also create precedence rules for reconciling data. The frequency with which a given data source should update CI attributes may also be defined by configuring update rules.

The reconciliation rules (including precedence rules, update rules) support the inheritance of CI model rules, i.e., the reconciliation rules of the subclass model will automatically inherit the reconciliation rules of the parent class.

(2) Data reconciliation flow processing

According to the identification rule, the reconciliation engine service starts a reconciliation update process for the source data identified as needing to be updated, and the reconciliation update process flow of the data is as shown in fig. 2:

1) firstly, the reconciliation updating process matches whether the reconciliation rule is configured for the attribute in the reconciliation rule one by one according to the attribute of the identified source data, if the reconciliation rule is not configured for the attribute, the current attribute value of the current source data is updated to the CMDB; if the attribute is configured with a reconciliation rule, if the data source of the configured attribute is different from the data source of the current identification, the attribute value of the current identification source is not allowed to be updated into the CMDB, if the data source of the reconciled attribute is the same as the source of the current identification data, the processing logic of updating the rule is entered;

2) the current process processing node firstly searches a data source with the updated attribute last time according to the attribute of the identified source data (if the attribute has no last update or the data source with the last updated attribute is the same as the current time, the identified source data attribute is directly updated into the CMDB), matches the validity period of the update rule according to the data source with the updated attribute last time (different from the current source), and allows the current source data attribute to be updated into the CMDB if the update date of the attribute last time and the current update time interval are not within the validity period of the update rule; if the updating rule is in the validity period of the updating rule or the updating rule is not configured in the last data source, the current processing flow enters the processing logic of the data priority rule;

3) the current process processing node matches the priority of the data source updated last time according to the attribute of the identified source data, and if the priority of the data source updated last time is lower than or the same as the priority of the source at this time, the attribute value of the source data at this time is allowed to be updated into the CMDB; if the priority of the last time is higher than the priority of the source at the time, the attribute of the source data at the time is not updated;

It should be noted here that the key points and points to be protected of the present invention are:

1) the data identification configuration can respectively configure identification rules according to different data sources, and the reconciliation engine service uniformly processes the data identification logic according to the identification rules when the CMDB data is newly added, so that the key point of the proposal is that protection is required.

2) In the data identification process, the reconciliation engine service matches the identification one by one with the configured high-priority attribute according to the priority defined in the data rule, and the identification is successful if the CMDB instance data is successfully matched by the high-priority attribute, which is a key point of the proposal and needs to be protected.

3) The data reconciliation configuration can respectively configure reconciliation rules, priority rules and renewal rules according to different data sources, and the reconciliation engine service can uniformly process data reconciliation logic according to the identified source data through the reconciliation rules, the renewal rules and the priority rules, which is the key point of the proposal and needs to be protected.

4) In the data reconciliation process, the reconciliation engine service determines whether the attribute of the current data source is updated to the CMDB base according to the reconciliation attribute defined in the data reconciliation rule in combination with the renewal rule and the priority rule configured by the current data source, which is the key point of the proposal and needs to be protected.

The above description is only a preferred embodiment of the present invention, and not intended to limit the present invention in other forms, and any person skilled in the art may apply the above modifications or changes to the equivalent embodiments with equivalent changes, without departing from the technical spirit of the present invention, and any simple modification, equivalent change and change made to the above embodiments according to the technical spirit of the present invention still belong to the protection scope of the technical spirit of the present invention.

Claims

1. A CMDB-based external data reconciliation method sequentially comprises two steps of data identification and data reconciliation, and is characterized in that,

the configuration of the recognition rule is to configure CI for different data sources according to a specified CI model

The identification attribute is a multi-attribute combination, a plurality of identification attribute combinations can be configured for the same data source of the current configuration model, and the combination of the plurality of identification attributes needs to be configured optimally

In the data identification flow processing logic, the CI instances are matched according to the attribute combination with high priority;

the data identification flow processing steps are as follows:

3) according to configured identification rules and priorities, from high priority to low priority, carrying out one-by-one matching of CI instances in a CMDB library by using received source original data, and if the CI instances are not matched to the CI instances

If a plurality of pieces of source data are matched, prompting that the CI example is not unique, and if the unique CI example is matched, identifying;

4) according to the identified data condition, the identified CI class can be self class, father class and brother class, and the system

After identification, entering a logic part of a reconciliation flow;

the configuration of the reconciliation rule for a given CI model, for different data sources, configures the data source component with authority to update the CI attribute list, and for the attribute authorization specifying permission for null value update, the reconciliation rule configured with multiple data sources will also create a priority rule for the reconciliation data, or

Defining the frequency of updating CI attributes of a specified data source by configuring a renewal rule;

2. The CMDB-based external data reconciliation method of claim 1, wherein the data reconciliation flow process specifically comprises the following steps: