CN115550382A - Configuration item synchronization method, device, system and equipment - Google Patents

Configuration item synchronization method, device, system and equipment Download PDF

Info

Publication number
CN115550382A
CN115550382A CN202211214414.5A CN202211214414A CN115550382A CN 115550382 A CN115550382 A CN 115550382A CN 202211214414 A CN202211214414 A CN 202211214414A CN 115550382 A CN115550382 A CN 115550382A
Authority
CN
China
Prior art keywords
configuration item
item data
configuration
data
change event
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211214414.5A
Other languages
Chinese (zh)
Inventor
刘庆
来宾
王宇鹏
于子淇
林溢星
郭昊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Agricultural Bank of China
Original Assignee
Agricultural Bank of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agricultural Bank of China filed Critical Agricultural Bank of China
Priority to CN202211214414.5A priority Critical patent/CN115550382A/en
Publication of CN115550382A publication Critical patent/CN115550382A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a configuration item synchronization method, a device, a system and equipment, which are applied to the technical field of financial science and technology/financial data centers, and the method comprises the following steps: monitoring the change event of the configuration items in each cluster; acquiring first configuration item data corresponding to the change event, and performing aggregation processing on the first configuration item data to obtain second configuration item data; carrying out format conversion on the second configuration item data to obtain target configuration item data; and sending the target configuration item data to a configuration management database. Therefore, the configuration item data is obtained by monitoring the change event of each cluster configuration item, and the configuration item data is reported and stored after aggregation processing and format conversion, so that the change of the configuration item can be dynamically sensed, dynamic monitoring is realized, and meanwhile, the incremental reporting of the configuration item is carried out aiming at the change event, and the updating accuracy and the real-time performance of the configuration item are improved.

Description

Configuration item synchronization method, device, system and equipment
Technical Field
The present application relates to the technical field of financial technology/financial data centers, and in particular, to a method, an apparatus, a system, and a device for synchronizing configuration items.
Background
In the process of digital transformation, the use of cloud platforms is becoming more and more widespread in various financial enterprises. The current cloud platform gradually evolves towards the cloud protogenesis direction, and each financial enterprise gradually builds a container cloud platform as a service operating computing environment. The container cloud platform is an enterprise-level cloud platform capable of managing a plurality of Kubernets, container arrangement and scheduling are achieved through open-source Kubernets, the capacity of service agile release, elastic expansion and fault self-healing is provided, and the capacity of basic resources is greatly played.
However, the introduction of the kubernets technology brings great complexity in the field of configuration management, services of a container cloud platform are frequently released, resource delivery is usually second-level delivery, the life cycle of a container is short, and the attribute change of a configuration item is rapid.
In the related art, the configuration item is often collected by each host in a timed reporting manner, but due to the characteristics of the container cloud platform, the configuration item collection manner in the related art cannot dynamically sense the change of the configuration item, so that the accuracy and the real-time performance of updating the configuration item are poor, and the method is obviously not suitable for the container cloud platform.
Disclosure of Invention
The application provides a method, a device, a system and equipment for synchronizing configuration items, which can dynamically sense the change of the configuration items and improve the accuracy and the real-time performance of updating the configuration items.
In a first aspect, an embodiment of the present application provides a configuration item synchronization method, including:
monitoring the change event of the configuration items in each cluster;
acquiring first configuration item data corresponding to the change event, and performing aggregation processing on the first configuration item data to obtain second configuration item data;
carrying out format conversion on the second configuration item data to obtain target configuration item data;
and sending the target configuration item data to a configuration management database.
In a possible embodiment, the monitoring a change event of a configuration item in each cluster includes:
registering a monitoring function in an interface gateway in each cluster;
and monitoring the change event of the configuration items in the cluster through the monitoring function.
In a possible implementation manner, the aggregating the first configuration item data to obtain second configuration item data includes:
determining a change event corresponding to the same configuration item within a preset time threshold;
determining version information of a change event corresponding to the same configuration item according to the first configuration item data;
and in the change events corresponding to the same configuration item, determining the first configuration item information corresponding to the change event with the latest version information as the second configuration item data.
In a possible implementation manner, the performing format conversion on the second configuration item data to obtain target configuration item data includes:
determining a data format in the configuration management database;
and analyzing the second configuration item data and assembling the second configuration item data into the data format to obtain the target configuration item data.
In a possible embodiment, the sending the target configuration item data to the configuration management database includes:
and sending the target configuration item data to the configuration management database through a distributed message system Kafka.
In a second aspect, an embodiment of the present application provides a configuration item synchronization apparatus, including:
the monitoring module is used for monitoring the change event of the configuration items in each cluster;
the processing module is used for acquiring first configuration item data corresponding to the change event and performing aggregation processing on the first configuration item data to obtain second configuration item data;
the conversion module is used for carrying out format conversion on the second configuration item data to obtain target configuration item data;
and the sending module is used for sending the target configuration item data to a configuration management database.
In a possible implementation manner, the monitoring module is specifically configured to:
registering a monitoring function in an interface gateway in each cluster;
and monitoring the change event of the configuration items in the cluster through the monitoring function.
In a possible implementation manner, the processing module is specifically configured to:
determining a change event corresponding to the same configuration item within a preset time threshold;
determining version information of a change event corresponding to the same configuration item according to the first configuration item data;
and in the change events corresponding to the same configuration item, determining the first configuration item information corresponding to the change event with the latest version information as the second configuration item data.
In a possible implementation manner, the conversion module is specifically configured to:
determining a data format in the configuration management database;
and analyzing the second configuration item data and assembling the second configuration item data into the data format to obtain the target configuration item data.
In a possible implementation manner, the sending module is specifically configured to:
and sending the target configuration item data to the configuration management database through a distributed message system Kafka.
In a third aspect, an embodiment of the present application provides a configuration item synchronization system, including: a listener, a callback, and a translator; the listener is connected with the callback device; the callback device is connected with the converter; wherein the content of the first and second substances,
the monitor is used for monitoring the change event of the configuration items in each cluster;
the callback device is used for acquiring first configuration item data corresponding to the change event and carrying out aggregation processing on the first configuration item data to obtain second configuration item data;
the converter is used for carrying out format conversion on the second configuration item data to obtain target configuration item data;
the converter is further configured to send the target configuration item data to a configuration management database.
In a fourth aspect, an embodiment of the present application provides a configuration item synchronization apparatus, including: a processor, a memory;
the memory stores computer execution instructions;
the processor executes computer-executable instructions stored by the memory to implement the method of any one of the first aspect.
In a fifth aspect, embodiments provide a computer-readable storage medium having stored therein computer-executable instructions for implementing the method of any one of the first aspects when the computer-executable instructions are executed.
In a sixth aspect, the present application provides a computer program product comprising a computer program that, when executed, implements the method of any one of the first aspects.
The method, the device, the system and the equipment for synchronizing the configuration items provided by the embodiment of the application monitor the change events of the configuration items in each cluster; acquiring first configuration item data corresponding to the change event, and performing aggregation processing on the first configuration item data to obtain second configuration item data; carrying out format conversion on the second configuration item data to obtain target configuration item data; and sending the target configuration item data to the configuration management database. Therefore, the configuration item data is obtained by monitoring the change event of each cluster configuration item, aggregation processing and format conversion are carried out, and then reporting and storing are carried out, so that the change of the configuration items can be dynamically sensed, dynamic monitoring is realized, and meanwhile, incremental reporting of the configuration items is carried out aiming at the change event, and the accuracy and the real-time performance of updating the configuration items are improved.
Drawings
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;
fig. 2 is a schematic flowchart of a configuration item synchronization method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of another configuration item synchronization method according to an embodiment of the present application;
FIG. 4 is a block diagram illustrating a configuration item synchronization method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of processing logic of a configuration item synchronization method according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a configuration item synchronization apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a configuration item synchronization system according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a configuration item synchronization apparatus according to an embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present application, the present application is further described in detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments and figures described herein are merely illustrative of the application and are not limiting of the application. It should be emphasized that, in the technical solutions of the present application, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the related information, such as financial data or user data, are all in compliance with the regulations of the relevant laws and regulations, and do not violate the customs of the public order.
Data center clouding is a necessary way for enterprises such as financial banks to carry out digital transformation at present, and the data center of the mainstream financial bank at present realizes the management of large-scale servers, network equipment and storage equipment by building a cloud platform; meanwhile, the automatic operation and maintenance platform is synchronously built to realize the asset management automation, the monitoring automation, the change automation and the basic service delivery automation of the cloud platform, the management cost of the cloud platform is reduced through the automatic operation and maintenance platform, the supporting capacity of the service is improved, and the quick iteration and the operation continuity of the service are guaranteed.
The Management objects of the automation operation and maintenance platform are physical objects and virtual objects of the data center, that is, configuration items managed by a Configuration Management Database (CMDB). The configuration management database is a logical database and contains configuration items of the whole life cycle of the configuration items and the relations (including physical relations, communication relations and dependency relations) among the configuration items. When the automation change is implemented, the inaccurate operation configuration items or the untimely update of the attributes of the configuration items can bring production faults/have production hidden dangers, possibly cause production accidents and cause unavailable services, so that the guarantee of the accuracy and the timeliness of the configuration items is the core premise of the automation operation and maintenance platform.
Currently, cloud platforms are evolving towards the cloud primordial direction, that is, container cloud platforms are built as a runtime computing environment of services. The container cloud platform is an enterprise-level cloud platform capable of managing a plurality of Kubernets clusters. The container cloud platform realizes the arrangement and scheduling of containers by adopting open-source Kubernets, provides the capabilities of service agile release, elastic expansion and fault self-healing, greatly exerts the capability of basic resources and forms a new foundation of a foundation environment for enterprise digital rapid transformation. Specifically, the container is an application container engine, so that a developer can package an application program and dependent software thereof into a portable container in a uniform manner, light virtualization can be achieved, and cross-environment seamless deployment is supported. Kubernets is an open source software cluster for large-scale container deployment and management, has the capabilities of resource scheduling, application deployment and management, automatic repair, service discovery, load balancing and elastic expansion, and is a container arrangement standard.
However, the introduction of the kubernets technology in the container cloud platform brings great complexity in the field of configuration management, on one hand, the service release of the container cloud platform is more frequent than that of the virtual machine era, and the resource delivery realizes second-level delivery; on the other hand, the life cycle of the container cloud platform container is short, the attribute of the container cloud platform container can be changed due to the eviction caused by the fault and the like, and the challenges are brought to the accuracy and timeliness of the operation and maintenance automatic management.
In the related art, the conventional configuration item acquisition is based on a mode that a client acquires and reports periodically, and this mode needs to be deployed on all cloud platform nodes to perform full scanning and acquisition reporting of configuration items on one hand, and needs to set a timing policy to perform acquisition reporting periodically on the other hand. The traditional configuration item acquisition mode cannot dynamically sense the change of the configuration item, cannot realize the real-time acquisition and report of the configuration increment, has low updating accuracy of the configuration item and poor real-time performance, and is not suitable for the operation and maintenance of a container cloud platform.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application. Referring to fig. 1, the system includes a cluster 101, a configuration management database 102, and a cluster processing end 103. The cluster processing terminal 103 may be an entity terminal, such as a mobile phone, a computer, or a virtual terminal, which is not limited in this application.
As shown in fig. 1 (a), the cluster 101 includes a plurality of hosts, specifically, a host 1011, a host 1012, a host 1013, and a host 1014, and a host Agent (Agent) is deployed on each host, and the Agent collects configuration items of the host at regular time and reports the configuration items to the configuration management database 102. The configuration item acquisition mode cannot dynamically sense the change of the configuration item, and has low accuracy and poor real-time performance.
In this embodiment, as shown in (b) in fig. 1, each cluster 101 corresponds to one cluster processing end 103, and the cluster processing end 103 may monitor changes of configuration items in the corresponding cluster 101 in real time and send configuration item data to the configuration management database 102, so as to implement accurate, real-time, and incremental reporting of the configuration items.
In the embodiment of the application, a terminal device (a cluster processing end) monitors a change event of a configuration item in each cluster; acquiring first configuration item data corresponding to the change event and carrying out aggregation processing on the first configuration item data to obtain second configuration item data; carrying out format conversion on the second configuration item data to obtain target configuration item data; and sending the target configuration item data to the configuration management database. Therefore, the configuration item data is obtained by monitoring the change event of each cluster configuration item, and the configuration item data is reported and stored after aggregation processing and format conversion, so that the change of the configuration item can be dynamically sensed, dynamic monitoring is realized, and meanwhile, the incremental reporting of the configuration item is carried out aiming at the change event, and the updating accuracy and the real-time performance of the configuration item are improved.
The embodiments shown in the present application will be described in detail below with reference to specific examples. It should be noted that the following embodiments may exist independently or may be combined with each other, and description of the same or similar contents is not repeated in different embodiments.
The following describes the configuration item synchronization process with reference to the embodiment shown in fig. 2.
Fig. 2 is a schematic flowchart of a configuration item synchronization method according to an embodiment of the present application. Referring to fig. 2, the method may include:
s201, monitoring the change event of the configuration items in each cluster.
The execution main body in the embodiment of the application may be a terminal device, specifically a cluster management end, or the like, and may also be a configuration item synchronization device arranged in the terminal device. The configuration item synchronization means may be implemented by software, or by a combination of software and hardware. For ease of understanding, the following description will be made by taking an execution subject as an example of a terminal device.
In the embodiment of the present application, the cluster specifically refers to a Kubernetes cluster. A Configuration Item (CI) may refer to a management object in a cluster, specifically, an entity object, or a virtual object, such as a namespace, a workload, or a container. The change event may refer to a change event of the configuration item, such as an addition event, a deletion event, or an update (modification) event.
The container cloud platform may include a management end, where the management end is responsible for deployment of the cluster processing end, and when a new cluster is registered in the management end, the management end may automatically deploy a corresponding cluster processing end for the cluster, and then the cluster processing end monitors and reports configuration items for the cluster. Each cluster management end may include a listener for monitoring change events of configuration items in the corresponding cluster in real time.
S202, first configuration item data corresponding to the change event are obtained, and aggregation processing is carried out on the first configuration item data to obtain second configuration item data.
In this embodiment of the present application, the first configuration item data may refer to configuration data corresponding to a change event, and the first configuration item data may include configuration item attribute information, time information, version information, and the like. The attribute information of the configuration item may refer to specific parameters of the configuration item. The time information may refer to a time corresponding to the change event. The version information may refer to an updated version of the configuration item, and the like.
The second configuration item data may be configuration data obtained by merging change events of the same configuration item. In a cloud platform, which is a cloud-native container, versions are changed more quickly, and the configuration items are changed very quickly. Multiple changes may occur to the same configuration item in a shorter period of time. After the cluster management terminal acquires the first configuration item data, the change events of the same configuration item can be merged to acquire the merged second configuration item data, so that transmission resources can be saved, and the processing complexity can be reduced.
Illustratively, when version upgrading is performed in a cluster, for the same configuration item, there may be a destruction and creation process in a short time, and there may be two change events, namely a delete event and a new event, and the cluster management end may merge first configuration item data corresponding to the delete event and the new event respectively to obtain merged second configuration item data, so as to save transmission resources.
And S203, carrying out format conversion on the second configuration item data to obtain target configuration item data.
In this embodiment, the target configuration item data may refer to configuration data that conforms to a database storage format and can be directly stored by the database. Because the data storage formats corresponding to different databases are different, the cluster management terminal can perform format conversion on the second configuration item data to obtain target configuration item data conforming to the data storage formats of the databases, so that data distortion caused by format non-correspondence can be avoided, and accurate reporting of the configuration item data is ensured.
And S204, sending the target configuration item data to a configuration management database.
In the embodiment of the present application, the configuration management database CMDB may refer to a preset database for storing configuration item data. After the target configuration item data is obtained, the cluster processing end may send the target configuration item data to the configuration management database, and the configuration management database receives the target configuration item data for persistent storage, thereby completing database dropping.
According to the configuration item synchronization method provided by the embodiment of the application, the terminal equipment monitors the change event of the configuration items in each cluster; acquiring first configuration item data corresponding to the change event and carrying out aggregation processing on the first configuration item data to obtain second configuration item data; carrying out format conversion on the second configuration item data to obtain target configuration item data; and sending the target configuration item data to a configuration management database. Therefore, the configuration item data is obtained by monitoring the change event of each cluster configuration item, aggregation processing and format conversion are carried out, and then reporting and storing are carried out, so that the change of the configuration items can be dynamically sensed, dynamic monitoring is realized, and meanwhile, incremental reporting of the configuration items is carried out aiming at the change event, and the accuracy and the real-time performance of updating the configuration items are improved.
On the basis of any of the above embodiments, the following describes the process in detail with reference to the embodiment shown in fig. 3.
Fig. 3 is a flowchart illustrating another configuration item synchronization method according to an embodiment of the present application. Referring to fig. 3, the method may include:
s301, registering a monitoring function in an interface gateway in each cluster; and monitoring the change event of the configuration items in the cluster through a monitoring function.
In the embodiment of the application, the monitoring of the change event can be realized by a monitor, and also can be realized by a List-Watch, wherein the List-Watch is a Kubernetes unified asynchronous message processing mechanism, and the real-time property, the reliability, the sequentiality and the high performance of the message can be ensured. The snoop function may be used to snoop a change event of the configuration item, and the snoop function may also be referred to as a snoop event, which is not limited in this embodiment of the application. The cluster processing end corresponding to each cluster may register a monitoring function in an Application Programming Interface (API) gateway of the cluster, and may subsequently monitor a change event of the configuration item through the monitoring function.
Specifically, the cluster processing end may first determine the configuration items that need to be monitored, and then register a monitoring function for each configuration item. Configuration items to be monitored may specifically include a container instance (Pod), a workload (Deployment), an internal route (Service), a Namespace (Namespace), and the like. The Namespace may refer to a Namespace in a kubernets cluster, and is a collection of a series of resources, including internal routes, workloads, containers, and the like. The internal routing Service may refer to a kubernets cluster internal routing through which container instances inside the workload may be accessed. The workload Deployment may be a resource object within a kubernets cluster, which is a collection of container instance replicas. A container instance Pod may refer to a collection of container instances and the storage, configurations on which they depend. Other configuration items can be included, and the embodiment of the present application is not limited to this.
In the embodiment of the application, based on the monitoring function, when the configuration item changes, the cluster processing terminal can realize quick sensing and improve the real-time property of reporting the subsequent configuration item; meanwhile, the cluster processing end reports the changed configuration item data, so that the acquired data volume is less, and incremental acquisition and reporting are realized.
S302, acquiring first configuration item data corresponding to a change event; determining a change event corresponding to the same configuration item within a preset time threshold; and determining the version information of the change event corresponding to the same configuration item according to the first configuration item data.
In this application embodiment, the preset time threshold may refer to a preset time critical value, specifically may be 10 seconds, 30 seconds, 60 seconds, or the like, and specifically may be set based on an actual requirement, which is not limited in this application embodiment.
The version information may refer to an updated version corresponding to the change event. Since the version of the container cloud platform is faster, when the configuration item is updated, information such as the updated version is usually included. When the cluster processing terminal monitors the change event, version information, time information and the like can be added to the first configuration item data, and the subsequent combination processing can be performed on the version information.
And S303, in the change events corresponding to the same configuration item, determining the first configuration item information corresponding to the change event with the latest version information as second configuration item data.
In the embodiment of the present application, when versions are changed, a cluster may change for many times in a short time for the same configuration item. For example, the upgrade of the configuration item usually has two processes of destroying and creating, and there are two change events, namely a delete event and a new add event, corresponding to a short time. Because the interval time of the plurality of change events is short, if the first configuration item data of the plurality of change events are reported, the processing complexity is high, the data transmission quantity is large, and the transmission resource consumption is high.
In this case, the cluster processing end may compare version information of the first configuration item information corresponding to each of the plurality of change events of the same configuration item within the preset time threshold, and use the second configuration item data with the latest version information as the second configuration item data for subsequent reporting, so that data transmission amount may be reduced, transmission resources may be saved, and processing complexity may be reduced.
When determining the old and new version information, the cluster processing terminal may determine based on the specific form of the version information. For example, when the version information is represented by a string of numbers, the cluster processing end can determine whether the version is old or new according to the size of the numbers, and the larger the number is, the newer the version is. Of course, the version information may also be expressed in other manners, and corresponds to other determination methods, which is not limited in this embodiment of the present application.
S304, determining a data format in a configuration management database; and analyzing the second configuration item data and assembling the second configuration item data into a data format to obtain target configuration item data.
In this embodiment of the present application, the data format may refer to a specific storage manner of data in the configuration management database, and specifically may include fields included in the configuration item data, an arrangement order of the fields, and the like. After determining the second configuration item data, the cluster processing end may invoke a preset data conversion method, analyze the second configuration item data, and reassemble according to the data format in the configuration management database to obtain the target configuration item data. Therefore, normal reporting and storage of the configuration item data can be ensured, and the updating accuracy of the configuration item is improved.
S305, sending the target configuration item data to the configuration management database through the distributed message system Kafka.
In the embodiment of the present application, kafka is a high-throughput distributed publish-subscribe messaging system, which can be used for processing action flow data. When determining the target configuration item data, the cluster management terminal may report the target configuration item data to Kafka, and then the configuration management database may receive the target configuration item data from Kafka and perform persistent storage.
In the related art, the conventional configuration item acquisition mode is to realize configuration acquisition through a host Agent program Agent, that is, one Agent is deployed on each host in a cluster, the Agent periodically executes an instruction to scan each configuration item on the host, the processed configuration item data is sent to a CMDB platform through a hypertext Transfer Protocol (HTTP), and the CMDB performs persistence to realize configuration item management of a distributed architecture. The sending mode of the configuration item data is that the host sends the configuration item data to the CMDB one by one, and when the data volume is large, the situation of configuration item data loss is easy to occur, so that the updating of the configuration item is not accurate enough.
In the embodiment of the application, the cluster management terminal takes Kafka as a high-throughput distributed message system, target configuration item data of all clusters are reported to the Kafka, a configuration management database CMDB accesses the Kafka to perform configuration consumption, configuration item data are obtained to be stored, high-performance processing of the overall configuration data is realized, the performance of a configuration data database can be improved by adopting the Kafka, and the processing time can be shortened; meanwhile, the situation of configuration data loss is avoided, and the updating accuracy of the configuration items is improved.
Fig. 4 is an architecture diagram of a configuration item synchronization method according to an embodiment of the present application. As shown in fig. 4, a management end is deployed in the container cloud platform, and the management end is responsible for the deployment of the cluster processing end, and when a new cluster is registered in the management end, the management end automatically deploys a corresponding cluster processing end for the cluster. The cluster processing end comprises 3 modules of a listener, a callback device and a converter. The monitor is used for registering a monitoring function in an interface gateway (API Server) of the cluster and monitoring the change event of the configuration item. The callback device comprises a callback function, and can be used for acquiring first configuration item data corresponding to the change event, performing aggregation processing on the first configuration item data to obtain second configuration item data, and sending the second configuration item data to the converter. The converter may be configured to receive the second configuration item data sent by the callback device, perform format conversion according to a data format in a Configuration Management Database (CMDB), obtain target configuration item data, send the target configuration item data to Kafka, and receive the target configuration item data in Kafka by the configuration management database CMDB, and finally complete library dropping.
In the related technology, the configuration items of the container cloud platform are all in an Etcd database of Kubernets at present, the Etcd database is distributed Key-Value storage developed based on a Raft protocol, is used for service discovery, shared configuration and consistency guarantee, and is a component in the Kubernets. Based on the method, the configuration items of the container cloud platform cannot be acquired based on a host Agent mode.
In the embodiment of the application, aiming at the characteristics of a container cloud platform, the acquisition of configuration items is realized by monitoring the interface gateway of Kubernets, and the nano-tubes of a plurality of Kubernets clusters are supported; monitoring the change events of the configuration items, realizing incremental acquisition of the configuration items of the container cloud platform, and reporting the configuration items to the CMDB platform in real time, thereby realizing accurate and real-time acquisition of the configuration items, constructing a base stone of the automatic operation and maintenance platform, supporting the development of rich operation and maintenance scenes, and enhancing the control capability of the container cloud platform.
Exemplarily, fig. 5 is a schematic processing logic diagram of a configuration item synchronization method according to an embodiment of the present application. As shown in fig. 5, for each managed Kubernetes cluster, a CMDB configuration collection instance, i.e., a cluster management end, is initialized, thereby implementing management of multiple clusters; the listener, callback, and translator instances are initialized in the cluster management side. The cluster management end monitors a configuration item change event in the cluster through a monitor; after receiving the first configuration item data, triggering a callback function in a callback device to acquire the first configuration item data, and performing aggregation processing on the first configuration item data to acquire second configuration item data; and the cluster management terminal converts the second configuration item data into target configuration item data conforming to the format of the configuration management database through a converter, and receives the data from the configuration management database through a high-throughput message processing mechanism Kafka and stores the data in a persistent manner. Therefore, incremental acquisition and real-time reporting of the configuration items are realized, operation and maintenance consumption activities of the automatic operation and maintenance platform based on the accurate configuration items are realized, and manageability and controllability of the container cloud platform are enhanced.
Fig. 6 is a schematic structural diagram of a configuration item synchronization apparatus according to an embodiment of the present application. Referring to fig. 6, the configuration item synchronization apparatus 10 may include:
a monitoring module 11, configured to monitor a change event of a configuration item in each cluster;
the processing module 12 is configured to obtain first configuration item data corresponding to the change event, and perform aggregation processing on the first configuration item data to obtain second configuration item data;
the conversion module 13 is configured to perform format conversion on the second configuration item data to obtain target configuration item data;
and a sending module 14, configured to send the target configuration item data to the configuration management database.
In a possible implementation, the listening module 11 is specifically configured to:
registering a monitoring function in an interface gateway in each cluster;
and monitoring the change events of the configuration items in the cluster through a monitoring function.
In a possible implementation, the processing module 12 is specifically configured to:
determining a change event corresponding to the same configuration item within a preset time threshold;
determining version information of a change event corresponding to the same configuration item according to the first configuration item data;
and in the change events corresponding to the same configuration item, determining the first configuration item information corresponding to the change event with the latest version information as second configuration item data.
In a possible implementation, the conversion module 13 is specifically configured to:
determining a data format in a configuration management database;
and analyzing the second configuration item data and assembling the second configuration item data into a data format to obtain target configuration item data.
In a possible implementation, the sending module 14 is specifically configured to:
and sending the target configuration item data to the configuration management database through a distributed message system Kafka.
The configuration item synchronization apparatus 10 provided in the embodiment of the present application may execute the technical solutions shown in the foregoing method embodiments, and the implementation principle and the beneficial effects are similar, which are not described herein again.
Fig. 7 is a schematic structural diagram of a configuration item synchronization system according to an embodiment of the present application. Referring to fig. 7, the configuration item synchronization system 20 may include: a listener 21, a callback 22, and a translator 23, wherein,
the listener 21 is configured to listen to a change event of the configuration item in each cluster.
The callback device 22 is configured to obtain first configuration item data corresponding to the variation event, and aggregate the first configuration item data to obtain second configuration item data.
The converter 23 is configured to perform format conversion on the second configuration item data to obtain target configuration item data.
The translator 23 is also operable to send the target configuration item data to the configuration management database.
The configuration item synchronization system 20 provided in the embodiment of the present application may execute the technical solutions shown in the foregoing method embodiments, and the implementation principles and beneficial effects thereof are similar, and are not described herein again.
Fig. 8 is a schematic structural diagram of a configuration item synchronization apparatus according to an embodiment of the present application. Referring to fig. 8, the configuration item synchronization apparatus 30 may include: memory 31, processor 32. Illustratively, the memory 31, the processor 32, and the various parts are interconnected by a bus 33.
The memory 31 is used for storing program instructions;
the processor 32 is configured to execute the program instructions stored in the memory to implement the configuration item synchronization method shown in the above embodiment.
The configuration item synchronization device 30 shown in the embodiment of fig. 8 may execute the technical solution shown in the above method embodiment, and the implementation principle and the beneficial effect thereof are similar, and are not described again here.
The embodiment of the application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is used for implementing the above configuration item synchronization method.
The embodiment of the present application may further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for synchronizing configuration items may be implemented.
It should be noted that, the processor mentioned in the embodiments of the present application may be a Central Processing Unit (CPU), and may also be other general-purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be appreciated that the memory referred to in the embodiments of the application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SLDRAM (synchronous link DRAM), and direct rambus RAM (DR RAM). It should be noted that when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, the memory (memory module) is integrated in the processor. It should be noted that the memory described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
With regard to each module/unit included in each apparatus and product described in the above embodiments, it may be a software module/unit, or may also be a hardware module/unit, or may also be a part of a software module/unit and a part of a hardware module/unit. The respective apparatuses, products may be applied to or integrated in a chip, a chip module or a terminal device. For example, for each apparatus and product applied to or integrated into a chip, each module/chip included in the apparatus and product may be implemented by using hardware such as a circuit, or at least part of the modules/units may be implemented by using a software program running on a processor integrated within the chip, and the rest of the modules/units may be implemented by using hardware such as a circuit.
In this application, the terms "include," "includes," and variations thereof may refer to non-limiting inclusions; the term "or" and variations thereof may mean "and/or". The terms "first," "second," and the like in this application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. In the present application, "a plurality" means two or more. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.

Claims (10)

1. A configuration item synchronization method, comprising:
monitoring the change event of the configuration items in each cluster;
acquiring first configuration item data corresponding to the change event, and performing aggregation processing on the first configuration item data to obtain second configuration item data;
carrying out format conversion on the second configuration item data to obtain target configuration item data;
and sending the target configuration item data to a configuration management database.
2. The method of claim 1, wherein the monitoring for a change event of a configuration item in each cluster comprises:
registering a monitoring function in an interface gateway in each cluster;
and monitoring the change event of the configuration items in the cluster through the monitoring function.
3. The method according to claim 1, wherein the aggregating the first configuration item data to obtain second configuration item data comprises:
determining a change event corresponding to the same configuration item within a preset time threshold;
determining version information of a change event corresponding to the same configuration item according to the first configuration item data;
and in the change events corresponding to the same configuration item, determining the first configuration item information corresponding to the change event with the latest version information as the second configuration item data.
4. The method of claim 1, wherein converting the format of the second configuration item data to obtain target configuration item data comprises:
determining a data format in the configuration management database;
and analyzing the second configuration item data and assembling the second configuration item data into the data format to obtain the target configuration item data.
5. The method of claim 1, wherein sending the target configuration item data to a configuration management database comprises:
and sending the target configuration item data to the configuration management database through a distributed message system Kafka.
6. A configuration item synchronization apparatus, comprising:
the monitoring module is used for monitoring the change event of the configuration items in each cluster;
the processing module is used for acquiring first configuration item data corresponding to the change event and performing aggregation processing on the first configuration item data to obtain second configuration item data;
the conversion module is used for carrying out format conversion on the second configuration item data to obtain target configuration item data;
and the sending module is used for sending the target configuration item data to a configuration management database.
7. A configuration item synchronization system, comprising: a listener, a callback, and a translator; the listener is connected with the callback device; the callback device is connected with the converter; wherein, the first and the second end of the pipe are connected with each other,
the monitor is used for monitoring the change event of the configuration items in each cluster;
the callback device is used for acquiring first configuration item data corresponding to the change event and performing aggregation processing on the first configuration item data to obtain second configuration item data;
the converter is used for carrying out format conversion on the second configuration item data to obtain target configuration item data;
the converter is further configured to send the target configuration item data to a configuration management database.
8. A configuration item synchronization apparatus, comprising: a processor, a memory;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored by the memory to implement the method of any one of claims 1 to 5.
9. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed perform the method of any one of claims 1 to 5.
10. A computer program product, characterized in that it comprises a computer program which, when executed, implements the method of any one of claims 1 to 5.
CN202211214414.5A 2022-09-30 2022-09-30 Configuration item synchronization method, device, system and equipment Pending CN115550382A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211214414.5A CN115550382A (en) 2022-09-30 2022-09-30 Configuration item synchronization method, device, system and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211214414.5A CN115550382A (en) 2022-09-30 2022-09-30 Configuration item synchronization method, device, system and equipment

Publications (1)

Publication Number Publication Date
CN115550382A true CN115550382A (en) 2022-12-30

Family

ID=84731068

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211214414.5A Pending CN115550382A (en) 2022-09-30 2022-09-30 Configuration item synchronization method, device, system and equipment

Country Status (1)

Country Link
CN (1) CN115550382A (en)

Similar Documents

Publication Publication Date Title
US10481948B2 (en) Data transfer in a collaborative file sharing system
US10560544B2 (en) Data caching in a collaborative file sharing system
US9712410B1 (en) Local metrics in a service provider environment
CN103338135B (en) A kind of method for real-time monitoring of cluster storage capacity
CN105653425B (en) Monitoring system based on complex event processing engine
CN111381983B (en) Lightweight message middleware system and method of virtual test target range verification system
CN111142971B (en) Cloud platform application ready checking method suitable for traditional application clouding
CN110750592B (en) Data synchronization method, device and terminal equipment
CN112667362B (en) Method and system for deploying Kubernetes virtual machine cluster on Kubernetes
CN111124277A (en) Deep learning data set caching method, system, terminal and storage medium
US10498817B1 (en) Performance tuning in distributed computing systems
US11294740B2 (en) Event to serverless function workflow instance mapping mechanism
CN112000353A (en) Application running method and device and storage medium
CN114363144B (en) Fault information association reporting method and related equipment for distributed system
Di Sanzo et al. A flexible framework for accurate simulation of cloud in-memory data stores
Tang et al. Achieving convergent causal consistency and high availability for cloud storage
CN112559525B (en) Data checking system, method, device and server
US11777810B2 (en) Status sharing in a resilience framework
CN115550382A (en) Configuration item synchronization method, device, system and equipment
CN114201508A (en) Data processing method, data processing apparatus, electronic device, and storage medium
CN115373757A (en) Solving method and device for cluster monitoring data loss in Promethues fragmentation mode
CN115103012A (en) Geospatial information microservice integration system and method
CN114969199A (en) Method, device and system for processing remote sensing data and storage medium
Ngo et al. K2: Reading quickly from storage across many datacenters
US11966723B2 (en) Automatic management of applications in a containerized environment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination