CN117555932A - Multi-cluster retrieval method, device, equipment, medium and program product - Google Patents

Multi-cluster retrieval method, device, equipment, medium and program product Download PDF

Info

Publication number
CN117555932A
CN117555932A CN202311519624.XA CN202311519624A CN117555932A CN 117555932 A CN117555932 A CN 117555932A CN 202311519624 A CN202311519624 A CN 202311519624A CN 117555932 A CN117555932 A CN 117555932A
Authority
CN
China
Prior art keywords
cluster
cloud storage
information
storage database
search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311519624.XA
Other languages
Chinese (zh)
Inventor
陈锦涛
孙政清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202311519624.XA priority Critical patent/CN117555932A/en
Publication of CN117555932A publication Critical patent/CN117555932A/en
Pending legal-status Critical Current

Links

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The disclosure provides a multi-cluster search method, which can be applied to the technical field of big data and the field of financial science and technology. The method comprises the following steps: responding to the self-defined resource of the original cluster information to update, and accessing the target cluster according to the updated self-defined resource; acquiring cluster information of a target cluster, and storing the cluster information of the target cluster into a cloud storage database, wherein the cloud storage database contains pre-stored original cluster information; and completing the search in the cloud storage database according to the user requirements to obtain a search result. The present disclosure also provides an apparatus, device, medium, and program product.

Description

Multi-cluster retrieval method, device, equipment, medium and program product
Technical Field
The present disclosure relates to the field of big data technology, and more particularly, to a multi-cluster search method, apparatus, device, medium, and program product.
Background
The current enterprise data center generally adopts a multi-cluster deployment model, and for a management plane component of the data center, unified management of multi-cluster resource information is required, and processing logic such as retrieval, screening and the like of multi-cluster resources is involved.
At present, the common mode of multi-cluster information retrieval is to poll each cluster, acquire the whole amount of information and then execute screening logic to complete the functions of aggregation, classification and the like of the multi-cluster information, so that not only is the retrieval efficiency low, but also when the number of clusters is large in scale and the number and the types of related resources are huge, the single-point pressure of a cluster API (application programming interface) server is extremely easy to be caused to be overlarge, and the stability of the cluster performance is influenced.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a multi-cluster retrieval method, apparatus, device, medium, and program product that enable stable cluster performance.
According to a first aspect of the present disclosure, there is provided a multi-cluster search method, including: responding to the self-defined resource of the original cluster information to update, and accessing the target cluster according to the updated self-defined resource; acquiring cluster information of a target cluster, and storing the cluster information of the target cluster into a cloud storage database, wherein the cloud storage database contains pre-stored original cluster information; and completing the search in the cloud storage database according to the user requirements to obtain a search result.
According to an embodiment of the present disclosure, in response to an update of a custom resource of original cluster information, accessing a target cluster according to the updated custom resource includes: analyzing the updated custom resources to obtain field information, wherein the field information comprises a cluster name of a target cluster, a cluster application programming interface IP and a certificate required by accessing the cluster application programming interface IP; verifying the validity of the field information; creating a cluster synchronizer in response to the field information verification passing; the control cluster synchronizer accesses the target cluster to the original cluster.
According to an embodiment of the present disclosure, saving cluster information of a target cluster to a cloud storage database includes: establishing connection between a target cluster and a cloud storage database; constructing cluster information of a target cluster into a data object corresponding to a data structure of a cloud storage database; and saving the data object to a cloud storage database.
According to an embodiment of the present disclosure, saving cluster information of the target cluster to the cloud storage database further includes: in response to the data object being updated as compared to the original cluster information, the data object is saved to the cloud storage database.
According to an embodiment of the present disclosure, completing a search in a cloud storage database according to a user requirement, and obtaining a search result includes: inquiring cluster information in a cloud storage database; constructing search conditions according to user requirements, wherein the search conditions comprise cluster names, namespaces, resource types, resource names and resource tag dimensions; and applying the search condition to the cluster information in the cloud storage database to obtain a search result.
According to an embodiment of the present disclosure, applying the search condition to the cluster information in the cloud storage database, after obtaining the search result, further includes: analyzing and processing the search result; and presenting the search result after analysis processing to a user in a report form or in a mode of providing an application programming interface.
According to an embodiment of the present disclosure, querying cluster information in a cloud storage database includes: and inquiring cluster information in the cloud storage database by using the application programming interface gateway.
According to an embodiment of the present disclosure, querying cluster information in a cloud storage database includes: and inquiring cluster information in the cloud storage database by using the inquiry language.
A second aspect of the present disclosure provides a multi-cluster search apparatus, comprising: the access module is used for responding to the self-defined resource of the original cluster information to update, and accessing the target cluster according to the updated self-defined resource; the storage module is used for acquiring the cluster information of the target cluster and storing the cluster information of the target cluster into the cloud storage database, wherein the cloud storage database contains the pre-stored original cluster information; and the retrieval module is used for completing retrieval in the cloud storage database according to the user requirements to obtain a retrieval result.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the multi-cluster search method described above.
A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described multi-cluster search method.
A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the multi-cluster search method described above.
According to the multi-cluster retrieval method, device, equipment, medium and program product, the storage and retrieval work of cluster information can be separated from kube-apiserver (cluster API server) by updating the changed cluster information into the cloud storage database, namely, the reading and query operation of the cluster information is transferred from kube-apiserver to the cloud storage database, so that the load pressure of kube-apiserver is reduced, cluster retrieval is carried out based on the cloud storage database, and the retrieval and screening efficiency is improved relative to a common polling cluster retrieval mode.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an application scenario diagram of a multi-cluster retrieval method according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a multi-cluster retrieval method according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow diagram for accessing a target cluster according to a new custom resource in a multi-cluster retrieval method according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flowchart of saving cluster information of a target cluster to a cloud storage database in a multi-cluster retrieval method according to an embodiment of the present disclosure;
FIG. 5 schematically illustrates a flowchart of saving cluster information of a target cluster in which an update occurs to a cloud storage database in a multi-cluster retrieval method according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a flow chart for performing a search in a cloud storage database according to user requirements in a multi-cluster search method according to an embodiment of the present disclosure;
FIG. 7 schematically illustrates a flow chart of processing search results in a multi-cluster search method according to an embodiment of the disclosure;
FIG. 8 schematically illustrates a schematic diagram of a multi-cluster search method according to an embodiment of the disclosure;
FIG. 9 schematically illustrates a block diagram of a multi-cluster search method according to an embodiment of the disclosure; and
fig. 10 schematically illustrates a block diagram of an electronic device adapted to implement a multi-cluster retrieval method in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
It should be noted that, the multi-cluster search method and apparatus in the present disclosure may be used in the case of multi-cluster search in the financial field, and may also be used in the case of multi-cluster search in any field other than the financial field, and the application field of the multi-cluster search method and apparatus in the present disclosure is not limited.
In the technical scheme of the invention, the related user information (including but not limited to user personal information, user image information, user equipment information, such as position information and the like) and data (including but not limited to data for analysis, stored data, displayed data and the like) are information and data authorized by a user or fully authorized by all parties, and the processing of the related data such as collection, storage, use, processing, transmission, provision, disclosure, application and the like are all conducted according to the related laws and regulations and standards of related countries and regions, necessary security measures are adopted, no prejudice to the public welfare is provided, and corresponding operation inlets are provided for the user to select authorization or rejection.
Keyword introduction:
clustering: by cluster is meant a loosely coupled multiprocessor system of a group of independent computer systems that communicate between processes over a network. The application program can transmit information through the network shared memory to realize the distributed computer. In popular terms, several computers are combined to work (serve), which may be parallel or backup.
Kubernetes: the method is characterized in that K8s is abbreviated by replacing 8 characters in the middle of names with 8, is a container arrangement engine, supports automatic deployment, large-scale expansion and application containerization management of containers, is used for managing containerized applications on a plurality of hosts in a cloud platform, aims at enabling the deployment of containerized applications to be simple and efficient, and provides a mechanism for deployment, planning, updating and maintenance of the applications.
Multi-cluster search: multi-cluster retrieval (Multi-cluster Retrieval) refers to the simultaneous use of multiple independent clusters for searching and retrieving in an information retrieval system. Typically, a cluster is made up of a set of related documents and indexes, and multi-cluster retrieval is to connect a plurality of such clusters so that users can perform global searches in different clusters. The main purpose of multi-cluster search is to provide more comprehensive and accurate search results. By using multiple clusters, a wider range of document fields and data sources can be covered, providing more relevant information. In addition, multi-cluster retrieval may also improve system scalability and fault tolerance, as even if one cluster fails, the other clusters may continue to provide service.
The current enterprise data center generally adopts a deployment model of multiple Kubernetes clusters, and for a management plane component of the data center, unified management of resource information of the multiple Kubernetes clusters is required, and processing logic such as retrieval, screening and the like of multiple cluster resources is involved. At present, the common mode of multi-cluster information retrieval is to poll each cluster, acquire the whole amount of information and then execute screening logic to complete the functions of aggregation, classification and the like of the multi-cluster information, so that not only is the retrieval efficiency low, but also when the number of clusters is large in scale and the number and the types of related resources are huge, the single-point pressure of the cluster kube-apiserver is extremely easy to be overlarge, and the stability of the cluster performance is influenced.
The embodiment of the disclosure provides a multi-cluster retrieval method, which comprises the following steps: responding to the self-defined resource of the original cluster information to update, and accessing the target cluster according to the updated self-defined resource; acquiring cluster information of a target cluster, and storing the cluster information of the target cluster into a cloud storage database, wherein the cloud storage database contains pre-stored original cluster information; and completing the search in the cloud storage database according to the user requirements to obtain a search result. According to the multi-cluster retrieval method, the storage and retrieval work of cluster information can be separated from kube-apiserver by updating the changed cluster information into the cloud storage database, namely, the reading and query operation of the cluster information is transferred from kube-apiserver to the cloud storage database, so that the load pressure of kube-apiserver is reduced, and then cluster retrieval is carried out based on the cloud storage database, so that the retrieval and screening efficiency is improved compared with a common polling cluster retrieval mode.
Fig. 1 schematically illustrates an application scenario diagram of a multi-cluster search method according to an embodiment of the present disclosure.
As shown in fig. 1, an application scenario 100 according to this embodiment may include terminal devices 101, 102, 103. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that, the multi-cluster search method provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the multi-cluster search apparatus provided by the embodiments of the present disclosure may be generally disposed in the server 105. The multi-cluster search method provided by the embodiments of the present disclosure may also be performed by a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the multi-cluster search apparatus provided by the embodiments of the present disclosure may also be provided in a server or server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The multi-cluster search method of the disclosed embodiment will be described in detail below with reference to fig. 2 to 8 based on the scenario described in fig. 1.
Fig. 2 schematically illustrates a flow chart of a multi-cluster search method according to an embodiment of the disclosure.
As shown in fig. 2, the multi-cluster search method of this embodiment includes operations S210 to S230, and the multi-cluster search method may be performed in a server.
In operation S210, in response to the update of the custom resources of the original cluster information, the target cluster is accessed according to the updated custom resources.
The custom resource, i.e. the CRD resource, is called Custom Resource Definitions, and k8s has some built-in resources, such as Pod, replyment, replicaSet, etc., while the CRD resource provides a way to enable a user to customize a new resource to expand the function of k8s, and the use of the CRD resource can conveniently expand the function of k8s without modifying the source code of k8 s.
In this embodiment, a monitoring mechanism is pre-established to monitor the change of the original cluster information CRD resource, and when the change of the original cluster information CRD resource is monitored, it is determined whether a new cluster information CRD resource is created or updated, and if the new cluster information CRD resource is updated, the target clusters are accessed according to the updated custom resources, and the number of the target clusters can be increased correspondingly according to the needs of the user.
In operation S220, cluster information of the target cluster is obtained, and the cluster information of the target cluster is stored in a cloud storage database, where the cloud storage database contains the pre-stored original cluster information.
Cloud storage is a model of online storage (english: cloud storage) that stores data on multiple virtual servers, typically hosted by third parties, rather than on dedicated servers. Hosting companies operate large data centers and require data storage to be hosted by people who purchase or rent storage space to them to meet the data storage requirements. The data center operator prepares the storage virtualized resource at the back end according to the requirement of the client and provides the storage virtualized resource in a storage resource pool (storage poo 1), and the client can use the storage resource pool to store files or objects.
In this embodiment, the cloud storage database provides cloud storage capability, and cluster information of the newly accessed target cluster is stored in the cloud storage database, so that independent storage of multiple cluster resources is realized.
In operation S230, the search is completed in the cloud storage database according to the user' S requirement, and the search result is obtained.
In this embodiment, since the cluster information is updated to the cloud storage database, the storage and retrieval of the cluster information are separated from kube-apiserver, that is, the reading and querying operations of the cluster information are transferred from kube-apiserver to the cloud storage database, so that the load pressure of kube-apiserver is reduced, and the user only needs to retrieve in the cloud storage database according to the retrieval condition during retrieval, so that the retrieval and screening efficiency is improved compared with the common polling cluster retrieval mode.
Fig. 3 schematically illustrates a flowchart of accessing a target cluster according to a new custom resource in a multi-cluster retrieval method according to an embodiment of the present disclosure.
As shown in fig. 3, the multi-cluster search method of this embodiment includes operations S310 to S340.
In operation S310, the updated custom resource is parsed to obtain field information, where the field information includes a cluster name of the target cluster, a cluster application programming interface IP, and a certificate required to access the cluster application programming interface IP.
In operation S320, validity verification is performed on the field information.
In this embodiment, the field information obtained by parsing is verified to ensure its integrity and validity, specifically including checking whether necessary fields exist, whether the value is legal, and the like.
In operation S330, a cluster synchronizer is created in response to the field information verification passing.
In this embodiment, in the case that the field information verification passes, a cluster synchronizer is created. The cluster synchronizer is a component for communicating and synchronizing with a specific cluster, and can configure corresponding connection and authentication parameters according to cluster information.
In operation S340, the control cluster synchronizer accesses the target cluster to the original cluster.
In this embodiment, a newly created cluster synchronizer is started to establish connection with a target cluster and perform data synchronization, and because the cluster synchronizer can perform communication and data synchronization with a specific cluster, the information acquisition and synchronization of multiple clusters can be realized by creating the cluster synchronizer to access the target cluster, and basic data is provided for subsequent multi-cluster retrieval.
Fig. 4 schematically illustrates a flowchart of saving cluster information of a target cluster to a cloud storage database in a multi-cluster retrieval method according to an embodiment of the present disclosure.
As shown in fig. 4, the multi-cluster search method of this embodiment includes operations S410 to S430.
In operation S410, a connection of the target cluster with the cloud storage database is established.
In this embodiment, the connection between the target cluster and the cloud storage database is implemented through the storage docking layer, and standard unified storage docking API interfaces are defined in a plug-in manner, including interfaces such as addition, deletion, modification, query, and the like, and specific contents are implemented according to storage interfaces provided by different cloud storage manufacturers, so that the cloud storage manufacturers are compatible.
In operation S420, cluster information of the target cluster is constructed as a data object corresponding to a data structure of the cloud storage database.
In this embodiment, the cluster information of the target cluster is constructed as a data object corresponding to the data structure of the cloud storage database by mapping the cluster information into a field of a row or document of the database table.
In operation S430, the data object is saved to the cloud storage database.
In the embodiment, the data object is stored in the cloud storage database so as to facilitate subsequent multi-cluster retrieval and data management, thus realizing the persistent storage and the random access of cluster information and facilitating the multi-cluster data analysis and operation.
In this embodiment, the cluster information is first constructed as the data object corresponding to the data structure of the cloud storage database, and then stored, so that the storage space of the cloud storage database can be saved.
Fig. 5 schematically illustrates a flowchart of saving cluster information of a target cluster, in which an update occurs, to a cloud storage database in a multi-cluster search method according to an embodiment of the present disclosure.
As shown in fig. 5, the multi-cluster search method of this embodiment includes operations S510 to S520.
In operation S510, it is determined whether the data object is updated compared to the current cluster information.
In this embodiment, before the cluster information is stored in the cloud storage database, whether the cluster information is updated is determined, and if not, the cluster information does not need to be stored.
In operation S520, in case of an update, the data object is saved to the cloud storage database.
In this embodiment, in response to updating of the data object compared with the original cluster information, the data object is saved to the cloud storage database, so that repeated saving of the cluster information is avoided, and resources are saved.
Fig. 6 schematically illustrates a flowchart for completing a search in a cloud storage database according to a user's needs in a multi-cluster search method according to an embodiment of the present disclosure.
As shown in fig. 6, the multi-cluster search method of this embodiment includes operations S610 to S630.
In operation S610, cluster information in a cloud storage database is queried.
In this embodiment, the cluster information in the cloud storage database is queried using a cluster retrieval API gateway or query language, for example, including related data such as the name of the cluster, a node list, a service list, and a Pod list.
In operation S620, a search condition is constructed according to the user' S requirement, wherein the search condition includes a cluster name, a namespace, a resource type, a resource name, and a resource tag dimension.
In this embodiment, for example, if the user wants to query the running state of the container deployed in the current application, the label of the container may be specified to obtain the container information with a specific label, and assuming that there is a label named "app", the label may be used to retrieve the container with the specific application, and the following cluster retrieval conditions may be constructed:
tag name: app tag value: myapp
By using the above-described search condition, container information having the label "app=myapp" can be searched. This will return all containers with the tag, enabling the user to obtain container information related to the particular application, and more conveniently monitor and manage the container status of the particular application.
In operation S630, the search condition is applied to the cluster information in the cloud storage database, resulting in a search result.
According to the embodiment of the disclosure, the targeted query is performed according to the retrieval conditions, so that the retrieval efficiency and accuracy can be improved.
Fig. 7 schematically illustrates a flowchart of processing a search result in a multi-cluster search method according to an embodiment of the present disclosure.
As shown in fig. 7, the multi-cluster search method of this embodiment includes operations S710 to S720.
In operation S710, an analysis process is performed on the search result.
In this embodiment, the search results are processed and presented as needed, and operations such as sorting, aggregation, statistics and the like may be performed on the cluster information to obtain the required multi-cluster search result.
In operation S720, the search result after the analysis processing is presented to the user in a report form or by providing an application programming interface.
In this embodiment, the search results are presented to the user in an appropriate manner, such as generating reports, exposing charts, or providing API interfaces, etc.
According to the embodiment of the disclosure, after the search result is obtained, the search result is displayed to the user in a mode of generating a report, displaying a chart or providing an application programming interface, and the like, so that the user can conveniently review.
In another embodiment, the multi-cluster retrieval method of the embodiments of the present disclosure utilizes an application programming interface gateway to query cluster information in a cloud storage database.
Through using the cluster retrieval API gateway, a user can interact with the cloud storage database in a programming mode, so that the user can write customized query logic and operation flow according to own requirements and business logic.
In another embodiment, the multi-cluster search method of the embodiments of the present disclosure utilizes a query language to query cluster information in a cloud storage database.
The query language is designed for facilitating the data query and operation of the user, has simple and readable grammar, and can simplify the query process and reduce the workload of writing complex queries.
Fig. 8 schematically illustrates a schematic diagram of a multi-cluster search method according to an embodiment of the present disclosure.
As shown in fig. 8, in the multi-cluster search method of this embodiment, the interaction principle among the related cloud storage, cluster synchronizer, storage docking layer, and cluster search API gateway is as follows:
the cluster retrieval API gateway provides cluster retrieval based on cluster names, namespaces, resource types, resource names and resource tag dimensions, and supports more complex retrieval conditions by improving timeliness of multi-cluster resource information retrieval through self-building a database based on cloud storage.
The cluster synchronizer provides cluster management and cluster information synchronization capabilities. And completing cluster access and management functions by customizing CRD resources. The system administrator adds managed K8S clusters in the system by adding cluster information CRD. Compared with a method of checking and using the clusters at once, the method of periodically acquiring information of the change parts of each cluster and updating the information to the cloud storage database can reduce kube-apiserver load pressure and continuously keep the latest information of the database information by aiming at the added K8S cluster.
The storage interfacing layer defines standard unified storage interfacing API interfaces including adding, deleting, modifying, inquiring and the like through a plug-in mode, and realizes specific contents according to storage interfaces provided by different cloud storage manufacturers, so that the storage interfacing layer is compatible with various cloud storage manufacturers. The storage butt layer and the cluster synchronizer form a cluster information synchronous controller together.
Cloud storage provides cloud storage capability and realizes independent storage of multiple cluster resources.
By establishing a monitoring mechanism, the change of the cluster information CRD resource is monitored, whether a new cluster information CRD resource exists is judged, and when the change of the cluster information CRD resource is monitored, whether the new cluster information CRD resource is created or updated is judged. If new cluster information CRD resources are created or updated, a cluster synchronizer is newly created according to the contents of the resources, and the cluster synchronizer is a component used for communicating and synchronizing with a specific cluster. The cluster synchronizer accesses the cluster information of the target clusters k8s1, k8s2, k8s3 and the like, and can flexibly adjust and acquire the information of the clusters at fixed time intervals or according to requirements, wherein the information comprises related data such as nodes, services, pod and the like. Comparing the currently acquired cluster information with the previously stored information, judging whether the cluster information is updated, and if the cluster information is found to be updated, storing the updated information into a cloud storage database for subsequent multi-cluster retrieval and use.
Based on the financial product pushing method, the disclosure also provides a multi-cluster searching device. The device will be described in detail below in connection with fig. 9.
Fig. 9 schematically illustrates a block diagram of a multi-cluster search apparatus according to an embodiment of the present disclosure.
As shown in fig. 9, the multi-cluster search apparatus 900 of this embodiment includes: an access module 910, a save module 920, and a retrieve module 930.
The access module 910 is configured to, in response to updating the custom resource of the original cluster information, access the target cluster according to the updated custom resource. In an embodiment, the first identification module 910 may be used to perform the operation S210 described above, which is not described herein.
The storage module 920 is configured to obtain cluster information of the target cluster, and store the cluster information of the target cluster to a cloud storage database, where the cloud storage database includes the pre-stored original cluster information. In an embodiment, the second identifying module 920 may be used to perform the operation S220 described above, which is not described herein.
The retrieval module 930 is configured to complete retrieval in the cloud storage database according to a user requirement, and obtain a retrieval result. In an embodiment, the third identifying module 930 may be used to perform the operation S230 described above, which is not described herein.
According to the embodiment of the disclosure, the multi-cluster searching device 900 of the embodiment can separate the storage and searching of the cluster information from kube-apiserver by updating the changed cluster information into the cloud storage database, that is, transfer the reading and searching operations of the cluster information from kube-apiserver to the cloud storage database, thereby reducing the load pressure of kube-apiserver, and then performing cluster searching based on the cloud storage database, so as to improve the searching and screening efficiency compared with the common polling cluster searching mode.
Any of the access module 910, the save module 920, and the retrieve module 930 may be combined in one module to be implemented, or any of them may be split into multiple modules, according to embodiments of the present disclosure. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. At least one of the access module 910, the save module 920, and the retrieve module 930 may be implemented, at least in part, as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-a-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware, such as any other reasonable way of integrating or packaging the circuitry, or in any one of, or in any suitable combination of, three of software, hardware, and firmware, in accordance with embodiments of the present disclosure. Alternatively, at least one of the access module 910, the save module 920, and the retrieve module 930 may be at least partially implemented as a computer program module, which when executed, may perform the corresponding functions.
Fig. 10 schematically illustrates a block diagram of an electronic device adapted to implement a multi-cluster retrieval method in accordance with an embodiment of the present disclosure.
As shown in fig. 10, the electronic device 110 according to the embodiment of the present disclosure includes a processor 111 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 112 or a program loaded from a storage section 118 into a Random Access Memory (RAM) 113. Processor 111 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 111 may also include on-board memory for caching purposes. Processor 111 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the disclosure.
In the RAM 113, various programs and data required for the operation of the electronic device 110 are stored. The processor 111, the ROM 112, and the RAM 113 are connected to each other through a bus 114. The processor 111 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 112 and/or the RAM 113. Note that the program may be stored in one or more memories other than the ROM 112 and the RAM 113. The processor 111 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.
According to embodiments of the present disclosure, the electronic device 110 may also include an input/output (I/O) interface 115, the input/output (I/O) interface 115 also being connected to the bus 114. The electronic device 110 may also include one or more of the following components connected to the I/O interface 115: an input section 116 including a keyboard, a mouse, and the like; an output portion 117 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage section 118 including a hard disk or the like; and a communication section 119 including a network interface card such as a LAN card, a modem, and the like. The communication section 119 performs communication processing via a network such as the internet. The drive 120 is also connected to the I/O interface 115 as needed. A removable medium 121 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 120 so that a computer program read out therefrom is installed as needed into the storage section 118.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium described above carries one or more programs that, when executed, implement the multi-cluster search method according to the embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 112 and/or RAM 113 described above and/or one or more memories other than ROM 112 and RAM 113.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to implement the multi-cluster retrieval method provided by embodiments of the present disclosure.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 111. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of signals over a network medium, and downloaded and installed via the communication part 119, and/or from the removable medium 121. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 119, and/or installed from the removable medium 121. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 111. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims (12)

1. A multi-cluster retrieval method comprising:
responding to the self-defined resource of the original cluster information to update, and accessing the target cluster according to the updated self-defined resource;
acquiring cluster information of the target cluster, and storing the cluster information of the target cluster into a cloud storage database, wherein the cloud storage database contains pre-stored original cluster information;
and completing the search in the cloud storage database according to the user requirements to obtain a search result.
2. The multi-cluster searching method according to claim 1, wherein the responding to the update of the customized resource of the original cluster information, accessing the target cluster according to the updated customized resource comprises:
analyzing the updated user-defined resources to obtain field information, wherein the field information comprises a cluster name of the target cluster, a cluster application programming interface IP and a certificate required by accessing the cluster application programming interface IP;
verifying the validity of the field information;
creating a cluster synchronizer in response to the field information verification passing;
and controlling the cluster synchronizer to access the target cluster to the original cluster.
3. The multi-cluster search method of claim 1, the saving cluster information of the target cluster to a cloud storage database comprising:
establishing connection between the target cluster and the cloud storage database;
constructing cluster information of the target cluster into a data object corresponding to a data structure of the cloud storage database;
and storing the data object to the cloud storage database.
4. The multi-cluster search method of claim 3, the saving cluster information of the target cluster to a cloud storage database further comprising:
and storing the data object to a cloud storage database in response to the data object being updated compared to the original cluster information.
5. The multi-cluster searching method according to claim 1, wherein the searching is completed in the cloud storage database according to the user requirement, and the obtaining the searching result comprises:
inquiring cluster information in the cloud storage database;
constructing a search condition according to user requirements, wherein the search condition comprises a cluster name, a name space, a resource type, a resource name and a resource tag dimension;
and applying the search condition to the cluster information in the cloud storage database to obtain a search result.
6. The multi-cluster search method according to claim 5, wherein the applying the search condition to cluster information in the cloud storage database, after obtaining a search result, further comprises:
analyzing and processing the search result;
and presenting the search result after analysis processing to a user in a report form or in a mode of providing an application programming interface.
7. The multi-cluster retrieval method of claim 5, the querying cluster information in the cloud storage database comprising:
and inquiring cluster information in the cloud storage database by using an application programming interface gateway.
8. The multi-cluster retrieval method of claim 5, the querying cluster information in the cloud storage database comprising:
and inquiring the cluster information in the cloud storage database by using an inquiry language.
9. A multi-cluster search apparatus comprising:
the access module is used for responding to the self-defined resource of the original cluster information to update, and accessing the target cluster according to the updated self-defined resource;
the storage module is used for acquiring the cluster information of the target cluster and storing the cluster information of the target cluster into a cloud storage database, wherein the cloud storage database contains the pre-stored original cluster information;
and the retrieval module is used for completing retrieval in the cloud storage database according to the user requirements to obtain a retrieval result.
10. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.
11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-8.
12. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.
CN202311519624.XA 2023-11-15 2023-11-15 Multi-cluster retrieval method, device, equipment, medium and program product Pending CN117555932A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311519624.XA CN117555932A (en) 2023-11-15 2023-11-15 Multi-cluster retrieval method, device, equipment, medium and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311519624.XA CN117555932A (en) 2023-11-15 2023-11-15 Multi-cluster retrieval method, device, equipment, medium and program product

Publications (1)

Publication Number Publication Date
CN117555932A true CN117555932A (en) 2024-02-13

Family

ID=89814125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311519624.XA Pending CN117555932A (en) 2023-11-15 2023-11-15 Multi-cluster retrieval method, device, equipment, medium and program product

Country Status (1)

Country Link
CN (1) CN117555932A (en)

Similar Documents

Publication Publication Date Title
US11636105B2 (en) Generating a subquery for an external data system using a configuration file
US11921672B2 (en) Query execution at a remote heterogeneous data store of a data fabric service
US11494380B2 (en) Management of distributed computing framework components in a data fabric service system
US11580107B2 (en) Bucket data distribution for exporting data to worker nodes
US11593377B2 (en) Assigning processing tasks in a data intake and query system
US11586627B2 (en) Partitioning and reducing records at ingest of a worker node
US11023463B2 (en) Converting and modifying a subquery for an external data system
US20200050612A1 (en) Supporting additional query languages through distributed execution of query engines
US20200065303A1 (en) Addressing memory limits for partition tracking among worker nodes
US20200050607A1 (en) Reassigning processing tasks to an external storage system
US10609140B2 (en) Dynamic resource management systems and methods
US9235636B2 (en) Presenting data in response to an incomplete query
CN108536778B (en) Data application sharing platform and method
US10942774B1 (en) Dynamic reassignment of search processes into workload pools in a search and indexing system
US11100152B2 (en) Data portal
CN115587575A (en) Data table creation method, target data query method, device and equipment
US20130346405A1 (en) Systems and methods for managing data items using structured tags
US11243756B1 (en) Extensible resource compliance management
US9602575B2 (en) Monitoring social media for specific issues
CN116069725A (en) File migration method, device, apparatus, medium and program product
US11520782B2 (en) Techniques for utilizing patterns and logical entities
CN115033574A (en) Information generation method, information generation device, electronic device, and storage medium
CN117555932A (en) Multi-cluster retrieval method, device, equipment, medium and program product
CN114363172B (en) Decoupling management method, device, equipment and medium for container group
CN116401319B (en) Data synchronization method and device, electronic equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination