CN112632092A - Cluster management method, device, equipment and storage medium - Google Patents
Cluster management method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112632092A CN112632092A CN202011505788.3A CN202011505788A CN112632092A CN 112632092 A CN112632092 A CN 112632092A CN 202011505788 A CN202011505788 A CN 202011505788A CN 112632092 A CN112632092 A CN 112632092A
- Authority
- CN
- China
- Prior art keywords
- data
- cluster
- cache
- component operation
- writing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 230000003993 interaction Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004883 computer application Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a cluster management method, which is applied to a management server and comprises the following steps: acquiring component operation data of each node of the cluster; under the condition that a data cache switch is turned on, writing the obtained component operation data into a target cache, wherein the target cache is a sub-cluster constructed based on at least two nodes of a cluster; and writing the component operation data in the target cache into the database under the condition that the set database writing triggering condition is reached. By applying the technical scheme provided by the application, the component operation data is cached through the sub-cluster constructed by at least two nodes of the cluster, and then the component operation data in the target cache is written into the database when the database writing triggering condition is met, so that IO interaction between the management server and the database can be reduced, the pressure of the node where the management server is located is relieved, and the safety and the stability of the data are guaranteed. The application also discloses a cluster management device, equipment and a storage medium, which have corresponding technical effects.
Description
Technical Field
The present application relates to the field of computer application technologies, and in particular, to a cluster management method, apparatus, device, and storage medium.
Background
With the rapid development of computer technology, the application range of the cluster is more and more extensive, and the cluster can be used in the aspects of storage and service processing. A cluster may be made up of multiple nodes with the same or different types of components, such as storage components, computing components, etc., deployed in each node.
In practical application, the component operation data of each node of the cluster needs to be stored in a database for persistent recording, so as to be read and called from the database when needed.
At present, after a management server of a cluster obtains component operation data of each node of the cluster, the component operation data can be directly stored in a database. When the cluster scale is large, the data volume of the component operation data of each node is large, the pressure of the node where the management server is located is increased, the management server needs to continuously interact with the database, data is written into the database, and the safety and the stability of the data cannot be guaranteed.
Disclosure of Invention
The application aims to provide a cluster management method, a cluster management device, a cluster management equipment and a storage medium, so as to relieve the pressure of a node where a management server is located and ensure the safety and stability of data.
In order to solve the technical problem, the application provides the following technical scheme:
a cluster management method is applied to a management server of a cluster, and comprises the following steps:
obtaining component operating data of each node of the cluster;
under the condition that a data cache switch is turned on, writing the obtained component operation data into a target cache, wherein the target cache is a sub-cluster constructed based on at least two nodes of the cluster;
and writing the component operation data in the target cache into a database under the condition that a set database writing triggering condition is reached.
In a specific embodiment of the present application, when the data cache switch is turned on, before the writing the obtained component operation data into the target cache, the method further includes:
determining whether a currently enabled cache is the target cache;
if so, the step of writing the obtained component operation data into a target cache is executed.
In a specific embodiment of the present application, in a case that it is determined that the currently enabled cache is not the target cache, the method further includes:
writing the obtained component operation data into a local cache;
and writing the component operation data in the local cache into the database under the condition that a set database writing triggering condition is reached.
In a specific embodiment of the present application, when the data cache switch is not turned on, the method further includes:
writing the obtained component operation data into the database.
In a specific embodiment of the present application, before the writing the obtained component operation data into the database, the method further includes:
determining whether the set data caching condition is met currently;
if yes, outputting prompt information for whether the data cache switch is turned on;
and if a non-opening instruction fed back by the user based on the prompt information is received, executing the step of writing the obtained component operation data into the database.
In a specific embodiment of the present application, after the outputting the prompt information for whether the data cache switch is turned on, the method further includes:
if a starting instruction fed back by the user based on the prompt information is received, starting the data cache switch;
and executing the step of writing the obtained component operation data into a target cache.
In one embodiment of the present application, whether to turn on the data cache switch is determined by:
obtaining performance data of the cluster, wherein the performance data comprises at least one of the number of nodes of the cluster and the processing capacity of the node where the management server is located;
and determining whether to start the data cache switch or not according to the performance data.
A cluster management device applied to a management server of a cluster, the device comprising:
the component operation data acquisition module is used for acquiring component operation data of each node of the cluster;
the device comprises a component operation data caching module, a target cache and a data caching module, wherein the component operation data caching module is used for writing the obtained component operation data into the target cache under the condition that a data caching switch is turned on, and the target cache is a sub-cluster constructed based on at least two nodes of the cluster;
and the component operation data writing module is used for writing the component operation data in the target cache into a database under the condition that a set database writing triggering condition is reached.
A cluster management device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of any of the above described cluster management methods when executing the computer program.
A computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the cluster management method of any of the preceding claims.
By applying the technical scheme provided by the embodiment of the application, after obtaining the component operation data of each node of the cluster, the management server of the cluster writes the obtained component operation data into the target cache under the condition that the data cache switch is turned on, the target cache is a sub-cluster constructed based on at least two nodes of the cluster, and when the set database write triggering condition is reached, the component operation data in the target cache is written into the database. The component operation data are cached through the sub-cluster constructed by at least two nodes of the cluster, and then when the database write triggering condition is met, the component operation data in the target cache are written into the database, so that IO interaction between the management server and the database can be reduced, the pressure of the node where the management server is located is relieved, and the safety and the stability of the data are guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of an implementation of a cluster management method in an embodiment of the present application;
fig. 2 is a schematic structural diagram of a cluster management apparatus in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a cluster management server in an embodiment of the present application.
Detailed Description
The core of the application is to provide a cluster management method, which can be applied to a management server of a cluster, such as an Ambri server. Ambri is a Web-based tool and supports the supply, management and monitoring of clusters such as Apache Hadoop and the like. The Ambri server is a server program and can interact with an Agent client (Agent) program deployed on each node of the cluster to obtain component operation data of each node.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, an implementation flowchart of a cluster management method provided in an embodiment of the present application is shown, where the method may include the following steps:
s110: component operating data for each node of the cluster is obtained.
In practical application, the management server may be deployed on one node of the cluster, and the proxy client may be deployed on each node of the cluster, and the management server may obtain the component operation data of each node of the cluster through the proxy client. After the agent client collects the component operation data on the node where the agent client is located, the component operation data can be subjected to information screening, summarizing and other processing and then transmitted to the management server. The component operation data transmitted to the management server by each proxy server has the same data format.
On one hand, the proxy client in each node can report the component operation data of all the components of the node, the component operation data can be reported every 10 seconds in a heartbeat mode, and websocket connection can be established through Stomp to be pushed in time. No matter which data reporting mode is adopted, centralized processing is carried out at the management server side. Stomp is a simple, interoperable protocol used for asynchronous messaging between clients through an intermediate server, defining a text format for messaging between clients and servers.
On the other hand, when the Web page initiates an Interface request such as a component operation, the management server may analyze an API (Application Programming Interface) request sent by the page, issue the API request to each proxy client for operation, and after the proxy client performs a specific operation, send component operation data, such as operation information, a necessary log, an operation state, and a series of data to the management server.
S120: and writing the obtained component operation data into the target cache under the condition that the data cache switch is turned on.
The target cache is a sub-cluster constructed based on at least two nodes of the cluster.
In practical application, the component operation data of each node obtained by the management server is finally written into the database for persistent storage. If the component operation data is directly written into the database, when the cluster scale is large, the data volume of the component operation data of each node is large, the pressure of the management server is increased, the management server needs to continuously interact with the database, data is written into the database, and the safety and the stability of the data cannot be guaranteed.
Therefore, the embodiment of the application presets the data cache switch to be turned on or off. Specifically, whether to turn on the data cache switch may be determined by:
the method comprises the following steps: acquiring performance data of the cluster, wherein the performance data comprises at least one of the number of nodes of the cluster and the processing capacity of the node where the management server is located;
step two: and determining whether to start a data cache switch according to the performance data.
For convenience of description, the above two steps are combined for illustration.
When the cluster deployment is completed, the performance data of the cluster may be obtained first. The performance data may include at least one of the number of nodes of the cluster and the processing capability of the node where the management server is located, and of course, other information may also be included according to the actual situation. It can be understood that the larger the number of nodes of the cluster is, the larger the data volume of the component operation data is, the larger the processing capacity of the node where the management server is located is, and the faster the component operation data can be processed. The processing capacity can be quantified by memory size, number of CPUs, etc.
Whether to turn on the data cache switch may be determined based on the performance data. If the number of the nodes of the cluster is greater than the set node number threshold, the data cache switch can be determined to be turned on, and if the processing capacity of the node where the management server is located is less than the preset capacity threshold, the data cache switch can be determined to be turned on. Of course, different factors can be considered comprehensively, and whether the data cache switch is turned on or not is determined according to a set rule.
After the management server obtains the component operation data of each node of the cluster, whether the component operation data is written into the cache can be determined through the turning on or off of the data cache switch.
And under the condition that the data cache switch is turned on, the obtained component operation data can be written into the target cache. The target cache is a sub-cluster constructed based on at least two nodes of the cluster. At least two nodes based on the cluster may build a sub-cluster, such as a Remote Dictionary service (rdis) sub-cluster. The redis is an open-source log-type and Key-Value database which is written by using ANSI C language, supports network, can be based on memory and can also be persistent.
The obtained component operation data is written into the target cache, namely the component operation data is dispersed into a plurality of nodes for storage, so that the pressure of the node where the management server is located can be well relieved.
S130: and writing the component operation data in the target cache into the database under the condition that the set database writing triggering condition is reached.
In this embodiment of the present application, a database write trigger condition may be preset, for example, when a set time interval is reached, the set database write trigger condition is considered to be reached, or when the data amount of the component running data in the target cache reaches a set data amount threshold, the set database write trigger condition is considered to be reached.
After the component operation data of each node of the cluster is obtained, the obtained component operation data is written into the target cache only under the condition that the data cache switch is turned on. And writing the component operation data in the target cache into the database under the condition that the set database writing triggering condition is reached. Writing component operational data into the target cache and writing component operational data in the target cache into the database may be performed simultaneously. When the set database writing triggering condition is reached, the component running data in the target cache can be written into the database for multiple times at regular time, and the component running data with the set data volume in the target cache can be written into the database each time.
The management server may initiate multiple threads simultaneously to process the associated transaction. For example, a thread is used to collect component running data actively reported by each agent client, a thread is used to process interface requests such as component operation initiated by a Web page, a thread is used to write the component running data into a target cache, and a thread is used to write the component running data in the target cache into a database.
By applying the method provided by the embodiment of the application, after obtaining the component operation data of each node of the cluster, the management server of the cluster writes the obtained component operation data into the target cache under the condition that the data cache switch is turned on, the target cache is a sub-cluster constructed based on at least two nodes of the cluster, and when the set database write triggering condition is reached, the component operation data in the target cache is written into the database. The component operation data are cached through the sub-cluster constructed by at least two nodes of the cluster, and then when the database write triggering condition is met, the component operation data in the target cache are written into the database, so that IO interaction between the management server and the database can be reduced, the pressure of the node where the management server is located is relieved, and the safety and the stability of the data are guaranteed.
The embodiment of the application has better use effect when the cluster scale is larger. When the cluster scale is large, all the nodes report information together, which may result in the increase of the concurrency of the management server, possibly result in memory leakage, and problems of operation jam, server downtime and the like. The pressure of a node where the management server is located can be effectively relieved by using the target cache for caching, the component operation data is written into the target cache, the target cache can be separated from the management server, the pressure of the server is reduced, the stability of the server is improved, meanwhile, under the condition that the target cache is a redis sub-cluster, the reading and writing speed is high, and the performance is improved while the stability is improved. In addition, the redis sub-cluster is highly available, so that the data security is further ensured, and even if the management server is down and restarted, the cache data in the redis sub-cluster can be well synchronized after the server is restarted.
In an embodiment of the present application, in a case that the data cache switch is turned on, before writing the obtained component operation data into the target cache, the method may further include the following steps:
determining whether the currently enabled cache is a target cache; if so, the step of writing the obtained component operation data into the target cache is executed.
In the embodiment of the present application, various cache ways may be set according to actual situations. After the management server obtains the component operating data of each node of the cluster, it can further determine whether the currently enabled cache is the target cache under the condition that the data cache switch is turned on. And if the target cache is available, writing the obtained component operation data into the target cache.
And under the condition that the currently enabled cache is determined not to be the target cache, writing the obtained component operation data into a local cache, and under the condition that a set database writing triggering condition is reached, writing the component operation data in the local cache into the database.
The caching mode set by the embodiment of the application can comprise a target cache and a local cache, wherein the target cache is a sub-cluster constructed based on at least two nodes of a cluster, and the local cache is a node where a management server is located. When the cluster size is small, a local cache mode can be used, and when the cluster size is large, a target cache mode can be used, so that the problem of memory leakage easily caused by the use of the local cache mode can be effectively avoided, and cache information can be better processed.
Upon determining that the currently enabled cache is not the target cache, the currently enabled cache may be determined to be a local cache. The obtained component operational data may be written to a local cache. And when the set database writing triggering condition is reached, writing the component operation data in the local cache into the database.
And determining whether to use the target cache or the local cache according to actual needs, so that the management server can conveniently and quickly process the operating data of the components.
In an embodiment of the present application, in a case that the data cache switch is not turned on, the method may further include the steps of:
and writing the obtained component operation data into a database.
In the embodiment of the application, after the management server obtains the component operation data of each node of the cluster, the obtained component operation data can be directly written into the database under the condition that the data cache switch is not turned on. So as to carry out information search on the database subsequently.
In a specific embodiment of the present application, before writing the obtained component operation data into the database, it may be further determined whether a set data cache condition is currently satisfied, if so, prompt information for whether the data cache switch is turned on is output, and if a non-turn-on instruction fed back by a user based on the prompt information is received, the step of writing the obtained component operation data into the database is performed.
After the management server obtains the component operation data of each node of the cluster, under the condition that the data cache switch is not started, whether the set data cache condition is met or not can be determined currently. Specifically, whether the set cache condition is met or not may be determined according to information on the number of nodes of the current cluster, the load condition of the node where the management server is located, and the like. If yes, prompt information for whether the data cache switch is turned on is output, so that a user can determine whether to turn on the data cache switch according to the prompt information.
If a non-opening instruction fed back by the user based on the prompt information is received, the obtained component operation data can be written into the database.
After the prompt information for whether the data cache switch is started or not is output, if a starting instruction fed back by a user based on the prompt information is received, the data cache switch is started, and the step of writing the obtained component operation data into the target cache is executed.
The user can feed back the prompt information according to actual conditions, if a starting instruction fed back by the user based on the prompt information is received, the data cache switch can be started, and then the obtained component operation data is written into the target cache.
Corresponding to the above method embodiment, an embodiment of the present application further provides a cluster management apparatus, which is applied to a management server of a cluster, and the cluster management apparatus described below and the cluster management method described above may be referred to in correspondence.
Referring to fig. 2, the apparatus may include the following modules:
a component operation data obtaining module 210, configured to obtain component operation data of each node of the cluster;
the component operating data caching module 220 is configured to write the obtained component operating data into a target cache under the condition that a data caching switch is turned on, where the target cache is a sub-cluster constructed based on at least two nodes of a cluster;
and the component running data writing module 230 is configured to write the component running data in the target cache into the database when the set database writing trigger condition is reached.
By applying the device provided by the embodiment of the application, after obtaining the component operation data of each node of the cluster, the management server of the cluster writes the obtained component operation data into the target cache under the condition that the data cache switch is turned on, the target cache is a sub-cluster constructed based on at least two nodes of the cluster, and when the set database write triggering condition is reached, the component operation data in the target cache is written into the database. The component operation data are cached through the sub-cluster constructed by at least two nodes of the cluster, and then when the database write triggering condition is met, the component operation data in the target cache are written into the database, so that IO interaction between the management server and the database can be reduced, the pressure of the node where the management server is located is relieved, and the safety and the stability of the data are guaranteed.
In a specific embodiment of the present application, the method further includes an enable cache determination module, configured to:
under the condition that a data cache switch is turned on, before the obtained component operation data is written into a target cache, determining whether the currently enabled cache is the target cache;
if so, the component execution data caching module 220 is triggered to perform the step of writing the obtained component execution data into the target cache.
In a specific embodiment of the present application, the component runs the data caching module 220, and is further configured to:
under the condition that the currently enabled cache is determined not to be the target cache, writing the obtained component operation data into a local cache;
and writing the component operation data in the local cache into the database under the condition that the set database writing triggering condition is reached.
In one embodiment of the present application, the component operation data writing module 230 is further configured to:
and writing the obtained component operation data into the database under the condition that the data cache switch is not opened.
In one embodiment of the present application, the method further includes:
a cache determining module, configured to determine whether a set data cache condition is currently met before writing the obtained component operation data into the database; if yes, triggering a prompt message output module;
the prompt information output module is used for outputting prompt information for judging whether the data cache switch is turned on or not;
the component operation data writing module 230 is further configured to, in a case that a non-start instruction fed back by the user based on the prompt information is received, perform a step of writing the obtained component operation data into the database.
In one embodiment of the present application, the component operation data writing module 230 is further configured to:
after the prompt information for whether the data cache switch is started or not is output, if a starting instruction fed back by a user based on the prompt information is received, the data cache switch is started; and executing the step of writing the obtained component operation data into the target cache.
In a specific embodiment of the present application, the data cache system further includes a module for determining whether to turn on the data cache switch, configured to determine whether to turn on the data cache switch by:
acquiring performance data of the cluster, wherein the performance data comprises at least one of the number of nodes of the cluster and the processing capacity of the node where the management server is located;
and determining whether to start a data cache switch according to the performance data.
Corresponding to the above method embodiment, an embodiment of the present application further provides a cluster management device, including:
a memory for storing a computer program;
and the processor is used for realizing the steps of the cluster management method when executing the computer program.
As shown in fig. 3, which is a schematic diagram of a composition structure of a cluster management device, the cluster management device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through a communication bus 13.
In the embodiment of the present application, the processor 10 may be a Central Processing Unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, etc.
The processor 10 may call a program stored in the memory 11, and in particular, the processor 10 may perform operations in an embodiment of the cluster management method.
The memory 11 is used for storing one or more programs, the program may include program codes, the program codes include computer operation instructions, in this embodiment, the memory 11 stores at least the program for implementing the following functions:
acquiring component operation data of each node of the cluster;
under the condition that a data cache switch is turned on, writing the obtained component operation data into a target cache, wherein the target cache is a sub-cluster constructed based on at least two nodes of a cluster;
and writing the component operation data in the target cache into the database under the condition that the set database writing triggering condition is reached.
In one possible implementation, the memory 11 may include a program storage area and a data storage area, where the program storage area may store an operating system, an application program required by at least one function (such as an information interaction function and a data reading and writing function), and the like; the data storage area can store data created in the using process, such as component operating data, cache data and the like.
Further, the memory 11 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.
The communication interface 12 may be an interface of a communication module for connecting with other devices or systems.
Of course, it should be noted that the structure shown in fig. 3 does not constitute a limitation to the cluster management device in the embodiment of the present application, and in practical applications, the cluster management device may include more or less components than those shown in fig. 3, or some components may be combined.
Corresponding to the above method embodiment, this application embodiment further provides a computer readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above cluster management method are implemented.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The principle and the implementation of the present application are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
Claims (10)
1. A cluster management method is applied to a management server of a cluster, and the method comprises the following steps:
obtaining component operating data of each node of the cluster;
under the condition that a data cache switch is turned on, writing the obtained component operation data into a target cache, wherein the target cache is a sub-cluster constructed based on at least two nodes of the cluster;
and writing the component operation data in the target cache into a database under the condition that a set database writing triggering condition is reached.
2. The method according to claim 1, wherein, before the writing the obtained component operation data into the target cache with the data cache switch turned on, further comprising:
determining whether a currently enabled cache is the target cache;
if so, the step of writing the obtained component operation data into a target cache is executed.
3. The method of claim 2, wherein in the event that it is determined that the currently enabled cache is not the target cache, further comprising:
writing the obtained component operation data into a local cache;
and writing the component operation data in the local cache into the database under the condition that a set database writing triggering condition is reached.
4. The method of claim 1, further comprising, in the event that the data cache switch is not turned on:
writing the obtained component operation data into the database.
5. The method of claim 4, further comprising, prior to said writing said obtained component operational data to said database:
determining whether the set data caching condition is met currently;
if yes, outputting prompt information for whether the data cache switch is turned on;
and if a non-opening instruction fed back by the user based on the prompt information is received, executing the step of writing the obtained component operation data into the database.
6. The method of claim 5, wherein after outputting the hint information as to whether the data cache switch is on, further comprising:
if a starting instruction fed back by the user based on the prompt information is received, starting the data cache switch;
and executing the step of writing the obtained component operation data into a target cache.
7. The method of any of claims 1 to 6, wherein determining whether to turn on the data cache switch is performed by:
obtaining performance data of the cluster, wherein the performance data comprises at least one of the number of nodes of the cluster and the processing capacity of the node where the management server is located;
and determining whether to start the data cache switch or not according to the performance data.
8. A cluster management apparatus, applied to a management server of a cluster, the apparatus comprising:
the component operation data acquisition module is used for acquiring component operation data of each node of the cluster;
the device comprises a component operation data caching module, a target cache and a data caching module, wherein the component operation data caching module is used for writing the obtained component operation data into the target cache under the condition that a data caching switch is turned on, and the target cache is a sub-cluster constructed based on at least two nodes of the cluster;
and the component operation data writing module is used for writing the component operation data in the target cache into a database under the condition that a set database writing triggering condition is reached.
9. A cluster management device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the cluster management method according to any of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the cluster management method according to any of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011505788.3A CN112632092A (en) | 2020-12-18 | 2020-12-18 | Cluster management method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011505788.3A CN112632092A (en) | 2020-12-18 | 2020-12-18 | Cluster management method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112632092A true CN112632092A (en) | 2021-04-09 |
Family
ID=75317362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011505788.3A Pending CN112632092A (en) | 2020-12-18 | 2020-12-18 | Cluster management method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112632092A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590666A (en) * | 2021-09-30 | 2021-11-02 | 苏州浪潮智能科技有限公司 | Data caching method, system, equipment and computer medium in AI cluster |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368583A (en) * | 2017-07-21 | 2017-11-21 | 郑州云海信息技术有限公司 | A kind of method and system of more cluster information inquiries |
| CN108183961A (en) * | 2018-01-04 | 2018-06-19 | 中电福富信息科技有限公司 | A kind of distributed caching method based on Redis |
| CN108710646A (en) * | 2018-04-25 | 2018-10-26 | 福建天泉教育科技有限公司 | A kind of method and device that data synchronize |
| CN110046029A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Data processing method and device applied to multi-level buffer in cluster |
| CN110874298A (en) * | 2019-11-13 | 2020-03-10 | 北京齐尔布莱特科技有限公司 | Request data storage method and terminal equipment |
| CN111026783A (en) * | 2019-10-22 | 2020-04-17 | 无锡天脉聚源传媒科技有限公司 | Anti-jamming data storage method, system and device |
| CN112039979A (en) * | 2020-08-27 | 2020-12-04 | 中国平安财产保险股份有限公司 | Distributed data cache management method, device, equipment and storage medium |
-
2020
- 2020-12-18 CN CN202011505788.3A patent/CN112632092A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368583A (en) * | 2017-07-21 | 2017-11-21 | 郑州云海信息技术有限公司 | A kind of method and system of more cluster information inquiries |
| CN108183961A (en) * | 2018-01-04 | 2018-06-19 | 中电福富信息科技有限公司 | A kind of distributed caching method based on Redis |
| CN108710646A (en) * | 2018-04-25 | 2018-10-26 | 福建天泉教育科技有限公司 | A kind of method and device that data synchronize |
| CN110046029A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Data processing method and device applied to multi-level buffer in cluster |
| CN111026783A (en) * | 2019-10-22 | 2020-04-17 | 无锡天脉聚源传媒科技有限公司 | Anti-jamming data storage method, system and device |
| CN110874298A (en) * | 2019-11-13 | 2020-03-10 | 北京齐尔布莱特科技有限公司 | Request data storage method and terminal equipment |
| CN112039979A (en) * | 2020-08-27 | 2020-12-04 | 中国平安财产保险股份有限公司 | Distributed data cache management method, device, equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590666A (en) * | 2021-09-30 | 2021-11-02 | 苏州浪潮智能科技有限公司 | Data caching method, system, equipment and computer medium in AI cluster |
| CN113590666B (en) * | 2021-09-30 | 2022-02-18 | 苏州浪潮智能科技有限公司 | Data caching method, system, equipment and computer medium in AI cluster |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106856489B (en) | Service node switching method and device of distributed storage system | |
| US10785320B2 (en) | Managing operation of instances | |
| JP4204769B2 (en) | System and method for handling failover | |
| EP3335120B1 (en) | Method and system for resource scheduling | |
| WO2019223155A1 (en) | Sql performance monitoring method and device, computer apparatus, and storage medium | |
| CN107577578B (en) | Method, device and equipment for counting delay of NFS operation word and storage medium | |
| CN112527879B (en) | Kafka-based real-time data extraction method and related equipment | |
| CN107872517B (en) | Data processing method and device | |
| CN113452774B (en) | Message pushing method, device, equipment and storage medium | |
| US9652307B1 (en) | Event system for a distributed fabric | |
| US10958523B1 (en) | Consistent deployment of monitoring configurations on multiple computing systems | |
| CN107402863B (en) | Method and equipment for processing logs of service system through log system | |
| CN110096664B (en) | Distributed text information processing method, device, system, equipment and storage medium | |
| WO2020036763A1 (en) | Testing data changes in production systems | |
| CN115102887A (en) | Cluster node monitoring method and related equipment | |
| CN112597202A (en) | Data query method and device, computer equipment and storage medium | |
| CN112650807A (en) | Data storage management method and device, electronic equipment and readable storage medium | |
| CN112416884A (en) | Data synchronization method and system | |
| CN112632092A (en) | Cluster management method, device, equipment and storage medium | |
| CN116781488A (en) | Database high availability implementation method, device, database architecture, equipment and product | |
| CN108111630B (en) | Zookeeper cluster system and connection method and system thereof | |
| US20150205675A1 (en) | Method and System for Improving Reliability of a Background Page | |
| US11372904B2 (en) | Automatic feature extraction from unstructured log data utilizing term frequency scores | |
| US11544091B2 (en) | Determining and implementing recovery actions for containers to recover the containers from failures | |
| CN111177104A (en) | Log downloading and flushing method and device of NAS storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |