CN111752907A - Centralized management and control method, system, terminal and storage medium for cluster logs - Google Patents
Centralized management and control method, system, terminal and storage medium for cluster logs Download PDFInfo
- Publication number
- CN111752907A CN111752907A CN202010470321.3A CN202010470321A CN111752907A CN 111752907 A CN111752907 A CN 111752907A CN 202010470321 A CN202010470321 A CN 202010470321A CN 111752907 A CN111752907 A CN 111752907A
- Authority
- CN
- China
- Prior art keywords
- log
- interface
- synchronization
- slave node
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000001360 synchronised effect Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 2
- 230000000737 periodic effect Effects 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a centralized management and control method, a centralized management and control system, a terminal and a storage medium for cluster logs, wherein the centralized management and control method comprises the following steps: recording operation information of all interfaces of a cluster main node to generate a record log; setting a log synchronization period; and synchronously updating the newly generated record logs to the slave nodes according to the synchronization period. The invention can reduce the inconsistency of the log format and granularity caused by adding the log in the interface, and can not provide service for the front end. The main node faults are reduced, time consumption when switching is needed is reduced, and user experience is improved. Effectively backing up log data, and seamlessly switching master nodes and slave nodes.
Description
Technical Field
The invention relates to the technical field of server clusters, in particular to a cluster log centralized management and control method, a system, a terminal and a storage medium.
Background
In the multi-node cluster, the main node provides service to the outside and records related logs. But the master node needs to change the slave node into the master node to provide services to the outside in case of failure. At this time, the service and the log are required to be migrated to a new node in a parallel mode, and transition is completed. The existing logs are recorded on each operation interface separately, after the migration of the main node is completed, when the interface interfaces are accessed, whether the main node is changed or not is judged, and if the change occurs, the log data combination is triggered, so that the first-time use of the interfaces is slow in response, the log quantity is large, the interfaces cannot be used after the midway failure, and the user experience is influenced.
Disclosure of Invention
In view of the above-mentioned deficiencies of the prior art, the present invention provides a centralized management and control method, system, terminal and storage medium for cluster logs, so as to solve the above-mentioned technical problems.
In a first aspect, the present invention provides a centralized management and control method for cluster logs, including:
recording operation information of all interfaces of a cluster main node to generate a record log;
setting a log synchronization period;
and synchronously updating the newly generated record logs to the slave nodes according to the synchronization period.
Further, the recording operation information of all interfaces of the cluster master node, and generating a recording log, includes:
setting interface information in a configuration file;
calling the interface information and positioning a target interface according to the interface information;
and recording operation information, packaging parameters and desensitizing treatment of the target interface, and adding a time stamp after the desensitizing treatment is finished.
Further, the synchronously updating the newly generated log records to the slave node according to the synchronization cycle includes:
checking whether a receiving module of the slave node is normal, and if so, continuing to synchronize the log;
making a synchronized mark on data synchronized to a slave node and recording the synchronization time;
data without a synchronization mark is synchronized to the slave node.
In a second aspect, the present invention provides a centralized management and control system for cluster logs, including:
the interface recording unit is configured to record all interface operation information of the cluster master node and generate a recording log;
a period setting unit configured to set a log synchronization period;
and the log synchronization unit is configured for periodically synchronizing the unit according to the synchronization period and synchronously updating the newly generated record log to the slave node according to the synchronization period.
Further, the interface recording unit includes:
the address configuration module is configured for setting interface information in a configuration file;
the interface positioning module is configured to call the interface information and position a target interface according to the interface information;
and the operation recording module is configured for recording operation information, encapsulating and desensitizing parameters of the target interface and adding a time stamp after desensitizing treatment is finished.
Further, the periodically synchronizing unit according to the synchronization cycle includes:
the receiving and checking module is configured to check whether the receiving module of the slave node is normal or not, and if the receiving module of the slave node is normal, the log is continuously synchronized;
the data marking module is configured to mark the data synchronized to the slave node as synchronized data and record the current synchronization time;
and the data sending module is configured to synchronize the data without the synchronization mark to the slave node.
In a third aspect, a terminal is provided, including:
a processor, a memory, wherein,
the memory is used for storing a computer program which,
the processor is used for calling and running the computer program from the memory so as to make the terminal execute the method of the terminal.
In a fourth aspect, a computer storage medium is provided having stored therein instructions that, when executed on a computer, cause the computer to perform the method of the above aspects.
The beneficial effect of the invention is that,
the centralized management and control method, the centralized management and control system, the centralized management and control terminal and the storage medium for the cluster logs can reduce the inconsistency of log formats and granularity caused by adding the logs in an interface, and cannot provide service for a front end. The main node faults are reduced, time consumption when switching is needed is reduced, and user experience is improved. Effectively backing up log data, and seamlessly switching master nodes and slave nodes.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a method of one embodiment of the invention.
FIG. 2 is a schematic flow diagram of a method of one embodiment of the invention.
FIG. 3 is a schematic block diagram of a system of one embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a schematic flow diagram of a method of one embodiment of the invention. The execution subject in fig. 1 may be a cluster log centralized management and control system.
As shown in fig. 1, the method 100 includes:
and step 130, synchronously updating the newly generated record log to the slave node according to the synchronous period.
Referring to fig. 2, specifically, the centralized management and control method for cluster logs includes:
and the log processing program is used for recording operation information, encapsulating parameters and desensitizing the parameters of different interfaces according to the configuration information. And adding a time stamp after the processing is finished, and recording the time stamp into a database.
And the log synchronization program synchronizes log information to the slave node at regular time, and each synchronization is only successful in the last synchronization and data between the synchronization and the current synchronization. And (3) synchronizing each node, namely firstly checking whether the slave node is reachable, checking whether a receiving program of the slave node is normal, if so, continuing synchronizing time, and if not, synchronizing data. And when the synchronization is successful, making a corresponding mark, when the synchronization is failed, recording the synchronization time, synchronizing the last unsynchronized data when the nodes can be synchronized, and manually triggering a log synchronization program after the nodes are switched on.
And the log receiving program receives the log data synchronized by the log synchronizing program and records the log data to the database of the slave node where the log receiving program is located.
In the multi-node cluster, the master node provides service to the outside and processes the log, namely records the operation information of each interface of the master node to generate a record log. When the log is processed, the log synchronization program of the master node synchronizes log data to the slave node at regular time, a synchronization check is executed before each synchronization, whether the slave node can normally receive the log synchronization message is checked, the slave node returns a result to the master node after receiving the synchronization check message of the master node, and if the synchronization condition is met, the log synchronization program synchronously records the log to the slave node. If the synchronization condition is not met, the node is not synchronized at this time.
As shown in fig. 3, the system 300 includes:
the interface recording unit 310 is configured to record all interface operation information of the cluster master node, and generate a record log;
a period setting unit 320 configured to set a log synchronization period;
and the log synchronization unit 330 is configured to periodically synchronize the unit according to the synchronization period, and is configured to periodically update the newly generated log to the slave node according to the synchronization period.
Optionally, as an embodiment of the present invention, the interface recording unit includes:
the address configuration module is configured for setting interface information in a configuration file;
the interface positioning module is configured to call the interface information and position a target interface according to the interface information;
and the operation recording module is configured for recording operation information, encapsulating and desensitizing parameters of the target interface and adding a time stamp after desensitizing treatment is finished.
Optionally, as an embodiment of the present invention, the periodically synchronizing unit according to the synchronization cycle includes:
the receiving and checking module is configured to check whether the receiving module of the slave node is normal or not, and if the receiving module of the slave node is normal, the log is continuously synchronized;
the data marking module is configured to mark the data synchronized to the slave node as synchronized data and record the current synchronization time;
and the data sending module is configured to synchronize the data without the synchronization mark to the slave node.
Fig. 4 is a schematic structural diagram of a terminal system 400 according to an embodiment of the present invention, where the terminal system 400 may be used to execute the method for centralized management and control of cluster logs according to the embodiment of the present invention.
The terminal system 400 may include: a processor 410, a memory 420, and a communication unit 430. The components communicate via one or more buses, and those skilled in the art will appreciate that the architecture of the servers shown in the figures is not intended to be limiting, and may be a bus architecture, a star architecture, a combination of more or less components than those shown, or a different arrangement of components.
The memory 420 may be used for storing instructions executed by the processor 410, and the memory 420 may be implemented by any type of volatile or non-volatile storage terminal or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The executable instructions in memory 420, when executed by processor 410, enable terminal 400 to perform some or all of the steps in the method embodiments described below.
The processor 410 is a control center of the storage terminal, connects various parts of the entire electronic terminal using various interfaces and lines, and performs various functions of the electronic terminal and/or processes data by operating or executing software programs and/or modules stored in the memory 420 and calling data stored in the memory. The processor may be composed of an Integrated Circuit (IC), for example, a single packaged IC, or a plurality of packaged ICs connected with the same or different functions. For example, the processor 410 may include only a Central Processing Unit (CPU). In the embodiment of the present invention, the CPU may be a single operation core, or may include multiple operation cores.
A communication unit 430, configured to establish a communication channel so that the storage terminal can communicate with other terminals. And receiving user data sent by other terminals or sending the user data to other terminals.
The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program may include some or all of the steps in the embodiments provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a Random Access Memory (RAM).
Therefore, the invention can reduce the inconsistency of the log format and granularity caused by adding the log in the interface, and can not provide service for the front end. The main node faults are reduced, time consumption when switching is needed is reduced, and user experience is improved. The log data is effectively backed up, and the master node and the slave node are seamlessly switched, and the technical effect that can be achieved by the embodiment can be referred to the description above, and is not described herein again.
Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied in the form of a software product, where the computer software product is stored in a storage medium, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like, and the storage medium can store program codes, and includes instructions for enabling a computer terminal (which may be a personal computer, a server, or a second terminal, a network terminal, and the like) to perform all or part of the steps of the method in the embodiments of the present invention.
The same and similar parts in the various embodiments in this specification may be referred to each other. Especially, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the description in the method embodiment.
In the embodiments provided in the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, systems or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A centralized management and control method for cluster logs is characterized by comprising the following steps:
recording operation information of all interfaces of a cluster main node to generate a record log;
setting a log synchronization period;
and synchronously updating the newly generated record logs to the slave nodes according to the synchronization period.
2. The method of claim 1, wherein the logging all interface operation information of the cluster master node and generating a log record comprises:
setting interface information in a configuration file;
calling the interface information and positioning a target interface according to the interface information;
and recording operation information, packaging parameters and desensitizing treatment of the target interface, and adding a time stamp after the desensitizing treatment is finished.
3. The method of claim 1, wherein the synchronously updating the newly generated log records to the slave node according to the synchronization period comprises:
checking whether a receiving module of the slave node is normal, and if so, continuing to synchronize the log;
making a synchronized mark on data synchronized to a slave node and recording the synchronization time;
data without a synchronization mark is synchronized to the slave node.
4. A cluster log centralized management and control system is characterized by comprising:
the interface recording unit is configured to record all interface operation information of the cluster master node and generate a recording log;
a period setting unit configured to set a log synchronization period;
and the periodic synchronization unit is configured to synchronously update the newly generated record log to the slave node according to the synchronization period.
5. The system of claim 4, wherein the interface recording unit comprises:
the address configuration module is configured for setting interface information in a configuration file;
the interface positioning module is configured to call the interface information and position a target interface according to the interface information;
and the operation recording module is configured for recording operation information, encapsulating and desensitizing parameters of the target interface and adding a time stamp after desensitizing treatment is finished.
6. The system of claim 4, wherein the periodic synchronization unit according to the synchronization cycle comprises:
the receiving and checking module is configured to check whether the receiving module of the slave node is normal or not, and if the receiving module of the slave node is normal, the log is continuously synchronized;
the data marking module is configured to mark the data synchronized to the slave node as synchronized data and record the current synchronization time;
and the data sending module is configured to synchronize the data without the synchronization mark to the slave node.
7. A terminal, comprising:
a processor;
a memory for storing instructions for execution by the processor;
wherein the processor is configured to perform the method of any one of claims 1-3.
8. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010470321.3A CN111752907A (en) | 2020-05-28 | 2020-05-28 | Centralized management and control method, system, terminal and storage medium for cluster logs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010470321.3A CN111752907A (en) | 2020-05-28 | 2020-05-28 | Centralized management and control method, system, terminal and storage medium for cluster logs |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111752907A true CN111752907A (en) | 2020-10-09 |
Family
ID=72673636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010470321.3A Withdrawn CN111752907A (en) | 2020-05-28 | 2020-05-28 | Centralized management and control method, system, terminal and storage medium for cluster logs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111752907A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112817989B (en) * | 2021-01-22 | 2023-07-25 | 网易（杭州）网络有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
-
2020
- 2020-05-28 CN CN202010470321.3A patent/CN111752907A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112817989B (en) * | 2021-01-22 | 2023-07-25 | 网易（杭州）网络有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111124277B (en) | Deep learning data set caching method, system, terminal and storage medium | |
| US8301600B1 (en) | Failover recovery in a distributed data store | |
| CN112286905A (en) | Data migration method and device, storage medium and electronic equipment | |
| CN109558215A (en) | Backup method, restoration methods, device and the backup server cluster of virtual machine | |
| CN108319617B (en) | Method and device for determining master-slave difference of database and switching control method and device | |
| CN112235136B (en) | Network file system backup method, system, terminal and storage medium | |
| CN111352700A (en) | Method, system, terminal and storage medium for online migration of virtual machine across clouds | |
| CN112463239A (en) | BIOS and BMC interaction option default value modification method, system, terminal and storage medium | |
| CN109361777B (en) | Synchronization method, synchronization system and related device for distributed cluster node states | |
| CN110659158A (en) | Influx DB data backup method based on dual-computer hot standby environment | |
| CN108810127A (en) | Disaster recovery method based on block chain and device | |
| CN111552701A (en) | Method for determining data consistency in distributed cluster and distributed data system | |
| CN111506331A (en) | Server BMC refreshing method, system, terminal and storage medium | |
| CN111752907A (en) | Centralized management and control method, system, terminal and storage medium for cluster logs | |
| CN114218193A (en) | Data migration method and device, computer equipment and readable storage medium | |
| CN114565502A (en) | GPU resource management method, scheduling method, device, electronic equipment and storage medium | |
| CN114020279A (en) | Application software distributed deployment method, system, terminal and storage medium | |
| CN111737226B (en) | Method for optimizing HBase cluster performance based on Redis cluster | |
| CN110851528B (en) | Database synchronization method and device, storage medium and computer equipment | |
| CN114610817B (en) | Data synchronization method and device, multi-activity system, electronic equipment and storage medium | |
| CN112003730A (en) | Method, system, terminal and storage medium for rapid cluster deployment | |
| CN114996033A (en) | Engineering data management method, system, terminal and storage medium | |
| CN108804463B (en) | Data synchronization method and device of MySQL database and electronic equipment | |
| CN114301777A (en) | Configuration data updating method, device and system and non-volatile storage medium | |
| CN112463399A (en) | Server BMC information management method, system, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201009 |
|
| WW01 | Invention patent application withdrawn after publication |