CN110995500A - Node log management and control method, system and related components - Google Patents
Node log management and control method, system and related components Download PDFInfo
- Publication number
- CN110995500A CN110995500A CN201911297397.4A CN201911297397A CN110995500A CN 110995500 A CN110995500 A CN 110995500A CN 201911297397 A CN201911297397 A CN 201911297397A CN 110995500 A CN110995500 A CN 110995500A
- Authority
- CN
- China
- Prior art keywords
- node
- log
- uploading
- configuration information
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 91
- 238000007726 management method Methods 0.000 claims description 35
- 238000004458 analytical method Methods 0.000 claims description 33
- 238000012544 monitoring process Methods 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000005856 abnormality Effects 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000010365 information processing Effects 0.000 abstract description 9
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 35
- 230000006870 function Effects 0.000 description 5
- 230000001960 triggered effect Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a node log control method, which determines a target node needing to upload logs at present by determining configuration information, controls the target node to upload log information, acquires the node log information needing to be collected at present, can avoid the situation that useful information is automatically filtered and lost in the traditional log collection, effectively retains the useful node detail data, ensures the comprehensiveness of data, avoids the occupation of useless node information on a memory and information processing resources, saves system resources, and ensures the high efficiency of useful information processing. The application also provides a server, a node and a node log management and control system, and the server, the node and the node log management and control system have the beneficial effects.
Description
Technical Field
The present application relates to the field of electronic technologies, and in particular, to a node log management and control method based on a central server, a node log management and control method based on a node, a server, a node, and a node log management and control system.
Background
With the wide use of content distribution networks, a large number of service node devices exist in the networks, how to ensure the stability of the system and monitor the service quality of the whole system is very important for the normal operation of the system. Wherein, observing and analyzing the log is an important and effective monitoring means.
The traditional method mainly monitors important operations of service node equipment through a point similar to app, but has the problem of incomplete data, relates to more detailed observation, and also needs related technical personnel to log in specific service node equipment to check logs; and the direct login of the equipment easily causes the leakage of the equipment and influences the safety of system information.
How to solve the negative influence of the logs on the system stability of a large number of service node devices is a technical problem to be solved by the technical personnel in the field.
Disclosure of Invention
The application aims to provide a node log control method based on a central server, a node log control method based on a node, a server, a node and a node log control system, and aims to solve the problem that the stability of the system is negatively affected by logs under a large number of nodes.
In order to achieve the above object, the present application provides a node log management and control method, including:
the central server determines configuration information; the configuration information comprises target node information needing log uploading;
controlling the target node to upload the node log;
and receiving and processing the node log.
Optionally, the determining, by the central server, configuration information includes:
the central server determines the current monitoring requirement;
and analyzing the monitoring requirement and determining corresponding configuration information.
Optionally, the central server determines the current monitoring requirement, including:
when the node abnormality exists, generating a monitoring requirement of a node to be monitored; the nodes to be monitored comprise abnormal nodes and associated nodes of the abnormal nodes.
Optionally, the receiving and storing the node log includes:
receiving a node log of the target node;
and uploading the node log to a network inquiry terminal so as to respond to a log viewing request of a user.
Optionally, before uploading the node to the network query end, the method further includes:
carrying out data preprocessing on the node logs corresponding to the nodes; wherein the data preprocessing comprises: data filtering, data filling and format conversion;
correspondingly, the uploading the node to the network query end specifically includes: and uploading the preprocessed node log to the network query terminal.
Optionally, before uploading the node log to a network query end, the method further includes:
carrying out data encryption processing on the node log;
correspondingly, the uploading the node log to a network query end specifically includes: and uploading the encrypted node log to the network query terminal.
Optionally, after uploading the node log to a network query end, the method further includes:
and the network inquiry terminal performs system analysis on the running state among the nodes according to the node log to generate a system node comprehensive analysis result so as to respond to a system comprehensive state analysis request of a user.
In order to achieve the above object, the present application provides a node log management and control method, including:
the method comprises the steps that a node acquires configuration information from a central server at regular time, wherein the configuration information comprises target node information needing log uploading;
detecting the configuration information, and judging whether the node needs to upload log information or not;
if yes, acquiring a node log;
and uploading the node log to the central server.
Optionally, the node is deployed with a plug-in and a file data collector, where the plug-in is used to detect the configuration information and determine whether the node needs to upload log information;
and when the plug-in judges that the node needs to upload the log, informing the file data collector to collect log data and uploading the log data to a central server.
To achieve the above object, the present application provides a server comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the node log management and control method when executing the computer program.
To achieve the above object, the present application provides a node, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the node log management and control method when executing the computer program.
In order to achieve the above object, the present application provides a node log management and control system, including:
the node is used for acquiring configuration information from the central server at regular time, wherein the configuration information comprises target node information needing log uploading; detecting the configuration information, and judging whether the node needs to upload log information or not; if yes, acquiring a node log; uploading the node log to the central server;
the central server is used for determining configuration information; the configuration information comprises target node information needing log uploading; controlling the target node to upload the node log; and receiving and processing the node log.
According to the node log control method provided by the application, the target node which needs to upload the log at present is determined by determining the configuration information, the target node is controlled to upload the log information, the node log information which needs to be collected at present is obtained, the situation that the useful information is automatically filtered and lost in the traditional log collection can be avoided, the useful node detail data is effectively reserved, the comprehensiveness of the data is ensured, meanwhile, the occupation of the memory and information processing resources by the useless node information is avoided, the system resources are saved, and the high efficiency of useful information processing is ensured.
This application still provides a server, a node and a node log management and control system simultaneously, has above-mentioned beneficial effect, no longer gives unnecessary details here.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a node log management and control method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a server according to an embodiment of the present application;
fig. 3 is a flowchart of another node log management and control method according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
Example one
Referring to fig. 1, fig. 1 is a flowchart of a node log management and control method provided in an embodiment of the present application, and it should be noted that an execution subject of each step in the embodiment is a central server in a node log management and control system, where the central server may be used to take charge of one or more functions of file indexing, heat calculation, file deployment, service scheduling, and the like, and mainly includes the following steps:
s101: the central server determines the configuration information.
The method includes the steps that configuration information uploaded by system node logs is configured in advance, the configuration information specifically comprises system node information which can be used for determining log information to be uploaded, specifically identity information, parameter type information to be collected in the system nodes and the like. The granularity of the configuration information is not limited, the granularity refers to the range of the monitoring node, can be a single node or a plurality of nodes, and can be set according to the monitoring requirement of the actual node.
The configuration information is a unique identifier, specifically designates one or a group of nodes to upload logs, and tracks the running condition of the logs, or is a node unique attribute, such as designating and analyzing a group of nodes with an external network ip, or designating and tracking the running condition of a new version node. Specifically, the configuration information may be configured according to node information such as province, large area, operator, network type, and version of deployed application program where the node is located, where one type of configuration information is: province-version number-operator, another kind of configuration information such as: region-version number-operator.
In addition, the manner in which the central server acquires the configuration information is not limited, and the configuration information may be automatically generated (for example, triggered by buried point data, when the system finds that some nodes are abnormal, the relevant nodes are immediately updated to be in a log reporting state), or may be set by a user, for example, the central server directly receives the configuration information set by the user, and the like.
S102: and controlling the target node to upload the node log.
The configuration information includes information, such as a version number, an area range, and the like, which can be used to determine a system node to which log information is to be uploaded, and a specific node to be monitored needs to be determined from the configuration information and is used as a target node.
And controlling the target node to upload the collected node log of the node. The node logs may be collected when the central server initiates a log feedback instruction to the node, or may be collected in advance, and the collected node logs are directly fed back when the central server initiates a log feedback instruction, which is not limited herein.
S103: the node log is received and processed.
And after receiving the node logs fed back by the target node, performing information processing on the received log information so as to facilitate the viewing of subsequent node logs. In this embodiment, the information processing process of the node log is not limited, and the existing information processing process may be referred to, which is not described herein again.
It should be noted that, the above description only introduces the single log collection uploading process, and the steps S101 to S103 may be triggered at regular time, so as to ensure the stability of the long-term operation of the system.
According to the node log control method provided by the embodiment, the method is different from the traditional node log control scheme, the target node which needs to upload the log at present is determined by determining the configuration information, the target node is controlled to upload the log information, the node log information which needs to be collected at present is obtained, the situation that useful information is lost due to automatic filtration in traditional log collection can be avoided, useful node detail data are effectively reserved, the comprehensiveness of data is guaranteed, meanwhile, occupation of useless node information on an internal memory and information processing resources is avoided, system resources are saved, and the high efficiency of useful information processing is guaranteed.
Example two
In this embodiment, collecting node logs of a target node, in some cases, the number of target nodes that may be to be collected is large, which results in a large amount of collected log data, in order to avoid excessive occupation of the local storage space, and simultaneously avoid cumbersome operation and inconvenience caused by logging in to a corresponding node when checking the running states of nodes in different areas, preferably, each collected node log may be uploaded to a network query end, and then specifically, the process of receiving and processing the node log in step S103 in the first embodiment may specifically include:
(1) receiving a node log of a target node;
(2) and uploading the node log to a network inquiry terminal so as to respond to a log viewing request of a user.
In the traditional method, most of the logs are deleted in a very short time after the node logs are acquired, so that the nodes or the server side can be ensured to keep enough storage space, but the method can cause the mistaken deletion of a large amount of useful log information and influence the operation and maintenance of the system.
In this embodiment, the received node log is uploaded to a network query end, the network query end is an open-source search engine, a query interface can be provided, a user can conveniently view data needing to be paid attention to in any node, the user does not need to log in to an independent node, the local space is not occupied, meanwhile, the long-term effectiveness of node log information is guaranteed, the situation that useful information is automatically filtered out and lost is avoided, the system maintenance is facilitated, and certainly, a part of log can be stored at the local end, and limitation is not performed here.
Specifically, the specific network type of the network query end is not limited, and may be, for example, an Elasticsearch, which can quickly store, search, and analyze mass data. The Elasticsearch provides an http query interface, and can conveniently deploy an alarm script for data needing attention, for example, the number of times of occurrence of a certain log exceeds a set threshold value within a certain period of time, and send alarm information and the like. In this embodiment, the network query end is referred to as an elastic search for example, and other engine types may refer to the description of this embodiment, which is not described herein again.
Because log data is generally unstructured, it contains much unnecessary information, or needs to enrich some fields, such as information about the regional operators where the IP padding plug-in according to Filebeat is located. Further, in order to improve the availability of data, before uploading the node log information to the network query end, data preprocessing can be performed on the node log information corresponding to each node; correspondingly, the process of uploading the node log information to the network query end specifically includes: and uploading the preprocessed node log information to a network query terminal.
The collected log data can be preprocessed, including operations of disassembling, filtering, enriching and the like, and one data preprocessing means includes: useless data in log data are filtered, marks, 0 and the like are refilled so as to distinguish data of all useful information, and then the filled data are converted into the same log data storage format, so that unified data format conversion is realized. The means can delete useless information and carry out data reprocessing, and is convenient for a user to check. The specific data operation means and execution sequence are not limited, and can be set according to the actual data processing requirements.
Further, before uploading the node log to the network query end, data encryption processing can be carried out on the node; accordingly, the process of uploading the node to the network query end specifically includes: and uploading the encrypted log information to a network query terminal. When data security is involved, the data security of important data can be guaranteed through encryption transmission, and the conditions of data leakage and the like in the transmission process are avoided. In this embodiment, the encryption manner is not limited, and may refer to an implementation manner in the related art, for example, the transmission may be encrypted by SSL (Secure Sockets Layer).
Correspondingly, the node may also perform encrypted transmission when uploading the node log to the central server, which is not limited herein, and reference may be made to the above description.
When the central server collects all target node logs, or after the central server sends the node logs of the mobile phone to the network query end, the central server and the network query end both comprise multi-node state information, so that the aim of integrally analyzing the state information among multiple nodes and evaluating the overall performance of the system is fulfilled.
In this embodiment, only the following analysis process of the network query end is taken as an example, and other analysis processes based on the central server of the present application may refer to the following description, which is not described herein again.
Specifically, after the nodes of the target node are collected and uploaded to the network query end, the network query end may further perform system analysis on the operating states among the nodes according to the node logs to generate a system node comprehensive analysis result, so as to respond to a system comprehensive state analysis request of the user.
In this embodiment, the analysis data object and the analysis means specifically included in the system analysis are not limited, and the analysis object may be a node log or may be a part of parameters; the analysis means may be mean analysis, fluctuation analysis, difference analysis, or the like.
The following description will be given by taking the overall analysis process of two system nodes as an example, and the following description can be referred to for other analysis means and the analysis process corresponding to the analysis object, which are not described herein again.
A process for carrying out system analysis on running states among nodes by a network inquiry end according to received node logs specifically comprises the following steps:
(1) the network inquiry end extracts the running state data of each node under the same parameters;
(2) and performing performance comparison analysis under the same parameters among the nodes according to the running state data, and taking the generated result as a system analysis result.
For example, the collected target nodes include: the logs of the three nodes all include data corresponding to the parameters 1, 2 and 3, and the data of the parameters 1 under the three target nodes can be counted as follows: a1, a2, a 3; the data of the parameters 2 under the three target nodes are respectively as follows: b1, b2, b 3; the data of the parameter 1 under the three target nodes are respectively as follows: c1, c2, c 3. The performance comparison analysis between the nodes is carried out according to the data displayed by each node under the same parameter, so that the node differentiation comparison between each parameter type can be realized, the system is close to the working reality, and the abnormal parameters and the abnormal nodes can be conveniently identified.
Another process for performing system analysis on the running state among the nodes by the network query end according to the received node logs specifically comprises the following steps:
(1) grading the running state of the node according to the node log to obtain a state score;
(2) and counting the state scores among the nodes, and taking the statistical result as a system analysis result.
For example, the collected target nodes include: the method comprises the steps that a node 1, a node 2 and a node 3 are arranged, logs of the three nodes all comprise data corresponding to a parameter 1, a parameter 2 and a parameter 3, the data corresponding to each parameter under the node 1 are scored (the parameter 1 is 60, the parameter 2 is 90, and the parameter 3 is 60), the overall state score of the node 1 is counted to be 70 by taking an average value, and by analogy, the state scores of the node 2 and the node 3 are obtained, the analysis on the overall performance of the node can be realized by the analysis method, and an abnormal node in the system is obtained through the overall performance.
In order to monitor the service quality of the system and ensure the time for fault recovery, preferably, after collecting the node log of the target node and uploading the node log to the network querying end, the network querying end may further perform the following steps:
the network inquiry end analyzes the node operation normality according to the node log to generate an analysis result;
and when the analysis result shows that the node operates abnormally, calling an alarm part to output alarm information.
The abnormal operation of each node is monitored, so that the abnormal conditions of the nodes can be found in time, and when the operation of the nodes is abnormal, the alarm part is called to give an alarm in time, so that the fault processing speed of related technicians can be increased, and the normal operation of the system is ensured.
The log alarm mode can be customized and can be set as required, which is not limited in this embodiment, and preferably, a remotely connected client can be called to output system node abnormality alarm information, so that related technical personnel can perform abnormality processing in time. For example, enterprise tools such as nailing and WeChat can be used in time, related technicians can know that the free behavior of the user is stronger by alarming through the client, and the alarm information can be received in time no matter where the user is located, so that the fault can be processed in time.
In addition, based on the solutions described in the above embodiments, the log reporting status of some machines may be further updated according to the collected node logs, so as to determine whether log information of all target nodes is normally collected, and other extensions based on the solutions of the present application are included as a protection scope of the present application and are not described herein again.
EXAMPLE III
In the above embodiment, the manner of determining the configuration information by the central server is not limited, and the configuration information may be set by a user, and the central server directly obtains the configuration information input by the user, for example, the central server inputs corresponding information of the node to be monitored by development, operation and maintenance or related technical personnel of the product (when the gray scale of the plug-in, update, failure, etc.) and the monitoring information is obtained by the central server; or the user inputs the monitoring requirement, and the central server automatically generates corresponding configuration information according to the monitoring requirement; there may also be a central server that automatically specifies and generates configuration information. In this embodiment, several ways for acquiring configuration information are described for the configuration information, and refer to the following description.
The process of determining the configuration information uploaded by the system node log by the central server may specifically include the following steps:
(1) the central server determines the current monitoring requirement;
(2) and analyzing the monitoring requirement and determining corresponding configuration information.
The monitoring requirement refers to a system performance monitoring requirement, for example, a monitoring requirement is to monitor a node having a history failure frequency more than 3 times, and the monitoring requirement may not directly include identity information of a target node, and may also include designation of a node, and when the monitoring requirement does not include the identity information of the target node, node directivity information is required to determine the target node meeting the monitoring requirement.
And after determining the current monitoring requirement, the central server screens the nodes meeting the monitoring requirement to generate corresponding configuration information.
The determination mode of the monitoring requirement in the step (1) is not limited, and the determination mode can be input by a user or automatically triggered by a system, so that automation of log management and control is realized, and meanwhile, the full retention of field information of the system when a fault occurs is ensured, so that subsequent fault troubleshooting is facilitated, the running stability of the system is ensured, and preferably, the monitoring requirement of a node to be monitored can be generated when the node is abnormal; the nodes to be monitored comprise abnormal nodes and associated nodes of the abnormal nodes.
For example, when a node 1 fails (node 1 failure information may be obtained by other node performance monitoring components, or node failure information may be obtained by a log management and control method provided in this embodiment, a determination method of node abnormality is not limited in this embodiment), a system node having service contact with the node 1, such as node 2 and node 3, may be determined, and then configuration information of "performing full parameter performance monitoring on node 1, node 2, and node 3" may be generated.
The configuration information determining method provided by the embodiment is simple to operate on the user side, has strong operability, and can ensure the recovery time of system faults and ensure the stable operation of the system.
Example four
In the above, a process of how a central server implements node log management and control is described in detail through some embodiments, the present application further provides an entity hardware device corresponding to the method, a content principle of this portion corresponds to a scheme portion, a portion of the implementation principle is not described herein again, a hardware composition of the entity hardware device is described below, please refer to fig. 2, and fig. 2 is a schematic structural diagram of a server provided in an embodiment of the present application:
the central server 500 includes a memory 510, a processor 520, and a bus 530, wherein the memory 510 stores a node log management program that can run on the processor 520, the node log management program is transmitted to the processor 520 through the bus 530, and when being executed by the processor 520, the node log management program can implement the steps of the node log management method described in the foregoing embodiment.
The memory 510 includes at least one type of readable storage medium, which includes flash memory, hard disk, multi-media card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, and the like. The memory 510 may be an internal storage unit of the central server 500 in some embodiments, such as a hard disk of the central server 500. The memory 510 may also be an external storage device of the central server 500 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the central server 500. Further, the memory 510 may also be simultaneously composed of an internal storage unit and an external storage device. Further, the memory 510 may be used not only to store various application software and various kinds of data installed in the center server 500, but also to temporarily store data that has been output or will be output.
The bus 530 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one bi-directional hollow indicator line is shown in FIG. 2, but does not indicate only one bus or one type of bus.
EXAMPLE five
Referring to fig. 3, fig. 3 is a flowchart of a node log management and control method provided in the embodiment of the present application, it should be noted that an execution main body of each step in the embodiment is a node in a system, and the node is mainly used for taking charge of functions of collecting, uploading, and the like of a node log, and mainly includes the following steps:
s201: the node acquires configuration information from the central server at regular time, wherein the configuration information comprises target node information needing log uploading;
the node is linked to the central server, and the configuration information including the target node information of the log to be uploaded is acquired at regular time.
S202: detecting configuration information, and judging whether the node needs to upload log information or not; if yes, go to step S203;
s203: acquiring a node log;
the configuration information comprises target node information of the logs to be uploaded, the nodes need to determine whether the nodes belong to the target node row or not after determining the configuration information, and if yes, the node logs of the nodes need to be acquired. The collecting process of the node logs may be triggered after determining that the node itself needs to upload the log information, or may be triggered in advance, and the node logs collected in advance are directly obtained after determining that the node itself needs to upload the log information.
A feasible log collection and uploading implementation mode mainly comprises the following steps:
(1) tracking and reading a node log collection process;
(2) when the collection of the node logs is finished, extracting the node logs;
when the node log collection is completed, the log information of the node is immediately extracted, the time delay caused by process supervision indirectly realized by other components can be avoided by actively tracking the log collection process, and the efficiency of extracting the log information is improved.
Specifically, the foregoing process may be implemented by calling filebear, and for deepening understanding, the foregoing process is specifically described by calling filebear: when the node plug-in detects that the log needs to be reported: and informing Filebeat deployed on the same machine, wherein the Filebeat is a file data collector and can track and read log files (node log files) and upload log data, so that logs of plug-ins running in different areas can be summarized to the central server.
In this embodiment, only the above-mentioned node log collection process is taken as an example for description, and other log collection implementation processes can refer to the description of this embodiment, which is not described herein again.
In addition, if the configuration information is detected and it is determined that the current node does not need to upload the log information, in this embodiment, the implementation manner in this case is not limited, and the process may jump to step S201 as shown in fig. 3, or the current process may be ended.
S204: and uploading the node logs to a central server.
And the nodes upload the collected node logs to a central server so that the central server can process the node logs of all the target nodes conveniently.
It should be noted that the implementation of the above-mentioned node steps may be implemented by directly adjusting a control program of the node itself, or by adding a plug-in or a functional component that can implement the above-mentioned steps, and optionally, the node is deployed with a plug-in and a file data collector, where the plug-in is used to detect configuration information and determine whether the node needs to upload log information; and when the plug-in judges that the node needs to upload the log, informing the file data collector to collect log data and uploading the log data to the central server. Through the addition of the plug-in and the functional components, the damage to the original functional components of the node is avoided, and the normal realization of the functions can be ensured.
It should be noted that, the node log management and control method based on a node described in this embodiment may refer to the description of the node log management and control method based on a central server in the foregoing embodiment, and the two parts may be referred to each other.
EXAMPLE six
In the above, a detailed description has been made on how a node implements a node log management and control process through some embodiments, and the present application further provides an entity hardware device corresponding to the method, where a content principle of the part corresponds to a scheme part, and a part of the implementation principle is not described herein again, and a description will be made below on a hardware composition of the entity hardware device.
Wherein the memory includes at least one type of readable storage medium including flash memory, hard disk, multi-media card, card type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory may in some embodiments be an internal storage unit of the node, e.g. a hard disk of the node. The memory may also be an external storage device of the node in other embodiments, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the node 500. Further, the memory may also be simultaneously composed of an internal memory unit and an external memory device. Further, the memory may be used not only to store various application software and various types of data installed in the node, but also to temporarily store data that has been output or is to be output.
The processor may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip in some embodiments, and is configured to run program code stored in a memory or process data, such as a node log manager or the like.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc.
It should be noted that, the structure of the node described in this embodiment may refer to the schematic structural diagram of the central server and the related description described in the foregoing embodiments, and is not described herein again.
EXAMPLE seven
In the foregoing, a practical electronic device is described in detail, and the present application further provides a corresponding system for implementing a node log management and control function, where the content principle of this portion corresponds to the scheme portion, and the portion implementing the principle is not described herein again, and the following will describe a hardware component of the system, where the system mainly includes: a node and a central server.
The nodes are mainly used for acquiring configuration information from the central server at regular time, wherein the configuration information comprises target node information needing log uploading; detecting configuration information, and judging whether the node needs to upload log information or not; if yes, acquiring a node log; uploading the node logs to a central server;
the central server is connected with nodes of each system and is mainly used for determining configuration information; the configuration information comprises target node information needing log uploading; controlling a target node to upload a node log; the node log is received and processed.
The implementation process of the interaction function between the devices can refer to the node log control method based on the central server and the introduction of the node log control method based on the nodes to the corresponding embodiments, and the node log control system provided by the embodiment can solve the negative influence of the logs on the system stability under the condition of massive nodes and ensure effective node log control.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method provided in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (12)
1. A node log management and control method is characterized by comprising the following steps:
the central server determines configuration information; the configuration information comprises target node information needing log uploading;
controlling the target node to upload the node log;
and receiving and processing the node log.
2. The node log management and control method according to claim 1, wherein the central server determines configuration information, including:
the central server determines the current monitoring requirement;
and analyzing the monitoring requirement and determining corresponding configuration information.
3. The node log management and control method according to claim 2, wherein the central server determines the current monitoring requirement, and includes:
when the node abnormality exists, generating a monitoring requirement of a node to be monitored; the nodes to be monitored comprise abnormal nodes and associated nodes of the abnormal nodes.
4. The node log management and control method according to claim 1, wherein the receiving and processing the node log includes:
receiving a node log of the target node;
and uploading the node log to a network inquiry terminal so as to respond to a log viewing request of a user.
5. The node log management and control method according to claim 4, wherein before uploading the node log to a network query end, the method further comprises:
carrying out data preprocessing on the node logs corresponding to the nodes; wherein the data preprocessing comprises: data filtering, data filling and format conversion;
correspondingly, the uploading the node log to a network query end specifically includes: and uploading the preprocessed log to the network query terminal.
6. The node log management and control method according to claim 4, wherein before uploading the node log to a network query end, the method further comprises:
carrying out data encryption processing on the node log;
correspondingly, the uploading the node log to a network query end specifically includes: and uploading the encrypted node log to the network query terminal.
7. The node log management and control method according to claim 4, wherein after uploading the node log to a network query end, the method further comprises:
and the network inquiry terminal performs system analysis on the running state among the nodes according to the node log to generate a system node comprehensive analysis result so as to respond to a system comprehensive state analysis request of a user.
8. A node log management and control method is characterized by comprising the following steps:
the method comprises the steps that a node acquires configuration information from a central server at regular time, wherein the configuration information comprises target node information needing log uploading;
detecting the configuration information, and judging whether the node needs to upload log information or not;
if yes, acquiring a node log;
and uploading the node log to the central server.
9. The node log management and control method according to claim 8, wherein a plug-in and a file data collector are deployed in the node, and the plug-in is configured to detect the configuration information and determine whether the node needs to upload log information;
and when the plug-in judges that the node needs to upload the log, informing the file data collector to collect log data and uploading the log data to a central server.
10. A server, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the node log management method according to any one of claims 1 to 7 when executing said computer program.
11. A node, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the node log management method according to claim 8 or 9 when executing said computer program.
12. A node log management and control system, comprising:
the node is used for acquiring configuration information from the central server at regular time, wherein the configuration information comprises target node information needing log uploading; detecting the configuration information, and judging whether the node needs to upload log information or not; if yes, acquiring a node log; uploading the node log to the central server;
the central server is used for determining configuration information; the configuration information comprises target node information needing log uploading; controlling the target node to upload the node log; and receiving and processing the node log.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911297397.4A CN110995500A (en) | 2019-12-16 | 2019-12-16 | Node log management and control method, system and related components |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911297397.4A CN110995500A (en) | 2019-12-16 | 2019-12-16 | Node log management and control method, system and related components |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110995500A true CN110995500A (en) | 2020-04-10 |
Family
ID=70094593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911297397.4A Pending CN110995500A (en) | 2019-12-16 | 2019-12-16 | Node log management and control method, system and related components |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110995500A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116579895A (en) * | 2023-04-27 | 2023-08-11 | 广州南翼信息科技有限公司 | Comprehensive intelligent education system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107707414A (en) * | 2017-11-22 | 2018-02-16 | 北京搜狐新媒体信息技术有限公司 | The monitoring system and method for CDN |
| CN109039724A (en) * | 2018-07-23 | 2018-12-18 | 北京奇艺世纪科技有限公司 | Log method for uploading and device |
| US20190108112A1 (en) * | 2017-10-05 | 2019-04-11 | Hcl Technologies Limited | System and method for generating a log analysis report from a set of data sources |
| CN110232048A (en) * | 2019-06-12 | 2019-09-13 | 腾讯科技(成都)有限公司 | Acquisition methods, device and the storage medium of journal file |
| CN110399350A (en) * | 2018-04-19 | 2019-11-01 | 沪江教育科技（上海）股份有限公司 | Processing method, device, storage medium and the electronic equipment of log information |
| CN110457195A (en) * | 2019-08-05 | 2019-11-15 | 深圳乐信软件技术有限公司 | Acquisition methods, device, server and the storage medium of client local log |
| CN110519077A (en) * | 2019-07-26 | 2019-11-29 | 济南浪潮数据技术有限公司 | A kind of Log Collect System, method, apparatus, server and storage medium |
-
2019
- 2019-12-16 CN CN201911297397.4A patent/CN110995500A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190108112A1 (en) * | 2017-10-05 | 2019-04-11 | Hcl Technologies Limited | System and method for generating a log analysis report from a set of data sources |
| CN107707414A (en) * | 2017-11-22 | 2018-02-16 | 北京搜狐新媒体信息技术有限公司 | The monitoring system and method for CDN |
| CN110399350A (en) * | 2018-04-19 | 2019-11-01 | 沪江教育科技（上海）股份有限公司 | Processing method, device, storage medium and the electronic equipment of log information |
| CN109039724A (en) * | 2018-07-23 | 2018-12-18 | 北京奇艺世纪科技有限公司 | Log method for uploading and device |
| CN110232048A (en) * | 2019-06-12 | 2019-09-13 | 腾讯科技(成都)有限公司 | Acquisition methods, device and the storage medium of journal file |
| CN110519077A (en) * | 2019-07-26 | 2019-11-29 | 济南浪潮数据技术有限公司 | A kind of Log Collect System, method, apparatus, server and storage medium |
| CN110457195A (en) * | 2019-08-05 | 2019-11-15 | 深圳乐信软件技术有限公司 | Acquisition methods, device, server and the storage medium of client local log |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116579895A (en) * | 2023-04-27 | 2023-08-11 | 广州南翼信息科技有限公司 | Comprehensive intelligent education system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107704360B (en) | Monitoring data processing method, equipment, server and storage medium | |
| CN113176978B (en) | Monitoring method, system, equipment and readable storage medium based on log file | |
| CN110046073B (en) | Log collection method and device, equipment and storage medium | |
| EP2523115A1 (en) | Operation management device, operation management method, and program storage medium | |
| CN108092836A (en) | The monitoring method and device of a kind of server | |
| CN111831514A (en) | Equipment monitoring method, device, equipment and storage medium | |
| CN109460343A (en) | System exception monitoring method, device, equipment and storage medium based on log | |
| JP2012198796A (en) | Log collection system, device, method and program | |
| CN104899130A (en) | Application program management method and system | |
| US10341182B2 (en) | Method and system for detecting network upgrades | |
| CN110750426A (en) | Service state monitoring method and device, electronic equipment and readable storage medium | |
| CN105610648A (en) | Operation and maintenance monitoring data collection method and server | |
| CN109656786B (en) | Log acquisition method and device | |
| CN113760652B (en) | Method, system, device and storage medium for full link monitoring based on application | |
| CN111212035A (en) | Host computer defect confirming and automatic repairing method and system based on same | |
| CN112306802A (en) | Data acquisition method, device, medium and electronic equipment of system | |
| CN114363151A (en) | Fault detection method and device, electronic equipment and storage medium | |
| CN108809729A (en) | The fault handling method and device that CTDB is serviced in a kind of distributed system | |
| CN107204868B (en) | Task operation monitoring information acquisition method and device | |
| CN110995500A (en) | Node log management and control method, system and related components | |
| CN111526109A (en) | Method and device for automatically detecting running state of web threat recognition defense system | |
| CN113411209A (en) | Distributed password service full-link detection system and method | |
| CN112306871A (en) | Data processing method, device, equipment and storage medium | |
| JP2010244137A (en) | Failure information collection device | |
| CN115794479A (en) | Log data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200410 |