WO2015139565A1

WO2015139565A1 - Heterogeneous logging system management configuration

Info

Publication number: WO2015139565A1
Application number: PCT/CN2015/073847
Authority: WO
Inventors: Mengwen LIU
Original assignee: Hangzhou H3C Technologies Co Ltd
Current assignee: Hangzhou H3C Technologies Co Ltd
Priority date: 2014-03-17
Filing date: 2015-03-09
Publication date: 2015-09-24
Anticipated expiration: 2016-09-17
Also published as: CN104935444A; CN104935444B

Abstract

A heterogeneous logging system is configured by receives a log configuration management command or a log data management command and determines a target logging system. The target logging system is assigned a log adapter adapted to the target logging system. The assigned log adapter is used to complete log management configuration of the target logging system based on the log configuration management command; or the assigned log adapter is used to complete log data management configuration of the target logging system based on the log data management command.

Description

HETEROGENEOUS LOGGING SYSTEM MANAGEMENT CONFIGURATION

Background

The logging system is an important part of a network management application. The logging system can record user behaviors in a network, related information of software, system errors, and network events, etc. and store them according to certain rules. Network administrators can monitor the operating status of the system, check why errors occurred, track triggers of operations, and if the network is under attack they may trace the attacker by analyzing the log data. In addition, network administrators may optimize network system performance, improve user behaviors in the network system, and audit network system security by collecting and analyzing log data.

Brief Description of Drawings

Features of the present disclosure are illustrated by way of an example and not limited in the following figure (s) , in which like numerals indicate like elements, in which:

FIG. 1 is an example of a hardware architecture diagram of an equipment for a heterogeneous logging system management configuration logic of the present disclosure；

FIG. 2 is an example of a block diagram of a heterogeneous logging system management configuration logic of the present disclosure；

FIG. 3 is an example of a system architecture diagram of a heterogeneous logging system management configuration logic of the present disclosure；

FIG. 4 is an example of a logic architecture diagram of a configuration management module in a heterogeneous logging system management configuration logic of the present disclosure； and

FIG. 5 is an example flowchart illustrating the procedures of a method for heterogeneous logging system management configuration of the present disclosure.

Detailed Description

For simplicity and illustrative purposes, a disclosure is described by referring mainly to an example thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. It will be readily apparent however, that the disclosure may be practiced without limitation to these specific details. In other instances, some methods and structures have not been described in detail so as not to unnecessarily obscure the disclosure. As used herein, the terms “a” and “an” are intended to denote at least one of a particular element, the term “includes” means includes but not limited to, the term “including” means including but not limited to, and the term “based on” means based at least in part on.

A typical network management application has a heterogeneous logging system. That is the network application has a plurality of different logging systems, and each logging system may correspond to a particular application function. The plurality of logging systems may have a variety of types, sources, and implementations. For example: if the logs are classified according to the log type, the logs can be divided into user operation logs, application system logs, and security logs； if the logs are classified according to the log source, the logs can be divided into network management application system record logs, network equipment report logs, and third-party collaboration application record logs； and if the logs are classified according to implementations of logging systems, the logs can be divided into Log4j logs (for Java Applications) , LogBack logs, JDKLog logs, XLog logs (for C++Applications) , persisted in relational database user operation logs, and Syslog logs (for network equipment) .

The logs with different types, sources, and implementations usually have their own unique design and configuration specifications. The different logs usually record, store, and manage data in accordance with their inherent implementations. For example, the log recording types may include various attributes of log data records, such as record ID, content format, time stamp mode, encoding format, and etc. The log storage types may include file, relational database, non-relational database, equipment MIB, and etc. The log division types may include file, database table, divisions of MIB node, such as divisions in accordance with date and time, capacity, and corresponding division strategies. The log backup strategies may include transferring history logs to local files and compressing to be stored, backup time period, and the name rule of the compressed file, and etc. The log cleaning strategies may include log saving time window, remove of long history logs, cleanup rules, and etc.

For the above reasons, it is difficult for user of the network management system to know the various record, storage and management policies and other implementation details of different logging systems. This is particularly the case because the implementation details of logging systems, their application functions and their recording types are usually not made public. Hence, it is difficult for network administrators or maintenance personnel of the network management system to achieve unified management and configuration of a heterogeneous logging system. It is also difficult to timely and effectively configure the error-related logging system or collect the error-related log records when errors occur, which in turn makes it difficult to promptly and effectively solve errors in the network.

In light of the above, the present disclosure proposes a heterogeneous logging system management configuration logic. For example, as shown in FIG. 1, the heterogeneous logging system management configuration logic 2 may be in the form of machine readable instructions stored in a storage 12 of a server. The hardware architecture of the server may include a processor (such as a CPU) 11 and the storage 12. The hardware architecture of the server may also include a network interface 13 for transmitting and receiving data, wherein the elements can be connected to each other through internal bus 13.

As shown in FIG. 2, in an example, the heterogeneous logging system management configuration logic of the present disclosure can be further divided into an inter-operating module 21, an adapting module 23, and a configuration management module 22. The adapting module 23 may communicate with a heterogeneous logging system 3. The configuration management module 22 is coupled with and communicates with the inter-operating module 21 and the adapting module 23. The inter-operating module 21 may distribute a log configuration management command and a log data management command to the configuration management module 22. The adapting module 23 may include a plurality of log adapters, wherein each log adapter is adapted to a particular logging system.

By “adapted to a logging system” it is meant that the log adapter is specifically designed for, or configured to interact with, a logging system of that type. For instance, a Log4j adaptor is adapted to a Log4j logging system, an XLog adaptor is adapted to an XLog logging system and an operation log adaptor is adapted to a user operation logging system. A log adaptor may use the same log recording types, log storage types, log division types, log back up strategies and log cleaning strategies as the logging system which it is adapted for.

A logging system may be assigned a log adapter which is adapted to it. E.g. a Log4j adaptor may be assigned to a Log4j logging system, an Xlog adaptor may be assigned to an XLog logging system, and etc.

The configuration management module 23 may use the assigned log adapter to complete the log management configuration of the logging system adapted to the log adapter based on the log configuration management command； or it may use the assigned log adapter to complete the log data management configuration of the logging system based on the log data management command.

In this example, the inter-operating module 21 is essentially a concise abstraction and combination for various log management functions encapsulated by the configuration management module 22, so as to provide the user with the option of a series of functional entries of log configuration management and log data management, or provide the third-party tool with a series of functional interfaces. As shown in FIG. 3, the inter-operating module 21 may further include a log configuration management interface 211 and a log data management interface 212. Herein the log configuration management interface 211 may be used for providing the user with the option of log management configuration loading/modifying entries of the logging system, for example, log level setting entries, log output mode configuration entries, and log storage management entries with the dedicated application function. The log data management interface 212 may be used for providing the user with the option of log data management entries of the logging system, for example, log viewer entries, log data collection entries, log cleaning entries, log backup entries, and log file download entries with the dedicated application function. Therefore, the inter-operating module 21 is able to allow the users, such as Administrators, to distribute the log configuration management command of the logging system to the configuration management module 22 through the log configuration management interface 211 or to distribute the log data management command of the logging system to the configuration management module 22 through the log data management interface 212.

In an example, the inter-operating module 21 may further include a configuration management interface and a data management interface, wherein the configuration management interface is used for providing the third-party application with the log configuration management interface of the logging system, and the data management interface is used for providing the third-party application with the log data management interface of the logging system. Therefore, the inter-operating module 21 can be used by the third-party application, such that the third-party application may distribute the log configuration management command of the logging system to the configuration management module 22 through the log configuration management interface 211, or the third-party application may distribute the log data management command of the logging system to the configuration management module 22 through the log data management interface 212.

The functions of the configuration management module 22 are implemented based on original metadata configuration entries that each logging system registers to the network management system. The metadata configuration entries define and describe the logging systems used in different application functions in the network management system. In an example, typical metadata configuration entries may include any of the contents shown in the following table:

The adapting module 23 may include a number of log adapters 231, wherein each of the log adapters 231 is associated with a logging system. The adapting module 231 upwardly provides a unified interface to the configuration management function and data management function of the configuration management module 22, and downwardly uses the corresponding logging system infrastructure to provide supports of the appointed configuration management function and the data management function of the log adapter interface. For example, as shown in FIG. 3, the heterogeneous logging system may include a Log4j logging system, an XLog logging system, and a user operation logging system, and thus the adapting module 23 will correspondingly include a Log4j log adapter, an XLog log adapter, and a user operation log adapter. In other examples, the adapting module 23 may further include a LOGBack log adapter, a JDKLog log adapter, a Syslog log adapter, and/or a customized log adapter, which can facilitate the third-party implementers to expand the scope of the heterogeneous logging system management configuration logic in order to meet their application requirements.

Turning back to the configuration management module 22, an example is shown in more detail in FIG. 4. The configuration management module 22 may further include a metadata configuration management unit 221, a pattern matching unit 222, a log configuration management unit 223, and a log data management unit 224.

Herein, the metadata configuration management unit 221 is mainly used for performing loading, receipt and parsing of the metadata configuration entries that each logging system registered in the network management system.

The pattern matching unit 222 is mainly used for maintaining the logging system and/or matching data based on the metadata configuration entries. For example, the pattern matching unit 222 may be used to maintain the logging system based on the log file or database table name pattern and file compression pattern defined in the metadata configuration entries. Additionally, the pattern matching unit 222 may also be used to match the log file or the database table of a configuration entry with a dedicated application function in a predetermined time point or a predetermined time period.

The log configuration management unit 223 is mainly used for assigning a log adapter to complete the log management configuration of the logging system adapted to the log adapter based on the log configuration management command. For example, the log configuration management unit 223 may be used to process the configuration management such as log level and log output mode with the dedicated application function, wherein the configuration management is executed by the assigned log adapter.

The log data management unit 224 is mainly used for assigning a log adapter to complete the data management configuration of the logging system adapted to the log adapter based on the log data management command. For example, the log data management unit 224 may be used to perform the log data management configuration entries of log acquisition, log cleaning, log backup, and log data collection with a dedicated application function based on the log data management command, wherein the log data management configuration entries is executed by the assigned log adapter. Additionally, since the log data management unit 224 is implemented based on the metadata configuration entries of each logging system, the log data management unit 224 may also be used to perform data fineness exhibition and collection of dedicated configuration entries or dedicated application function according to the keywords in the log file for corresponding log contents of metadata configuration entries when a number of logs are stored in the same log file.

Please note that, the modules or units above mentioned in the examples of the present disclosure can be integrated in one component, or can be deployed separately； that is to say, they can be combined into a single module or a single unit, or they can be split into a plurality of sub-units. In addition, the modules or units mentioned in the examples of the present disclosure can be implemented by software (such as, machine readable instructions stored in a non-transitory computer readable storage medium to be executed by a processor) or hardware (such as, a processor of Application Specific Integrated Circuit, ASIC) , or a combination thereof, which should not be a limitation of the present disclosure.

In accordance with the above mentioned heterogeneous logging system management configuration logic, the present disclosure also provides a method for heterogeneous logging system management configuration. In the following, an example that the heterogeneous logging system management configuration logic implemented in the form of a computer software product is taken for describing the method of the present disclosure. In this example, the heterogeneous logging system management configuration logic is implemented by a plurality of machine readable instructions stored in a storage, and the method is implemented by using a CPU to read the software (i.e., the machine readable instructions) stored in the storage. As shown in FIG. 5, in this example, the method includes the following steps.

At block 501, the log configuration management command or the log data management command distributed by Administrator is received.

At block 502, a target logging system is determined, and a log adapter adapted to the target logging system is assigned.

At block 503, the assigned log adapter is used to complete log management configuration of the logging system based on the log configuration management command； or the assigned log adapter is used to complete log data management configuration of the logging system based on the log data management command.

Furthermore, the method may further include the following steps implemented by a CPU to read the machine readable instructions stored in the storage to:

complete loading, receipt, and parsing of metadata configuration entries that each logging system registered in a network management system； and

maintain the logging system or match data based on the metadata configuration entries that each logging system registered in the network management system.

maintain the logging system based on log file or database table name pattern, and file compression pattern defined in the metadata configuration entry； or

match the log file or database table with a dedicated application function in a predetermined time point or a predetermined time period.

Herein the above mentioned log configuration management command may include: log level setting, log output mode configuration, and/or log storage management with the dedicated application function； and the above mentioned log data management command may include: log viewer, log data collection, log cleaning, log backup, and/or log file download with the dedicated application function.

load and/or modify the log management configuration for the log level or the log output mode with the dedicated application function based on the log configuration management command； wherein loading and/or modifying the log management configuration are executed by the assigned log adapter.

perform the log data management configuration of log acquisition, log cleaning, log backup, and log data collection with a dedicated application function based on the log data management command； wherein the log data management configuration is executed by the assigned log adapter.

provide a log configuration management interface of the logging system to third-party applications； and

provide a log data management interface of the logging system to the third-party applications.

As can be known from the above mentioned examples of the present disclosure, the heterogeneous logging system management configuration device of the present disclosure at least includes the following advantages.

Concise inter-operating module: network administrators can perform configuration modification and data management upon the heterogeneous logging system by using configuration modification interface and data management interface, and third-party tools can perform configuration modification and data management upon the heterogeneous logging system by using the open interfaces, such that no need to focus on the implementation details such as log recording type, log configuration file location, log modification, log storage, and log segment, which can minimize complexity of use.

Unified management of Log Configuration: consistent configuration modification and data management can be performed on the heterogeneous logging system by applying the metadata configuration entries of various log adapters and various logging systems, which can shield configuration differences of multi-functions and implementation differences of the heterogeneous logging system.

Log collection download of fineness: when multiple logs are stored in the same log file, data fineness exhibition and collection of dedicated configuration entries or dedicated application function can be performed.

Device extension: third-party implementers can build their own configuration management policies by implementing and extending the metadata management unit, the pattern matching unit, the configuration modification unit, and the data management unit interface of the configuration management module, and can extend customized log adapters by implementing the log adapter interface, in order to meet their own application requirements.

The figures are only illustrations of an example, wherein the units or procedure shown in the figures are not necessarily essential for implementing the disclosure. The units in the device in the example can be arranged in the device in the examples as described, or can be alternatively located in one or more devices different from that in the examples. The units in the examples described can be combined into one module or further divided into a plurality of sub-units.

Although the flowcharts described show a specific order of execution, the order of execution may differ from that which is depicted. For example, the order of execution of two or more blocks may be changed relative to the order shown. Also, two or more blocks shown in succession may be executed concurrently or with partial concurrence. All such variations are within the scope of the disclosure.

Throughout the disclosure, the word "comprise" , or variations such as "comprises" or "comprising" , will be understood to imply the inclusion of a stated element, integer, block, or group of elements, integers, block, but not the exclusion of any other element, integer or block, or group of elements, integers or blocks.

Numerous variations and/or modifications may be made to the above-described embodiments, without departing from the broad general scope of the disclosure. The embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

A non-transitory computer readable storage medium on which is stored machine readable instructions that when executed by a processor cause the processor to:

receive a log configuration management command or a log data management command directed to one of a plurality of logging systems in a heterogeneous logging system；

determine a target logging system of the log configuration management command or log data management command, and assign a log adaptor adapted to the target logging system； and

use the assigned log adapter to complete a log management configuration of the target logging system based on the log configuration management command； or use the assigned log adapter to complete a log data management configuration of the target logging system based on the log data management command.
The non-transitory computer readable storage medium according to claim 1, wherein the machine readable instructions are further to cause the processor to:

perform loading, receipt, and parsing of metadata configuration entries for each logging system registered in a network management system； and

maintain the logging system or match predetermined target data based on the metadata configuration entries that each logging system registered in the network management system.
The non-transitory computer readable storage medium according to claim 2, wherein the machine readable instructions are further to cause the processor to:

maintain the logging system based on a log file or a database table name pattern, and a file compression pattern defined in the metadata configuration entries； or

match the log file or database table with a dedicated application function in a predetermined time point or a predetermined time period.
The non-transitory computer readable storage medium according to claim 2, wherein the machine readable instructions are further to cause the processor to:

load and/or modify log management configuration entries for a log level or a log output mode with a dedicated application function based on the log configuration management command, wherein loading and/or modifying the log management configuration entries are executed by the assigned log adapter.
The non-transitory computer readable storage medium according to claim 2, wherein the machine readable instructions are further to cause the processor to:

load and/or modify log data management configuration entries of log acquisition, log cleaning, log backup, and log data collection with a dedicated application function based on the log data management command, wherein the log data management configuration entries are executed by the assigned log adapter.
The non-transitory computer readable storage medium according to claim 1, wherein the machine readable instructions are further to cause the processor to:

provide a log configuration management entry interface of the logging system to a user； and

provide a log data management entry interface of the logging system to the user.
The non-transitory computer readable storage medium according to claim 6, wherein,

the log management configuration includes log level setting, log output mode configuration, and/or log storage management with the dedicated application function； and

the log data management configuration includes log viewer, log data collection, log cleaning, log backup, and/or log file download with the dedicated application function.
The non-transitory computer readable storage medium according to claim 7, wherein the machine readable instructions are further to cause the processor to:

provide a log configuration management interface of the logging system to third-party tools； and

provide a log data management interface of the logging system to the third-party tools.
A method for heterogeneous logging system management configuration, comprising:

receiving a log configuration management command or a log data management command distributed by an administrator；

determining a target logging system, and assigning a log adapter adapted to the target logging system； and

using the assigned log adapter to complete log management configuration of the target logging system based on the log configuration management command； or using the assigned log adapter to complete log data management configuration of the target logging system based on the log data management command.
A method according to claim 9, further comprising:

completing loading, receipt, and parsing of metadata configuration entries that each logging system registered in a network management system； and

maintaining the logging system or matching predetermined target data based on the metadata configuration entries that each logging system registered in the network management system.
A method according to claim 10, further comprising:

maintaining the logging system based on a log file or a database table name pattern and a file compression pattern defined in the metadata configuration entry； or

matching the log file or database table with a dedicated application function in a predetermined time point or a predetermined time period.
A method according to claim 11, wherein,

the log configuration management command comprises:

log level setting, log output mode configuration, and/or log storage management with the dedicated application function； and

the log data management command comprises:

log viewer, log data collection, log cleaning, log backup, and/or log file download with the dedicated application function.
A method according to claim 12, further comprising:

loading and/or modifying the log management configuration for the log level or the log output mode with the dedicated application function based on the log configuration management command, wherein loading and/or modifying the log management configuration are executed by the assigned log adapter； and

performing the log data management configuration of log acquisition, log cleaning, log backup, and log data collection with the dedicated application function based on the log data management command, wherein the log data management configuration is executed by the assigned log adapter.
A method according to claim 9, further comprising:

providing a log configuration management interface of the logging system to third-party applications； and

providing a log data management interface of the logging system to the third-party applications.