CN112134719A - Method and system for analyzing base station security log - Google Patents
Method and system for analyzing base station security log Download PDFInfo
- Publication number
- CN112134719A CN112134719A CN201910556924.2A CN201910556924A CN112134719A CN 112134719 A CN112134719 A CN 112134719A CN 201910556924 A CN201910556924 A CN 201910556924A CN 112134719 A CN112134719 A CN 112134719A
- Authority
- CN
- China
- Prior art keywords
- log
- data
- safety
- characteristic value
- security
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The embodiment of the invention discloses a method and a system for analyzing a base station security log, wherein the method comprises the following steps: the data aggregation system caches the safety logs reported by the base station in real time to a first data cache system based on a first distributed cluster base station; the log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and caches the feature value of the security log into the second data cache system; the characteristic value real-time analysis system analyzes the characteristic value of the safety log in real time based on a stream type calculation frame to obtain an analysis result; and the data presentation device displays the analysis result. The embodiment of the invention realizes the real-time analysis of the safety log reported by the base station in real time based on the distributed cluster and the streaming computing framework, thereby shortening the discovery time of high-risk safety problems, greatly reducing the time of manual analysis, shortening the problem feedback period, reducing the labor input and improving the management efficiency.
Description
Technical Field
The embodiments of the present invention relate to, but not limited to, the field of computers and communications, and more particularly, to a method and system for analyzing a security log of a base station.
Background
With the development of 5G wireless network technology, the number of base stations in a wireless access network is increasing, and the requirements for acquisition and analysis of instruction operation and data operation and access security of the base stations are more and more important. The traditional base station safety logs are acquired from a base station through a timing task period, all the safety logs are analyzed and stored in a warehouse after acquisition, and after a fault occurs, manual analysis is carried out. 5G, the network scale is greatly expanded along with the development of wireless networks and Internet of things, and a large amount of manpower needs to be invested in the face of thousands of security logs in the existing network every day.
Disclosure of Invention
The embodiment of the invention provides a method and a system for analyzing a base station safety log, which can realize real-time analysis of the base station safety log, thereby shortening the time for discovering high-risk safety problems.
The embodiment of the invention provides a method for analyzing a base station security log, which comprises the following steps:
the data aggregation system caches the safety logs reported by the base station in real time to a first data cache system based on a first distributed cluster base station;
the log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and caches the feature value of the security log into the second data cache system;
the characteristic value real-time analysis system analyzes the characteristic value of the safety log in real time based on a stream type calculation frame to obtain an analysis result;
and the data presentation device displays the analysis result.
The embodiment of the invention provides a system for analyzing a base station security log, which comprises:
the data aggregation system is used for caching the safety logs reported by the base station in real time into the first data caching system based on the first distributed cluster base station;
the log characteristic computing system is used for computing the characteristic value of the security log based on the second distributed cluster and caching the characteristic value of the security log into the second data caching system;
the characteristic value real-time analysis system is used for carrying out real-time analysis on the characteristic value of the safety log based on a streaming computation framework to obtain an analysis result;
and the data presentation device is used for displaying the analysis result.
The embodiment of the invention comprises the following steps: the data aggregation system caches the safety logs reported by the base station in real time to a first data cache system based on the first distributed cluster; the log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and caches the feature value of the security log into the second data cache system; the characteristic value real-time analysis system analyzes the characteristic value of the safety log in real time based on a stream type calculation frame to obtain an analysis result; and the data presentation device displays the analysis result. The embodiment of the invention realizes the real-time analysis of the safety log reported by the base station in real time based on the distributed cluster and the streaming computing framework, thereby shortening the discovery time of high-risk safety problems, greatly reducing the time of manual analysis, shortening the problem feedback period, reducing the labor input and improving the management efficiency.
Additional features and advantages of embodiments of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of the invention. The objectives and other advantages of the embodiments of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the examples of the invention serve to explain the principles of the embodiments of the invention and not to limit the embodiments of the invention.
Fig. 1 is a flowchart of a method for analyzing a security log of a base station according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a system for analyzing a security log of a base station according to another embodiment of the present invention.
Detailed Description
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments of the present invention may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Referring to fig. 1, an embodiment of the present invention provides a method for analyzing a security log of a base station, including:
step 100, the data aggregation system caches the security log reported by the base station in real time to a first data caching system based on the first distributed cluster base station.
In an embodiment of the present invention, the security log includes one or more records, each record including: the log type, log level, base station Identification (ID), base station Internet Protocol (IP) address, service name, PID, and other basic information, and the specific content of the log is as follows: the source of the security log, the reason for the generation, etc.
For example, a certain record of a certain security log is:
45673491238900123<37>1 2017-12-13T05:42:16.156Z gnb1 g2z7p -2–src_ip:192.254.1.100,u:ad,content:login fail.
wherein 45673491238900123 is a recorded unique identifier (i.e. a security log identifier), <37> is PRI (), 1 is a version number, 2017-12-13T05:42:16.156Z is the generation time of the record, gnb1 is a host name, g2Z7p is a service name, PID number, no PID is occupied by "-", 2 is an event number, a structure definition is defined, no structure is occupied by "-", src _ ip:192.254.1.100, u: ad, content: logfail is the specific content of the log.
By the above explanation, each log record includes PRI, log version, time, base station identifier (i.e. host name), service name (i.e. service name), PID number, event number, structure, and log content.
The log type and the log level can be analyzed through the PRI value; different event numbers specify different formats of log content, and in this example, event number 2 is a user login failure, and the log content includes three attributes: the src _ IP is the IP of the login user; u is a login name used by the user; content is illustrative.
In the embodiment of the invention, the parameter modification or access login of the base station can generate the safety log, and the safety log is reported in real time after being generated.
In one illustrative example, the first data caching system is message middleware and is implemented based on an open source distributed message system Kafka.
In another embodiment of the present invention, the data aggregation system caching the security log in the first data caching system based on the first distributed cluster comprises:
the data aggregation system distributes a safety log identifier for the safety log based on a first distributed cluster, and caches a first corresponding relation between the safety log identifier and the safety log in the first data caching system;
the characteristic value of the security log comprises the security log identification;
the method further comprises the following steps: and the data storage system stores the first corresponding relation, the characteristic value of the safety log and the analysis result in a persistent mode.
In one illustrative example, the first distributed cluster includes: the system comprises an access gateway, a zookeeper and two or more than two Collection servers (Collection-Server); the data aggregation system allocates a security log identifier for the security log based on a first distributed cluster, and caches a first correspondence between the security log identifier and the security log in the first data caching system, including:
the zookeeper maintains cluster states of the two or more acquisition servers; wherein the cluster state comprises: logging in a cluster state or exiting the cluster state;
the access gateway acquires the cluster states of the two or two acquisition servers from the zookeeper, and distributes the security logs to the acquisition servers in the cluster logging state in the two or two acquisition servers;
and the acquisition server in the logging cluster state distributes a safety log identifier for the safety log, and caches the first corresponding relation between the safety log identifier and the safety log in the first data cache system.
Wherein, the access gateway exposes a uniform access entrance for the base station.
In one illustrative example, the method further comprises: zookeeper assigns an identifier to the acquisition server.
In an exemplary instance, the security log identifier is a globally unique identifier of each security log, and the security log identifier may be allocated by using a snowflash distributed global identifier allocation algorithm of Twitter open source.
In one illustrative example, the data storage system comprises: two or more data servers and search server clusters;
the data storage system persists the first corresponding relationship, the characteristic value of the security log, and the analysis result, including:
for each data server, the data server obtains the first corresponding relationship from the first data cache system, obtains the characteristic value from the second data cache system, and stores the first corresponding relationship and the characteristic value in the search server cluster;
and the data server acquires the analysis result from the characteristic value real-time analysis system and stores the analysis result into the search server cluster.
In one illustrative example, the cluster of search servers is an ES (elastic search) cluster.
In another illustrative example, the data server also maintains the first correspondence and a storage time of the feature values in the cluster of search servers.
In one illustrative example, the second data caching system is message middleware and is implemented based on an open source distributed message system Kafka.
In one illustrative example, the second distributed cluster includes: two or more feature computation servers; the log feature calculation system calculates a feature value of the security log based on the second distributed cluster, and caches the feature value of the security log in the second data cache system, including:
for each feature calculation server, the feature calculation server calculates a feature value of the security log and caches the feature value of the security log in the second data caching system.
In one illustrative example, the feature calculation server calculating the feature values of the security log comprises:
the feature computation server extracts one or more pre-defined feature vector values from the security log, combining the one or more feature vector values into the feature value.
For example, as shown in Table 1, a security log includes the following fields: id. The system comprises PRI, time, hostname, app-name and msgld, wherein ID is a unique identifier of a record, PRI is a primary key value, time is a recorded event, hostname is a host name, app-name is a service name, and magld is an event ID;
the feature vectors employed include: id. Facility, coverage, Time, NBId, ServerName, EventId; wherein ID is the unique identifier of the record, facility is a facility code, Severity is a Severity level, time is the generation time of the record, NBId is a base station identifier, SeverName is a service name, and eventId is an event ID.
Wherein, Facility and coverage are both calculated according to the field PRI, and other feature vectors are extracted from the security log, as shown in table 1, Facility ═ PRI > >3, coverage ═ PRI &0x 7.
| Feature vector | Fields in a security log |
| Id | Id |
| Facility | PRI>>3 |
| Severity | PRI&0x7 |
| Time | Time |
| NBId | Hostname |
| ServerName | App-name |
| EventId | Msgld |
TABLE 1
Extracting a characteristic vector Id, wherein the Id is a unique identifier distributed to log records by a data aggregation system, and the Id is extracted without an algorithm;
extracting feature vector Facility, wherein the Facility is obtained by bit operation (right shift by 3 bits) through the value of PRI in the log record;
extracting a feature vector, wherein the feature vector is obtained by bit operation (at 0x7) according to the value of PRI in the log record;
extracting a feature vector Time, wherein the Time is obtained through the Time in the log record;
extracting a feature vector NBId, wherein the NBId is obtained through the Hostname in the log record;
extracting a characteristic vector ServerName, wherein the ServerName is obtained through App-name in a log record;
extracting a characteristic vector EventId, wherein the EventId is obtained through Msgld in a log record, the MsgId also comprises some internal special marks, and the EventId can be stripped when being extracted;
other feature vectors such as client IP, login user name and the like are obtained from the log content, and the log content is in an internally defined normalized format and is suitable for each service module.
In one illustrative example, to associate a feature value with a security log, the security log may be identified as one of the feature vectors.
In the embodiment of the invention, different feature calculation servers can adopt the same feature vector and can also adopt different feature vectors so as to flexibly meet the service requirement.
And 102, the characteristic value real-time analysis system carries out real-time analysis on the characteristic value of the safety log based on a streaming computation framework to obtain an analysis result.
In the embodiment of the invention, the feature vector Facility extracts the type information of a program generating the log, the Severity level in the log record is extracted by the security, and the ServerName extracts a program module generating the security log by the base station, so that the score of the log record can be calculated by a scoring algorithm formed by predefining the Facility, the Severity and the ServerName, and whether the log is a serious security problem of a main program module is evaluated by the score;
the feature vector NBId extracts the base station identity and can therefore identify that base station that created a security problem.
In an exemplary example, the streaming computing framework may be a jstom streaming computing framework, topology is defined according to a service scene and deployed to a jstom cluster, the topology analyzes the characteristic value, sends the analysis result to the data presentation device for presentation, and stores the analysis result in the data storage system.
Wherein, Topology is a section of code running on the jstom cluster, that is, a data stream conversion diagram, which defines the flow of data acquisition, calculation and distribution.
In an illustrative example, a defined topology may include, but is not limited to, one or more of the following scenarios:
n base stations with the largest number of characteristic values meeting a first preset condition;
the number of the characteristic values meeting a second preset condition within a preset time;
and the safety log meets a third preset condition.
And 103, displaying the analysis result by the data presentation device.
In another embodiment of the present invention, the method further comprises: and the base station reports the safety log in real time.
In the embodiment of the present invention, the message of the security log reported by the base station may conform to any protocol definition, for example, RFC5424 protocol.
In another embodiment of the present invention, the method further comprises:
the data presentation device searches the first corresponding relationship corresponding to the analysis result and the characteristic value of the safety log in the data storage system, associates the searched first corresponding relationship with the characteristic value of the safety log to obtain a second corresponding relationship among the analysis result, the searched first corresponding relationship and the characteristic value of the safety log, and displays the second corresponding relationship;
or, the data presentation device searches the first corresponding relationship corresponding to a search instruction input by a user and the characteristic value of the security log in the data storage system, associates the searched first corresponding relationship with the characteristic value of the security log to obtain a second corresponding relationship among the searched analysis result, the first corresponding relationship and the characteristic value of the security log, and displays the second corresponding relationship.
Specifically, when searching, the search may be performed based on the security log identifier.
The embodiment of the invention realizes the real-time analysis of the safety log reported by the base station in real time based on the distributed cluster and the streaming computing framework, thereby shortening the discovery time of high-risk safety problems, greatly reducing the time of manual analysis, shortening the problem feedback period, reducing the labor input and improving the management efficiency.
The following examples are provided to illustrate specific implementations of the above method and are not intended to limit the scope of the embodiments of the present invention.
Example 1
In this example, the number of security logs of the service at the Warning level and above is counted, and the base station of the top 20 bits is displayed.
First defining the feature values includes the following feature vector values: id. Facility, preference, Time, NBId, ServerName, and the correspondence between these feature vectors and the fields in the security log is shown in table 2.
| Feature vector | Fields in a security log |
| Id | Id |
| Facility | PRI>>3 |
| Severity | PRI&0x7 |
| Time | Time |
| NBId | Hostname |
| ServerName | App-name |
TABLE 2
To count the number of security logs at the Warning level and above, it is necessary to obtain 8 values as shown in table 3 according to the feature vector change and according to the definition of RFC5424 protocol that the security logs conform to.
TABLE 3
As shown in Table 3, the value of Warning is 4, and the values of Warning and above are: 0,1,2,3,4 have 5 values. Due to the ranking of the base stations, the number of security logs needs to be counted according to the unique ID of the base station represented by the feature vector NBId, that is, for the base stations with the same NBId, the number of security logs with the feature vector sensitivity of 0,1,2,3, or 4 is counted every 30 minutes, and the base station with the top 20 digits in each calculation is sent to the data presentation device for display.
Example 2
In this example, the number of base station logins within a 5 minute window is counted.
First defining the feature values includes the following feature vector values: id. The correspondence between the feature vectors and the fields in the security log is shown in table 1.
To count the number of base station logins in a 5-minute window, a field of base station login needs to be determined first, for example, msgId of login time in a security log is defined as 1, and a value of a feature vector EventId is required to be 1. That is, for base stations with the same NBId, the number of security logs with the value of 1 for the feature vector EventId is counted, and the data presentation device displays the number of logins per base station in 5 minutes (i.e., the number of security logs with the value of 1 for the feature vector EventId).
Example 3
In this example, a security log with service as Critical is obtained.
First defining the feature values includes the following feature vector values: id. Facility, preference, Time, NBId, ServerName, and the correspondence between these feature vectors and the fields in the security log is shown in table 2.
To obtain a Critical-level security log, it is necessary that the security log can take 8 values as shown in table 3 according to the definition of RFC5424 protocol followed by the security log according to the feature vector security.
As shown in Table 3, the Critical value is 2. Because of the statistics of the base stations, the number of the security logs needs to be counted according to the unique ID of the base station represented by the feature vector NBId, that is, for the base stations with the same NBId, the feature vector sensitivity is obtained as 2 feature values, the corresponding security logs are obtained according to the security log identifiers in the feature values, and the obtained security logs are sent to the data presentation device for display.
Referring to fig. 2, another embodiment of the present invention provides a system for analyzing a security log of a base station, including:
the data aggregation system 201 is configured to cache, based on the first distributed cluster 2011 base station, a security log reported in real time by the base station in the first data caching system 202;
the log feature calculation system 203 is configured to calculate a feature value of the security log based on the second distributed cluster 2031, and cache the feature value of the security log in the second data cache system 204;
the characteristic value real-time analysis system 205 is configured to analyze the characteristic value of the security log in real time based on the streaming computation framework 2051 to obtain an analysis result;
and a data presentation device 206 for displaying the analysis result.
In another embodiment of the present invention, the method further comprises: and the base station 207 is used for reporting the security log in real time.
In this embodiment of the present invention, the data aggregation system 201 is specifically configured to:
allocating a security log identifier to the security log based on a first distributed cluster 2011, and caching a first corresponding relationship between the security log identifier and the security log in the first data caching system;
the characteristic value of the security log comprises the security log identification;
the system further comprises:
and the data storage system 208 is configured to perform persistent storage on the first corresponding relationship, the feature value of the security log, and the analysis result.
In this embodiment of the present invention, the data presentation device 206 is further configured to:
searching the first corresponding relation corresponding to the analysis result and the characteristic value of the safety log in the data storage system, associating the searched first corresponding relation with the characteristic value of the safety log to obtain a second corresponding relation among the analysis result, the searched first corresponding relation and the characteristic value of the safety log, and displaying the second corresponding relation;
or searching the first corresponding relation corresponding to a search instruction input by a user and the characteristic value of the safety log in the data storage system, associating the searched first corresponding relation with the characteristic value of the safety log to obtain a second corresponding relation among the searched analysis result, the first corresponding relation and the characteristic value of the safety log, and displaying the second corresponding relation.
In this embodiment of the present invention, the first distributed cluster 2011 includes: an access gateway 2012, a zookeeper2013 and two or more acquisition servers 2014;
the zookeeper2013 is used for maintaining the cluster state of the two or more acquisition servers; wherein the cluster state comprises: logging in a cluster state or exiting the cluster state;
the access gateway 2012 is configured to obtain cluster states of the two or more acquisition servers from the zookeeper, and allocate the security log to the acquisition server in the cluster logging state in the two or more acquisition servers;
the collection server 2014 in the log-in cluster state is configured to allocate a security log identifier to the security log, and cache the first correspondence between the security log identifier and the security log in the first data caching system.
In an embodiment of the present invention, the data storage system 208 includes: two or more data servers 2081 and search server clusters 2082;
the data server 2081, configured to obtain the first corresponding relationship from the first data cache system, obtain the feature value from the second data cache system, determine the second corresponding relationship according to the first corresponding relationship and the feature value, and store the second corresponding relationship in the search server cluster 2082;
the analysis result is obtained from the feature value real-time analysis system 201, and the analysis result is stored in the search server cluster 2082.
In this embodiment of the present invention, the second distributed cluster 2021 includes: two or more feature computation servers 2022;
the feature calculation server 2022 is configured to calculate a feature value of the security log, and cache the feature value of the security log in the second data cache system.
In this embodiment of the present invention, the feature calculation server 2022 is specifically configured to calculate the feature value of the security log by using the following method:
one or more predefined feature vector values are extracted from the security log, the one or more feature vector values being combined into the feature value.
In an exemplary example, the streaming computing framework may be a jstom streaming computing framework, topology is defined according to a service scene and deployed to a jstom cluster, the topology analyzes the characteristic value, sends the analysis result to the data presentation device for presentation, and stores the analysis result in the data storage system.
In an illustrative example, a defined topology may include, but is not limited to, one or more of the following scenarios:
n base stations with the largest number of characteristic values meeting a first preset condition;
the number of the characteristic values meeting a second preset condition within a preset time;
and the safety log meets a third preset condition.
The specific implementation process of the system for analyzing the base station security log is the same as that of the method for analyzing the base station security log in the foregoing embodiment, and is not described herein again.
The embodiment of the invention realizes the real-time analysis of the safety log reported by the base station in real time based on the distributed cluster and the streaming computing framework, thereby shortening the discovery time of high-risk safety problems, greatly reducing the time of manual analysis, shortening the problem feedback period, reducing the labor input and improving the management efficiency.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Although the embodiments of the present invention have been described above, the descriptions are only used for understanding the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the embodiments of the invention as defined by the appended claims.
Claims (16)
1. A method of analyzing a base station security log, comprising:
the data aggregation system caches the safety logs reported by the base station in real time to a first data cache system based on a first distributed cluster base station;
the log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and caches the feature value of the security log into the second data cache system;
the characteristic value real-time analysis system analyzes the characteristic value of the safety log in real time based on a stream type calculation frame to obtain an analysis result;
and the data presentation device displays the analysis result.
2. The method of claim 1, further comprising:
and the base station reports the safety log in real time.
3. The method of claim 1, wherein the data aggregation system caching the security log into the first data caching system based on the first distributed cluster comprises:
the data aggregation system distributes a safety log identifier for the safety log based on a first distributed cluster, and caches a first corresponding relation between the safety log identifier and the safety log in the first data caching system;
the characteristic value of the security log comprises the security log identification;
the method further comprises the following steps:
and the data storage system stores the first corresponding relation, the characteristic value of the safety log and the analysis result in a persistent mode.
4. The method of claim 3, further comprising:
the data presentation device searches the first corresponding relationship corresponding to the analysis result and the characteristic value of the safety log in the data storage system, associates the searched first corresponding relationship with the characteristic value of the safety log to obtain a second corresponding relationship among the analysis result, the searched first corresponding relationship and the characteristic value of the safety log, and displays the second corresponding relationship;
or, the data presentation device searches the first corresponding relationship corresponding to a search instruction input by a user and the characteristic value of the security log in the data storage system, associates the searched first corresponding relationship with the characteristic value of the security log to obtain a second corresponding relationship among the searched analysis result, the first corresponding relationship and the characteristic value of the security log, and displays the second corresponding relationship.
5. The method of claim 3, wherein the first distributed cluster comprises: the system comprises an access gateway, a zookeeper and two or more acquisition servers; the data aggregation system allocates a security log identifier for the security log based on a first distributed cluster, and caches a first correspondence between the security log identifier and the security log in the first data caching system, including:
the zookeeper maintains cluster states of the two or more acquisition servers; wherein the cluster state comprises: logging in a cluster state or exiting the cluster state;
the access gateway acquires the cluster states of the two or two acquisition servers from the zookeeper, and distributes the security logs to the acquisition servers in the cluster logging state in the two or two acquisition servers;
and the acquisition server in the logging cluster state distributes a safety log identifier for the safety log, and caches the first corresponding relation between the safety log identifier and the safety log in the first data cache system.
6. The method of claim 3, wherein the data storage system comprises: two or more data servers and search server clusters;
the data storage system persists the security log, the characteristic values of the security log, and the analysis result, and includes:
for each data server, the data server acquires the first corresponding relation from the first data cache system, acquires the characteristic value from the second data cache system, determines the second corresponding relation according to the first corresponding relation and the characteristic value, and stores the second corresponding relation into the search server cluster;
and the data server acquires the analysis result from the characteristic value real-time analysis system and stores the analysis result into the search server cluster.
7. The method of claim 1, wherein the second distributed cluster comprises: two or more feature computation servers; the log feature calculation system calculates a feature value of the security log based on the second distributed cluster, and caches the feature value of the security log in the second data cache system, including:
for each feature calculation server, the feature calculation server calculates a feature value of the security log and caches the feature value of the security log in the second data caching system.
8. The method of claim 7, wherein the computing the feature value of the security log by the feature computation server comprises:
the feature computation server extracts one or more pre-defined feature vector values from the security log, combining the one or more feature vector values into the feature value.
9. A system for analyzing a base station security log, comprising:
the data aggregation system is used for caching the safety logs reported by the base station in real time into the first data caching system based on the first distributed cluster base station;
the log characteristic computing system is used for computing the characteristic value of the security log based on the second distributed cluster and caching the characteristic value of the security log into the second data caching system;
the characteristic value real-time analysis system is used for carrying out real-time analysis on the characteristic value of the safety log based on a streaming computation framework to obtain an analysis result;
and the data presentation device is used for displaying the analysis result.
10. The system of claim 9, further comprising:
and the base station is used for reporting the security log in real time.
11. The system of claim 9, wherein the data aggregation system is specifically configured to:
the data aggregation system distributes a safety log identifier for the safety log based on a first distributed cluster, and caches a first corresponding relation between the safety log identifier and the safety log in the first data caching system;
the characteristic value of the security log comprises the security log identification;
the system further comprises:
and the data storage system is used for carrying out persistent storage on the first corresponding relation, the characteristic value of the safety log and the analysis result.
12. The system of claim 11, wherein the data presentation device is further configured to:
searching the first corresponding relation corresponding to the analysis result and the characteristic value of the safety log in the data storage system, associating the searched first corresponding relation with the characteristic value of the safety log to obtain a second corresponding relation among the analysis result, the searched first corresponding relation and the characteristic value of the safety log, and displaying the second corresponding relation;
or searching the first corresponding relation corresponding to a search instruction input by a user and the characteristic value of the safety log in the data storage system, associating the searched first corresponding relation with the characteristic value of the safety log to obtain a second corresponding relation among the searched analysis result, the first corresponding relation and the characteristic value of the safety log, and displaying the second corresponding relation.
13. The system of claim 11, wherein the first distributed cluster comprises: the system comprises an access gateway, a zookeeper and two or more acquisition servers;
the zookeeper is used for maintaining the cluster state of the two or more acquisition servers; wherein the cluster state comprises: logging in a cluster state or exiting the cluster state;
the access gateway is used for acquiring the cluster states of the two or two acquisition servers from the zookeeper, and distributing the security logs to the acquisition servers in the cluster logging state in the two or two acquisition servers;
the collection server in the log-in cluster state is used for distributing a safety log identifier for the safety log and caching the first corresponding relation between the safety log identifier and the safety log into the first data caching system.
14. The system of claim 11, wherein the data storage system comprises: two or more data servers and search server clusters;
the data server is configured to obtain the first corresponding relationship from the first data cache system, obtain the feature value from the second data cache system, determine the second corresponding relationship according to the first corresponding relationship and the feature value, and store the second corresponding relationship in the search server cluster;
and acquiring the analysis result from the characteristic value real-time analysis system, and storing the analysis result into the search server cluster.
15. The system of claim 9, wherein the second distributed cluster comprises: two or more feature computation servers;
the characteristic calculation server is used for calculating the characteristic value of the security log and caching the characteristic value of the security log into the second data caching system.
16. The system of claim 15, wherein the feature computation server is specifically configured to implement computing the feature value of the security log by:
one or more predefined feature vector values are extracted from the security log, the one or more feature vector values being combined into the feature value.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910556924.2A CN112134719A (en) | 2019-06-25 | 2019-06-25 | Method and system for analyzing base station security log |
| PCT/CN2020/083742 WO2020258982A1 (en) | 2019-06-25 | 2020-04-08 | Method and system for analyzing security log of base station, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910556924.2A CN112134719A (en) | 2019-06-25 | 2019-06-25 | Method and system for analyzing base station security log |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112134719A true CN112134719A (en) | 2020-12-25 |
Family
ID=73850153
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910556924.2A Pending CN112134719A (en) | 2019-06-25 | 2019-06-25 | Method and system for analyzing base station security log |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112134719A (en) |
| WO (1) | WO2020258982A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463772A (en) * | 2021-02-02 | 2021-03-09 | 北京信安世纪科技股份有限公司 | Log processing method and device, log server and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113792340B (en) * | 2021-09-09 | 2023-09-05 | 烽火通信科技股份有限公司 | Method and device for auditing logical logs of database |
| CN114860774A (en) * | 2022-05-19 | 2022-08-05 | 宁波奥克斯电气股份有限公司 | Big data real-time analysis method and system of air conditioner, storage medium and air conditioner |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838867A (en) * | 2014-03-20 | 2014-06-04 | 网宿科技股份有限公司 | Log processing method and device |
| US20160094620A1 (en) * | 2014-09-26 | 2016-03-31 | Lenovo Enterprise Solutions (Singapore) Pte, Ltd. | Scalable logging control for distributed network devices |
| CN107622084A (en) * | 2017-08-10 | 2018-01-23 | 深圳前海微众银行股份有限公司 | Blog management method, system and computer-readable recording medium |
| CN107786565A (en) * | 2017-11-02 | 2018-03-09 | 江苏物联网研究发展中心 | A kind of distributed real-time intrusion detection method and detecting system |
| CN108647139A (en) * | 2018-03-19 | 2018-10-12 | 北京趣拿软件科技有限公司 | Test method, device, storage medium and the electronic device of system |
| CN108985981A (en) * | 2018-06-28 | 2018-12-11 | 北京奇虎科技有限公司 | Data processing system and method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10210115B2 (en) * | 2015-06-02 | 2019-02-19 | Box, Inc. | System for handling event messages for file collaboration |
| US10560544B2 (en) * | 2015-08-25 | 2020-02-11 | Box, Inc. | Data caching in a collaborative file sharing system |
| CN105224445B (en) * | 2015-10-28 | 2017-02-15 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
-
2019
- 2019-06-25 CN CN201910556924.2A patent/CN112134719A/en active Pending
-
2020
- 2020-04-08 WO PCT/CN2020/083742 patent/WO2020258982A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838867A (en) * | 2014-03-20 | 2014-06-04 | 网宿科技股份有限公司 | Log processing method and device |
| US20160094620A1 (en) * | 2014-09-26 | 2016-03-31 | Lenovo Enterprise Solutions (Singapore) Pte, Ltd. | Scalable logging control for distributed network devices |
| CN107622084A (en) * | 2017-08-10 | 2018-01-23 | 深圳前海微众银行股份有限公司 | Blog management method, system and computer-readable recording medium |
| CN107786565A (en) * | 2017-11-02 | 2018-03-09 | 江苏物联网研究发展中心 | A kind of distributed real-time intrusion detection method and detecting system |
| CN108647139A (en) * | 2018-03-19 | 2018-10-12 | 北京趣拿软件科技有限公司 | Test method, device, storage medium and the electronic device of system |
| CN108985981A (en) * | 2018-06-28 | 2018-12-11 | 北京奇虎科技有限公司 | Data processing system and method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463772A (en) * | 2021-02-02 | 2021-03-09 | 北京信安世纪科技股份有限公司 | Log processing method and device, log server and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020258982A1 (en) | 2020-12-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11641319B2 (en) | Network health data aggregation service | |
| US20210119890A1 (en) | Visualization of network health information | |
| US10560465B2 (en) | Real time anomaly detection for data streams | |
| US10243820B2 (en) | Filtering network health information based on customer impact | |
| AU2019201687B2 (en) | Network device vulnerability prediction | |
| US10911263B2 (en) | Programmatic interfaces for network health information | |
| US10592666B2 (en) | Detecting anomalous entities | |
| US9379949B2 (en) | System and method for improved end-user experience by proactive management of an enterprise network | |
| US20150039749A1 (en) | Detecting traffic anomalies based on application-aware rolling baseline aggregates | |
| US20160359701A1 (en) | Parallel coordinate charts for flow exploration | |
| US10805163B2 (en) | Identifying device types based on behavior attributes | |
| CN109684052B (en) | Transaction analysis method, device, equipment and storage medium | |
| CN112134719A (en) | Method and system for analyzing base station security log | |
| US8533279B2 (en) | Method and system for reconstructing transactions in a communication network | |
| CN107092686B (en) | File management method and device based on cloud storage platform | |
| CN104486116A (en) | Multidimensional query method and multidimensional query system of flow data | |
| CN111258798A (en) | Fault positioning method and device for monitoring data, computer equipment and storage medium | |
| CN113505048A (en) | Unified monitoring platform based on application system portrait and implementation method | |
| US20170235785A1 (en) | Systems and Methods for Robust, Incremental Data Ingest of Communications Networks Topology | |
| CN111275495A (en) | Advertisement putting monitoring method, device and system based on block chain | |
| CN110677327A (en) | Chip-based real-time detection method for RTP flow fault | |
| CN113297253A (en) | Equipment identification method, device, equipment and readable storage medium | |
| CN108337100B (en) | Cloud platform monitoring method and device | |
| CN111338888A (en) | Data statistical method and device, electronic equipment and storage medium | |
| CN103414593A (en) | Trans-disciplinary engineering network element cascading shielding system and shielding method based on network resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20201225 |
|
| WD01 | Invention patent application deemed withdrawn after publication |