WO2020258982A1 - 一种分析基站安全日志的方法、系统及计算机可读存储介质 - Google Patents
一种分析基站安全日志的方法、系统及计算机可读存储介质 Download PDFInfo
- Publication number
- WO2020258982A1 WO2020258982A1 PCT/CN2020/083742 CN2020083742W WO2020258982A1 WO 2020258982 A1 WO2020258982 A1 WO 2020258982A1 CN 2020083742 W CN2020083742 W CN 2020083742W WO 2020258982 A1 WO2020258982 A1 WO 2020258982A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- security log
- data
- correspondence
- security
- log
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000004458 analytical method Methods 0.000 claims abstract description 53
- 238000010223 real-time analysis Methods 0.000 claims abstract description 18
- 239000013598 vector Substances 0.000 claims description 40
- 238000004364 calculation method Methods 0.000 claims description 25
- 238000013500 data storage Methods 0.000 claims description 20
- 230000002776 aggregation Effects 0.000 claims description 16
- 238000004220 aggregation Methods 0.000 claims description 16
- 230000000875 corresponding effect Effects 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002596 correlated effect Effects 0.000 claims description 2
- 230000003139 buffering effect Effects 0.000 abstract 4
- 238000000605 extraction Methods 0.000 description 5
- 238000004904 shortening Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 101150014601 GNB1 gene Proteins 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 241000533950 Leucojum Species 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001568 sexual effect Effects 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Definitions
- the embodiments of the present invention relate to, but are not limited to, the fields of computers and communications, and particularly refer to a method, system, and computer-readable storage medium for analyzing base station security logs.
- 5G wireless network technology With the development of 5G wireless network technology, the number of base stations in the wireless access network is increasing, and the requirements for the command operation and data operation of the base station and the collection and analysis of access security are becoming more and more important.
- the traditional base station security log is collected from the base station through a timed task cycle. After the collection, all the security logs are analyzed and stored. After the failure occurs, the analysis is performed manually. This process has a large delay in obtaining data, and manual analysis It may be that the ignoring of certain critical security logs causes great delays in the location and handling of the problem.
- 5G has greatly expanded the scale of the network. In the face of thousands of security logs on the live network every day, a lot of manpower is required.
- the embodiment of the present invention provides a method, system and computer-readable storage medium for analyzing the security log of a base station, which can realize real-time analysis of the security log of the base station, thereby shortening the discovery time of high-risk security problems.
- the embodiment of the present invention provides a method for analyzing the security log of a base station, including:
- the data aggregation system caches the security log reported by the base station in real time into the first data cache system
- the log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and caches the feature value of the security log in the second data cache system;
- the real-time eigenvalue analysis system performs real-time analysis on the eigenvalues of the security log based on the stream computing framework to obtain the analysis results;
- the data presentation device displays the analysis results.
- the embodiment of the present invention provides a system for analyzing base station security logs, including:
- the data aggregation system is used to cache the security log reported by the base station in real time in the first data cache system based on the first distributed cluster base station;
- the log feature calculation system is used to calculate the feature value of the security log based on the second distributed cluster, and cache the feature value of the security log in the second data caching system;
- Eigenvalue real-time analysis system for real-time analysis of the eigenvalues of the security log based on the streaming computing framework to obtain the analysis results
- Data presentation device for displaying analysis results.
- An embodiment of the present invention further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-mentioned method when running.
- Figure 1 is a flowchart of a method for analyzing base station security logs proposed by an embodiment of the present invention
- Fig. 2 is a schematic structural composition diagram of a system for analyzing base station security logs proposed by another embodiment of the present invention.
- an embodiment of the present invention provides a method for analyzing base station security logs, including:
- Step 100 Based on the first distributed cluster base station, the data aggregation system caches the security log reported by the base station in real time into the first data cache system.
- the security log includes one or more records, and each record includes: log type, log level, base station identification (ID), base station Internet Protocol (IP, Internet Protocol) address, service name, PID, etc.
- Information including the specific content of the log, such as the source and reason of the security log.
- a record of a security log is:
- 45673491238900123 is the unique identifier of the record (ie the security log identifier), ⁇ 37> is PRI(), 1 is the version number, 2017-12-13T05:42:16.156Z is the generation time of the record, gnb1 is the host name, g2z7p Is the service name,-is the PID number, if there is no PID, use "-" to occupy the position, 2 is the event number,-is the structure definition, when there is no structure, use the "—” to occupy the position, src_ip:192.254.1.100,u:ad ,content:login Fail is the specific content of the log.
- each log record includes PRI, log version, time, base station identification (ie host name), service name (ie service name), PID number, event number, structure, and log content.
- the log type and log level can be parsed through the PRI value; different event numbers specify different formats of the log content.
- the 2nd event is the user login failure
- the log content contains three attributes: src_ip is the login user’s IP; u is the login name used by the user; content is the description.
- the parameter modification or access login of the base station will generate a security log, and the security log is generated and then reported in real time.
- the first data cache system is a message middleware, which is implemented based on the open source distributed message system Kafka.
- the data aggregation system caching the security log in the first data caching system based on the first distributed cluster includes:
- the data aggregation system allocates a security log identifier to the security log based on the first distributed cluster, and caches the first correspondence between the security log identifier and the security log in the first data caching system;
- the characteristic value of the security log includes the security log identifier
- the method further includes: the data storage system persistently saves the first correspondence, the characteristic value of the security log, and the analysis result.
- the first distributed cluster includes: an access gateway, zookeeper, and two or more collection servers (Collection-Server); the data aggregation system is based on the first distributed cluster as the Allocating a security log identifier to the security log, and caching the first correspondence between the security log identifier and the security log in the first data caching system includes:
- the zookeeper maintains the cluster status of the two or more collection servers; wherein, the cluster status includes: logging in to the cluster status or exiting the cluster status;
- the access gateway obtains the cluster status of the two or two collection servers from the zookeeper, distributes the security log to the two or two collection servers, and the collection that is in the login cluster state server;
- the collection server in the log-in cluster state allocates a security log identifier to the security log, and caches the first correspondence between the security log identifier and the security log in the first data caching system.
- the access gateway exposes a unified access portal for the base station.
- the method further includes: zookeeper assigns an identifier to the collection server.
- the security log identifier is a globally unique identifier of each security log, and the security log identifier may be distributed using Twitter's open-source distributed global identifier distribution algorithm snowflake.
- the data storage system includes: two or more data servers and a search server cluster;
- the data storage system persistently storing the first correspondence, the characteristic value of the security log, and the analysis result includes:
- the data server obtains the first correspondence from the first data cache system, obtains the characteristic value from the second data cache system, and associates the first corresponding The relationship and the characteristic value are stored in the search server cluster;
- the data server obtains the analysis result from the characteristic value real-time analysis system, and saves the analysis result in the search server cluster.
- the search server cluster is an ES (ElasticSearch) cluster.
- the data server also maintains the first correspondence and the storage time of the feature value in the search server cluster.
- Step 101 The log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and caches the feature value of the security log in the second data caching system.
- the second data cache system is a message middleware, which is implemented based on the open source distributed message system Kafka.
- the second distributed cluster includes: two or more feature calculation servers; the log feature calculation system calculates the feature value of the security log based on the second distributed cluster, and calculates the feature value of the security log
- the value cache in the second data cache system includes:
- the characteristic calculation server calculates the characteristic value of the security log, and caches the characteristic value of the security log in the second data caching system.
- calculating the characteristic value of the security log by the characteristic calculation server includes:
- the feature calculation server extracts one or more predefined feature vector values from the security log, and composes the one or more feature vector values into the feature value.
- a security log includes the following fields: id, PRI, time, hostname, app-name, and msgld, where id is the unique identifier of the record, PRI is the primary key value, and time is the generation of the record
- id is the unique identifier of the record
- PRI is the primary key value
- time is the generation of the record
- hostname is the host name
- app-name is the service name
- magld is the event ID
- the feature vectors used include: id, Facility, Severity, Time, NBId, ServerName, EventId; among them, id is the unique identifier of the record, facility is the facility code, Severity is the severity level, time is the generation time of the record, and NBId is the base station identifier , SeverName is the service name, and EventId is the event ID.
- Id is the unique identifier assigned by the data aggregation system for log records, and there is no algorithm for Id extraction;
- Feature vector Facility extraction, Facility is obtained by bit operation (shifted 3 bits to the right) through the value of PRI in the log record;
- Severity is obtained by bit operation (located at 0x7) through the value of PRI in the log record;
- the feature vector NBId is extracted. NBId is obtained through the Hostname in the log record;
- EventId is extracted. EventId is obtained through Msgld in the log record. MsgId also contains some internal special identifiers, which will be stripped off when extracting EventId;
- log content is a normalized format defined internally and is suitable for each service module.
- the security log identifier may be used as one of the feature vectors.
- different feature calculation servers may use the same feature vector or different feature vectors to flexibly meet business needs.
- Step 102 The real-time feature value analysis system performs real-time analysis on the feature value of the security log based on the streaming computing framework to obtain an analysis result.
- the feature vector Facility extracts the type information of the program that generates the log, Severity extracts the severity level in the log record, and the ServerName extracts the program module of the base station generating the security log. Therefore, it can be pre-defined Facility, Severity
- the scoring algorithm composed of ServerName calculates the score of the log record, and uses the score to evaluate whether the log is a serious security problem for the main program module;
- the feature vector NBId extracts the base station identity, so it can identify which base station has a security problem.
- the streaming computing framework may be the JStorm streaming computing framework.
- the topology is defined according to the business scenario and deployed to the JStorm cluster. The topology analyzes the characteristic values, and sends the analysis results to the data presentation device for display, and the analysis The results are saved to the data storage system.
- Topology is a piece of code running on the JStorm cluster, that is, a data flow conversion graph, which defines the process of data acquisition, calculation, and distribution.
- the defined topology may include but is not limited to one or more of the following scenarios:
- Step 103 The data presentation device displays the analysis result.
- the method further includes: the base station reports the security log in real time.
- the message of the security log reported by the base station may conform to any protocol definition, such as the RFC5424 protocol.
- the method further includes:
- the data presentation device searches the data storage system for the first correspondence corresponding to the analysis result and the characteristic value of the security log, and compares the found first correspondence with the security log Associating the characteristic values to obtain the analysis result, the second correspondence between the found first correspondence and the characteristic value of the security log, and display the second correspondence;
- the data presentation device searches the data storage system for the first correspondence corresponding to the search instruction input by the user and the characteristic value of the security log, and compares the found first correspondence with the The characteristic values of the security log are correlated, and the found analysis result, the first correspondence relationship and the characteristic value of the security log are obtained by the second correspondence relationship, and the second correspondence relationship is displayed.
- the embodiment of the present invention implements real-time analysis of the security log reported by the base station in real time based on the distributed cluster and streaming computing framework, thereby shortening the discovery time of high-risk security problems, greatly reducing the time for manual analysis, and shortening the problem feedback cycle , Reducing manpower input and improving management efficiency.
- the number of security logs of Serverity at Warning and above is counted, and the top 20 base stations are displayed.
- the feature value including the following feature vector values: id, Facility, Severity, Time, NBId, ServerName.
- the correspondence between these feature vectors and the fields in the security log is shown in Table 2.
- Severity can take 8 values as shown in Table 3.
- the value of Warning is 4, and the value of Waring and above levels: 0, 1, 2, 3, and 4 in total. Since it is to sort the base stations, it is necessary to count the number of security logs based on the feature vector NBId, which represents the unique ID of the base station, that is, for the base station with the same NBId, the statistical feature vector Severity is 0,1,2,3,4 The number of security logs with any one value can be calculated every 30 minutes, and the top 20 base stations in each calculation are sent to the data presentation device for display.
- the feature value including the following feature vector values: id, Facility, Severity, Time, NBId, ServerName, EventId.
- the correspondence between these feature vectors and the fields in the security log is shown in Table 1.
- the msgId of the login time in the security log is defined as 1, and the value of the feature vector EventId is required. That is to say, for base stations with the same NBId, the number of security logs with the feature vector EventId value of 1 is counted, and the data presentation device displays the number of logins by each base station within 5 minutes (that is, the number of security logs with the feature vector EventId value of 1 Quantity).
- the feature value including the following feature vector values: id, Facility, Severity, Time, NBId, ServerName.
- the correspondence between these feature vectors and the fields in the security log is shown in Table 2.
- Severity can take 8 values as shown in Table 3.
- the value of Critical is 2. Since the statistics of base stations need to be based on the feature vector NBId, NBId represents the unique ID of the base station to count the number of security logs, that is, for base stations with the same NBId, the feature vector Severity is obtained as a feature value of 2, and the security in the feature value The log identifier obtains the corresponding security log, and the obtained security log is sent to the data presentation device for display.
- FIG. 2 another embodiment of the present invention provides a system for analyzing base station security logs, including:
- the data aggregation system 201 is configured to cache the security log reported by the base station in real time in the first data cache system 202 based on the first distributed cluster 2011 base station;
- the log feature calculation system 203 is configured to calculate the feature value of the security log based on the second distributed cluster 2031, and cache the feature value of the security log in the second data caching system 204;
- the characteristic value real-time analysis system 205 is used for real-time analysis of the characteristic value of the security log based on the streaming computing framework 2051 to obtain the analysis result;
- the data presentation device 206 is used to display the analysis result.
- it further includes: a base station 207, configured to report security logs in real time.
- the data aggregation system 201 is specifically used for:
- the characteristic value of the security log includes the security log identifier
- the system also includes:
- the data storage system 208 is configured to persistently store the first correspondence, the characteristic value of the security log, and the analysis result.
- the data presentation device 206 is further configured to:
- the association is performed to obtain the found second correspondence between the analysis result, the first correspondence and the characteristic value of the security log, and display the second correspondence.
- the first distributed cluster 2011 includes: an access gateway 2012, zookeeper 2013, and two or more collection servers 2014;
- the zookeeper2013 is used to maintain the cluster status of the two or more collection servers; wherein, the cluster status includes: logging in to the cluster status or exiting the cluster status;
- the access gateway 2012 is configured to obtain the cluster status of the two or two collection servers from the zookeeper, allocate the security log to the two or two collection servers, and be in the state of logging in to the cluster Said collection server;
- the collection server 2014 in the state of logging in to the cluster is configured to allocate a security log identifier to the security log, and cache the first correspondence between the security log identifier and the security log to the first data caching system in.
- the data storage system 208 includes: two or more data servers 2081 and a search server cluster 2082;
- the data server 2081 is configured to obtain the first correspondence from the first data caching system, obtain the characteristic value from the second data caching system, and obtain the characteristic value according to the first correspondence and the The feature value determines the second correspondence, and saves the second correspondence in the search server cluster 2082;
- the analysis result is obtained from the feature value real-time analysis system 201, and the analysis result is saved in the search server cluster 2082.
- the second distributed cluster 2021 includes: two or more feature computing servers 2022;
- the characteristic calculation server 2022 is configured to calculate the characteristic value of the security log, and cache the characteristic value of the security log in the second data caching system.
- the characteristic calculation server 2022 is specifically configured to calculate the characteristic value of the security log in the following manner:
- the streaming computing framework may be the JStorm streaming computing framework.
- the topology is defined according to the business scenario and deployed to the JStorm cluster. The topology analyzes the characteristic values, and sends the analysis results to the data presentation device for display, and the analysis The results are saved to the data storage system.
- the defined topology may include but is not limited to one or more of the following scenarios:
- a computer-readable storage medium wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the aforementioned analysis of the base station security log when running.
- the embodiment of the present invention implements real-time analysis of the security log reported by the base station in real time based on the distributed cluster and streaming computing framework, thereby shortening the discovery time of high-risk security problems, greatly reducing the time for manual analysis, and shortening the problem feedback cycle , Reducing manpower input and improving management efficiency.
- Such software may be distributed on a computer-readable medium, and the computer-readable medium may include a computer storage medium (or a non-transitory medium) and a communication medium (or a transitory medium).
- the term computer storage medium includes volatile and non-volatile memory implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Sexual, removable and non-removable media.
- Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage device, or Any other medium used to store desired information and that can be accessed by a computer.
- communication media usually contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery media .
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Debugging And Monitoring (AREA)
Abstract
本发明实施例公开了一种分析基站安全日志的方法、系统及计算机可读存储介质,所述方法包括:数据汇聚系统基于第一分布式集群基站将基站实时上报的安全日志缓存到第一数据缓存系统中;日志特征计算系统基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中;特征值实时分析系统基于流式计算框架对安全日志的特征值进行实时分析得到分析结果;数据呈现装置显示分析结果。
Description
相关申请的交叉引用
本申请基于申请号为201910556924.2、申请日为2019年06月25日的中国专利申请提出,并要求该中国专利申请的优先权,该中国专利申请的全部内容在此引入本申请作为参考。
本发明实施例涉及但不限于计算机和通信领域,尤指一种分析基站安全日志的方法、系统及计算机可读存储介质。
随着5G无线网络技术的发展,无线接入网络中基站数量越来越多,对基站的指令操作和数据操作以及访问安全的采集和分析需求越来越重要。传统的基站安全日志是通过定时任务周期从基站上采集得到的,采集后对所有的安全日志进行解析入库,故障发生后,人工进行分析,这一流程存在获取数据的延迟较大,人工分析可能会由于忽略了某些关键安全日志而导致问题的定位和处理出现极大的拖延。5G,随着无线网络和物联网的发展极大扩展了网络规模,面对现网每天成千上万条安全日志,需要投入大量人力。
发明内容
本发明实施例提供了一种分析基站安全日志的方法、系统及计算机可读存储介质,能够实现对基站的安全日志的实时分析,从而缩短高危安全问题的发现时间。
本发明实施例提供了一种分析基站安全日志的方法,包括:
数据汇聚系统基于第一分布式集群基站将基站实时上报的安全日志缓存到第一数据缓存系统中;
日志特征计算系统基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中;
特征值实时分析系统基于流式计算框架对安全日志的特征值进行实时分析得到分析结果;
数据呈现装置显示分析结果。
本发明实施例提供了一种分析基站安全日志的系统,包括:
数据汇聚系统,用于基于第一分布式集群基站将基站实时上报的安全日志缓存到第一数据缓存系统中;
日志特征计算系统,用于基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中;
特征值实时分析系统,用于基于流式计算框架对安全日志的特征值进行实时分析得到分析结果;
数据呈现装置,用于显示分析结果。
本发明实施例还提供一种计算机可读存储介质,其中,所述计算机可读存储介质中存储有计算机程序,所述计算机程序被设置为运行时执行上述的方法。
本发明实施例的其它特征和优点将在随后的说明书中阐述,并且,部分地从说明书中变得显而易见,或者通过实施本发明实施例而了解。本发明实施例的目的和其他优点可通过在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
附图用来提供对本发明实施例技术方案的进一步理解,并且构成说明书的一部分,与本发明实施例的实施例一起用于解释本发明实施例的技术方案,并不构成对本发明实施例技术方案的限制。
图1为本发明一个实施例提出的分析基站安全日志的方法的流程图;
图2为本发明另一个实施例提出的分析基站安全日志的系统的结构组成示意图。
下文中将结合附图对本发明实施例进行详细说明。需要说明的是,在不冲突的情况下,本发明中的实施例及实施例中的特征可以相互任意组合。
在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行。并且,虽然在流程图中示出了逻辑顺序,但是在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤。
参见图1,本发明一个实施例提出了一种分析基站安全日志的方法,包括:
步骤100、数据汇聚系统基于第一分布式集群基站将基站实时上报的安全日志缓存到第一数据缓存系统中。
在本发明实施例中,安全日志包括一条或一条以上记录,每一条记录包括:日志类型、日志级别、基站标识(ID)、基站互联网协议(IP,Internet Protocol)地址、业务名称、PID等基本信息,也包括日志的具体内容如:安全日志的产生源、产生原因等。
例如,某一个安全日志的某一条记录为:
45673491238900123<37>1 2017-12-13T05:42:16.156Z gnb1 g2z7p-2–src_ip:192.254.1.100,u:ad,content:login fail.
其中,45673491238900123为记录的唯一标识(即安全日志标识),<37>为PRI(),1为版本号,2017-12-13T05:42:16.156Z为记录的产生时间,gnb1为主机名,g2z7p为服务 名,-为PID号,无PID时用“-”占位,2为事件号,–为结构体定义,无结构体时用“—”占位,src_ip:192.254.1.100,u:ad,content:login fail为日志的具体内容。
由上说明,每一条日志记录包括PRI,日志版本,时间,基站标识(即主机名),业务名称(即服务名),PID号,事件号,结构体,日志内容组成。
其中,通过PRI值可以解析出日志类型和日志级别;不同事件号规定了日志内容的不同格式,在本例中2号事件为用户登录失败,则日志内容包含三个属性:src_ip为登录用户的IP;u为用户使用的登录名;content为说明。
在本发明实施例中,基站的参数修改或者访问登陆都会产生安全日志,产生安全日志后实时上报。
在一个示例性实例中,第一数据缓存系统为消息中间件,基于开源分布式消息系统Kafka实现。
在本发明另一个实施例中,数据汇聚系统基于第一分布式集群将安全日志缓存到第一数据缓存系统中包括:
所述数据汇聚系统基于第一分布式集群为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中;
所述安全日志的特征值包括所述安全日志标识;
所述方法还包括:数据存储系统将所述第一对应关系、所述安全日志的特征值和所述分析结果进行持久化保存。
在一个示例性实例中,所述第一分布式集群包括:接入网关、zookeeper和两个或两个以上采集服务器(Collection-Server);所述数据汇聚系统基于第一分布式集群为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中包括:
所述zookeeper维护所述两个或两个以上采集服务器的集群状态;其中,所述集群状态包括:登陆集群状态或退出集群状态;
所述接入网关从所述zookeeper中获取所述两个或两个采集服务器的集群状态,将所述安全日志分配给所述两个或两个采集服务器中,处于登陆集群状态的所述采集服务器;
所述处于登陆集群状态的采集服务器为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中。
其中,接入网关为基站暴露统一的接入入口。
在一个示例性实例中,该方法还包括:zookeeper为采集服务器分配标识。
在一个示例性实例中,安全日志标识为每一个安全日志的全局唯一的标识,安全日志标识可以采用Twitter开源的分布式全局标识分配算法snowflake来进行分配。
在一个示例性实例中,所述数据存储系统包括:两个或两个以上数据服务器和搜索服 务器集群;
所述数据存储系统将所述第一对应关系、所述安全日志的特征值和所述分析结果进行持久化保存包括:
对于每一个所述数据服务器,所述数据服务器从所述第一数据缓存系统中获取所述第一对应关系,从所述第二数据缓存系统中获取所述特征值,将所述第一对应关系和所述特征值保存到所述搜索服务器集群中;
数据服务器从特征值实时分析系统中获取所述分析结果,将所述分析结果保存到所述搜索服务器集群中。
在一个示例性实例中,搜索服务器集群为ES(ElasticSearch)集群。
在另一个示例性实例中,数据服务器还维护第一对应关系和特征值在搜索服务器集群中的存储时间。
步骤101、日志特征计算系统基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中。
在一个示例性实例中,第二数据缓存系统为消息中间件,基于开源分布式消息系统Kafka实现。
在一个示例性实例中,所述第二分布式集群包括:两个或两个以上特征计算服务器;所述日志特征计算系统基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中包括:
对于每一个所述特征计算服务器,所述特征计算服务器计算所述安全日志的特征值,将所述安全日志的特征值缓存到所述第二数据缓存系统中。
在一个示例性实例中,特征计算服务器计算所述安全日志的特征值包括:
所述特征计算服务器从所述安全日志中提取预先定义的一个或一个以上特征向量值,将所述一个或一个以上特征向量值组成所述特征值。
例如,如表1所示,某一安全日志包括以下字段:id、PRI、time、hostname、app-name和msgld,其中,id为记录的唯一标识,PRI为主键值,time为记录的产生事件,hostname为主机名,app-name为服务名,magld为事件ID;
采用的特征向量包括:id、Facility、Severity、Time、NBId、ServerName、EventId;其中,id为记录的唯一标识,facility为设施码,Severity为严重级别,time为记录的产生时间,NBId为基站标识,SeverName为服务名,EventId为事件ID。
其中,Facility、Severity均根据字段PRI计算获得,其他特征向量之间从安全日志中提取获得,如表1所示,Facility=PRI>>3,Severity=PRI&0x7。
特征向量 | 安全日志中的字段 |
Id | Id |
Facility | PRI>>3 |
Severity | PRI&0x7 |
Time | Time |
NBId | Hostname |
ServerName | App-name |
EventId | Msgld |
表1
其中,特征向量Id的提取,Id是数据汇聚系统为日志记录分配的唯一标识,Id的提取无算法;
特征向量Facility的提取,Facility是通过日志记录中PRI的值通过位运算(右移3位)得出;
特征向量Severity的提取,Severity是通过日志记录中PRI的值通过位运算(位于0x7)得出;
特征向量Time的提取,Time是通过日志记录中的Time获取;
特征向量NBId的提取,NBId是通过日志记录中的Hostname获取;
特征向量ServerName的提取,ServerName是通过日志记录中的App-name获取;
特征向量EventId的提取,EventId是通过日志记录中的Msgld获取,MsgId还包含一些内部的特殊标识,在提取EventId时会将其剥离;
其他一些特征向量,如客户端IP,登录用户名等从日志内容中获取,日志内容为内部定义的归一化格式,适用于各服务模块。
在一个示例性实例中,为了将特征值和安全日志进行关联,可以将安全日志标识作为其中一个特征向量。
在本发明实施例中,不同的特征计算服务器可以采用相同的特征向量,也可以采用不同的特征向量,以灵活满足业务需要。
步骤102、特征值实时分析系统基于流式计算框架对安全日志的特征值进行实时分析得到分析结果。
在本发明实施例中,特征向量Facility提取了产生日志的程序的类型信息,Severity提取了日志记录中的严重级别,ServerName提取了基站产生安全日志的程序模块,因此,可以通过预定义Facility、Severity和ServerName构成的评分算法计算日志记录的分值,通过分值来评估日志是否是主要程序模块严重的安全问题;
特征向量NBId提取了基站标识,因此可以识别那个基站产生了安全问题。
在一个示例性实例中,流式计算框架可以是JStorm流式计算框架,根据业务场景定义 topology并部署到JStorm集群,topology对特征值进行分析,将分析结果发送给数据呈现装置进行展示,将分析结果保存到数据存储系统中。
其中,Topology是运行在JStorm集群上的一段代码,即为一个数据流转换图,其定义了数据的获取,计算和分发的流程。
在一个示例性实例中,定义的topology可以包含但不限于以下一个或多个场景:
满足第一预设条件的特征值的数量最大的N个基站;
在预设时间内满足第二预设条件的特征值的数量;
满足第三预设条件的安全日志。
步骤103、数据呈现装置显示分析结果。
在本发明另一个实施例中,该方法还包括:基站实时上报安全日志。
在本发明实施例中,基站上报的安全日志的消息可以符合任何协议定义,例如RFC5424协议等。
在本发明另一个实施例中,该方法还包括:
所述数据呈现装置在所述数据存储系统中查找所述分析结果对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到所述分析结果、查找到的所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系;
或者,所述数据呈现装置在所述数据存储系统中查找用户输入的查找指令对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到查找到的所述分析结果、所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系。
具体在进行查找时,可以基于安全日志标识进行查找。
本发明实施例基于分布式集群和流式计算框架实现了对基站实时上报的安全日志的实时分析,从而缩短了高危安全问题的发现时间,大幅度降低了人工分析的时间,缩短了问题反馈周期,减少了人力投入,提升了管理效率。
下面列举几个实例说明上述方法的具体实现,所列举的例子不用于限定本发明实施例的保护范围。
实例1
本实例中,统计Serverity在Warning及以上级别的安全日志的数量,并显示排名前20位的基站。
首先定义特征值包括以下的特征向量值:id、Facility、Severity、Time、NBId、ServerName,这些特征向量与安全日志中的字段之间的对应关系如表2所示。
特征向量 | 安全日志中的字段 |
Id | Id |
Facility | PRI>>3 |
Severity | PRI&0x7 |
Time | Time |
NBId | Hostname |
ServerName | App-name |
表2
要统计Warning及以上级别的安全日志的数量,需要依据特征向量Severity,根据安全日志所遵循的RFC5424协议的定义,Severity可以取到如表3所示的8个值。
表3
如表3所示可知Warning的值为4,Waring及以上级别的值为:0,1,2,3,4共5个值。由于是对基站的排序,需要依据特征向量NBId,NBId代表基站的唯一ID对安全日志的数量进行统计,也就是对于相同NBId的基站,统计特征向量Severity为0,1,2,3,4中的任何一个的值的安全日志的数量,可以每30分钟计算一次,每次计算排名前20位的基站发 送至数据呈现装置进行显示。
实例2
本实例中,统计5分钟窗口内基站登陆的数量。
首先定义特征值包括以下的特征向量值:id、Facility、Severity、Time、NBId、ServerName、EventId,这些特征向量与安全日志中的字段之间的对应关系如表1所示。
要统计5分钟窗口内基站登陆的数量,需要先明确基站登陆的字段,例如安全日志中登陆时间的msgId定义为1,需要特征向量EventId的值为1。也就是说,对于相同NBId的基站,统计特征向量EventId的值为1的安全日志的数量,数据呈现装置显示5分钟内每一个基站登陆的数量(即特征向量EventId的值为1的安全日志的数量)。
实例3
本实例中,获取Serverity为Critical的安全日志。
首先定义特征值包括以下的特征向量值:id、Facility、Severity、Time、NBId、ServerName,这些特征向量与安全日志中的字段之间的对应关系如表2所示。
要获取Critical级别的安全日志,需要依据特征向量Severity,根据安全日志所遵循的RFC5424协议的定义,Severity可以取到如表3所示的8个值。
如表3所示可知Critical的值为2。由于是对基站的统计,需要依据特征向量NBId,NBId代表基站的唯一ID对安全日志的数量进行统计,也就是对于相同NBId的基站,获取特征向量Severity为2特征值,根据特征值中的安全日志标识获取对应的安全日志,获得的安全日志发送至数据呈现装置进行显示。
参见图2,本发明另一个实施例提出了一种分析基站安全日志的系统,包括:
数据汇聚系统201,用于基于第一分布式集群2011基站将基站实时上报的安全日志缓存到第一数据缓存系统202中;
日志特征计算系统203,用于基于第二分布式集群2031计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统204中;
特征值实时分析系统205,用于基于流式计算框架2051对安全日志的特征值进行实时分析得到分析结果;
数据呈现装置206,用于显示分析结果。
在本发明另一个实施例中,还包括:基站207,用于实时上报安全日志。
在本发明实施例中,所述数据汇聚系统201具体用于:
基于第一分布式集群2011为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中;
所述安全日志的特征值包括所述安全日志标识;
所述系统还包括:
数据存储系统208,用于将所述第一对应关系、所述安全日志的特征值和所述分析结果进行持久化保存。
在本发明实施例中,所述数据呈现装置206还用于:
在所述数据存储系统中查找所述分析结果对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到所述分析结果、查找到的所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系;
或者,在所述数据存储系统中查找用户输入的查找指令对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到查找到的所述分析结果、所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系。
在本发明实施例中,所述第一分布式集群2011包括:接入网关2012、zookeeper2013和两个或两个以上采集服务器2014;
所述zookeeper2013,用于维护所述两个或两个以上采集服务器的集群状态;其中,所述集群状态包括:登陆集群状态或退出集群状态;
所述接入网关2012,用于从所述zookeeper中获取所述两个或两个采集服务器的集群状态,将所述安全日志分配给所述两个或两个采集服务器中,处于登陆集群状态的所述采集服务器;
所述处于登陆集群状态的采集服务器2014,用于为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中。
在本发明实施例中,所述数据存储系统208包括:两个或两个以上数据服务器2081和搜索服务器集群2082;
所述数据服务器2081,用于从所述第一数据缓存系统中获取所述第一对应关系,从所述第二数据缓存系统中获取所述特征值,根据所述第一对应关系和所述特征值确定所述第二对应关系,将所述第二对应关系保存到所述搜索服务器集群2082中;
从特征值实时分析系统201中获取所述分析结果,将所述分析结果保存到所述搜索服务器集群2082中。
在本发明实施例中,所述第二分布式集群2021包括:两个或两个以上特征计算服务器2022;
所述特征计算服务器2022,用于计算所述安全日志的特征值,将所述安全日志的特征值缓存到所述第二数据缓存系统中。
在本发明实施例中,所述特征计算服务器2022具体用于采用以下方式实现计算所述 安全日志的特征值:
从所述安全日志中提取预先定义的一个或一个以上特征向量值,将所述一个或一个以上特征向量值组成所述特征值。
在一个示例性实例中,流式计算框架可以是JStorm流式计算框架,根据业务场景定义topology并部署到JStorm集群,topology对特征值进行分析,将分析结果发送给数据呈现装置进行展示,将分析结果保存到数据存储系统中。
在一个示例性实例中,定义的topology可以包含但不限于以下一个或多个场景:
满足第一预设条件的特征值的数量最大的N个基站;
在预设时间内满足第二预设条件的特征值的数量;
满足第三预设条件的安全日志。
上述分析基站安全日志的系统的具体实现过程与前述实施例分析基站安全日志的方法的具体实现过程相同,这里不再赘述。
根据本发明的再一实施例,提供了一种计算机可读存储介质,其中,所述计算机可读存储介质中存储有计算机程序,所述计算机程序被设置为运行时执行上述的分析基站安全日志的方法。
本发明实施例基于分布式集群和流式计算框架实现了对基站实时上报的安全日志的实时分析,从而缩短了高危安全问题的发现时间,大幅度降低了人工分析的时间,缩短了问题反馈周期,减少了人力投入,提升了管理效率。
本领域普通技术人员可以理解,上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中,在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分;例如,一个物理组件可以具有多个功能,或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器,如数字信号处理器或微处理器执行的软件,或者被实施为硬件,或者被实施为集成电路,如专用集成电路。这样的软件可以分布在计算机可读介质上,计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的,术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外,本领域普通技术人员公知的是,通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据,并且可包括任何信息递送介质。
虽然本发明实施例所揭露的实施方式如上,但所述的内容仅为便于理解本发明实施例 而采用的实施方式,并非用以限定本发明实施例。任何本发明实施例所属领域内的技术人员,在不脱离本发明实施例所揭露的精神和范围的前提下,可以在实施的形式及细节上进行任何的修改与变化,但本发明实施例的专利保护范围,仍须以所附的权利要求书所界定的范围为准。
Claims (17)
- 一种分析基站安全日志的方法,包括:数据汇聚系统基于第一分布式集群基站将基站实时上报的安全日志缓存到第一数据缓存系统中;日志特征计算系统基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中;特征值实时分析系统基于流式计算框架对安全日志的特征值进行实时分析得到分析结果;数据呈现装置显示分析结果。
- 根据权利要求1所述的方法,其中,该方法还包括:基站实时上报所述安全日志。
- 根据权利要求1所述的方法,其中,所述数据汇聚系统基于第一分布式集群将安全日志缓存到第一数据缓存系统中包括:所述数据汇聚系统基于第一分布式集群为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中;所述安全日志的特征值包括所述安全日志标识;所述方法还包括:数据存储系统将所述第一对应关系、所述安全日志的特征值和所述分析结果进行持久化保存。
- 根据权利要求3所述的方法,其中,该方法还包括:所述数据呈现装置在所述数据存储系统中查找所述分析结果对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到所述分析结果、查找到的所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系;或者,所述数据呈现装置在所述数据存储系统中查找用户输入的查找指令对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到查找到的所述分析结果、所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系。
- 根据权利要求3所述的方法,其中,所述第一分布式集群包括:接入网关、zookeeper和两个或两个以上采集服务器;所述数据汇聚系统基于第一分布式集群为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中包括:所述zookeeper维护所述两个或两个以上采集服务器的集群状态;其中,所述集群状 态包括:登陆集群状态或退出集群状态;所述接入网关从所述zookeeper中获取所述两个或两个采集服务器的集群状态,将所述安全日志分配给所述两个或两个采集服务器中,处于登陆集群状态的所述采集服务器;所述处于登陆集群状态的采集服务器为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中。
- 根据权利要求3所述的方法,其中,所述数据存储系统包括:两个或两个以上数据服务器和搜索服务器集群;所述数据存储系统将所述安全日志、所述安全日志的特征值和分析结果进行持久化保存包括:对于每一个所述数据服务器,所述数据服务器从所述第一数据缓存系统中获取所述第一对应关系,从所述第二数据缓存系统中获取所述特征值,根据所述第一对应关系和所述特征值确定所述第二对应关系,将所述第二对应关系保存到所述搜索服务器集群中;所述数据服务器从特征值实时分析系统中获取所述分析结果,将所述分析结果保存到所述搜索服务器集群中。
- 根据权利要求1所述的方法,其中,所述第二分布式集群包括:两个或两个以上特征计算服务器;所述日志特征计算系统基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中包括:对于每一个所述特征计算服务器,所述特征计算服务器计算所述安全日志的特征值,将所述安全日志的特征值缓存到所述第二数据缓存系统中。
- 根据权利要求7所述的方法,其中,所述特征计算服务器计算所述安全日志的特征值包括:所述特征计算服务器从所述安全日志中提取预先定义的一个或一个以上特征向量值,将所述一个或一个以上特征向量值组成所述特征值。
- 一种分析基站安全日志的系统,包括:数据汇聚系统,用于基于第一分布式集群基站将基站实时上报的安全日志缓存到第一数据缓存系统中;日志特征计算系统,用于基于第二分布式集群计算安全日志的特征值,将安全日志的特征值缓存到第二数据缓存系统中;特征值实时分析系统,用于基于流式计算框架对安全日志的特征值进行实时分析得到分析结果;数据呈现装置,用于显示分析结果。
- 根据权利要求9所述的系统,其特征在于,还包括:基站,用于实时上报安全日志。
- 根据权利要求9所述的系统,其特征在于,所述数据汇聚系统具体用于:所述数据汇聚系统基于第一分布式集群为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中;所述安全日志的特征值包括所述安全日志标识;所述系统还包括:数据存储系统,用于将所述第一对应关系、所述安全日志的特征值和所述分析结果进行持久化保存。
- 根据权利要求11所述的系统,其特征在于,所述数据呈现装置还用于:在所述数据存储系统中查找所述分析结果对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到所述分析结果、查找到的所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系;或者,在所述数据存储系统中查找用户输入的查找指令对应的所述第一对应关系和所述安全日志的特征值,将查找到的所述第一对应关系和所述安全日志的特征值进行关联,得到查找到的所述分析结果、所述第一对应关系和所述安全日志的特征值之间的第二对应关系,显示所述第二对应关系。
- 根据权利要求11所述的系统,其特征在于,所述第一分布式集群包括:接入网关、zookeeper和两个或两个以上采集服务器;所述zookeeper,用于维护所述两个或两个以上采集服务器的集群状态;其中,所述集群状态包括:登陆集群状态或退出集群状态;所述接入网关,用于从所述zookeeper中获取所述两个或两个采集服务器的集群状态,将所述安全日志分配给所述两个或两个采集服务器中,处于登陆集群状态的所述采集服务器;所述处于登陆集群状态的采集服务器,用于为所述安全日志分配安全日志标识,将所述安全日志标识和所述安全日志之间的第一对应关系缓存到所述第一数据缓存系统中。
- 根据权利要求11所述的系统,其特征在于,其中,所述数据存储系统包括:两个或两个以上数据服务器和搜索服务器集群;所述数据服务器,用于从所述第一数据缓存系统中获取所述第一对应关系,从所述第二数据缓存系统中获取所述特征值,根据所述第一对应关系和所述特征值确定所述第二对应关系,将所述第二对应关系保存到所述搜索服务器集群中;从特征值实时分析系统中获取所述分析结果,将所述分析结果保存到所述搜索服务器集群中。
- 根据权利要求9所述的系统,其特征在于,其中,所述第二分布式集群包括:两 个或两个以上特征计算服务器;所述特征计算服务器,用于计算所述安全日志的特征值,将所述安全日志的特征值缓存到所述第二数据缓存系统中。
- 根据权利要求15所述的系统,其特征在于,其中,所述特征计算服务器具体用于采用以下方式实现计算所述安全日志的特征值:从所述安全日志中提取预先定义的一个或一个以上特征向量值,将所述一个或一个以上特征向量值组成所述特征值。
- 一种计算机可读存储介质,其中,所述计算机可读存储介质中存储有计算机程序,所述计算机程序被设置为运行时执行权利要求1至8中任一项所述的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910556924.2A CN112134719A (zh) | 2019-06-25 | 2019-06-25 | 一种分析基站安全日志的方法和系统 |
CN201910556924.2 | 2019-06-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020258982A1 true WO2020258982A1 (zh) | 2020-12-30 |
Family
ID=73850153
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/083742 WO2020258982A1 (zh) | 2019-06-25 | 2020-04-08 | 一种分析基站安全日志的方法、系统及计算机可读存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112134719A (zh) |
WO (1) | WO2020258982A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113792340A (zh) * | 2021-09-09 | 2021-12-14 | 烽火通信科技股份有限公司 | 一种用于数据库逻辑日志审计的方法及装置 |
CN114860774A (zh) * | 2022-05-19 | 2022-08-05 | 宁波奥克斯电气股份有限公司 | 空调器的大数据实时分析方法、系统、存储介质和空调器 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112651026B (zh) * | 2020-12-30 | 2024-05-24 | 微梦创科网络科技(中国)有限公司 | 一种具有业务安全问题的应用版本挖掘方法及装置 |
CN112463772B (zh) * | 2021-02-02 | 2022-05-27 | 北京信安世纪科技股份有限公司 | 日志处理方法、装置、日志服务器及存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103838867A (zh) * | 2014-03-20 | 2014-06-04 | 网宿科技股份有限公司 | 日志处理方法和装置 |
CN105224445A (zh) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | 分布式跟踪系统 |
US20160357778A1 (en) * | 2015-06-02 | 2016-12-08 | David MacKenzie | System for handling event messages for file collaboration |
US20170064027A1 (en) * | 2015-08-25 | 2017-03-02 | Box, Inc. | Data caching in a collaborative file sharing system |
CN108647139A (zh) * | 2018-03-19 | 2018-10-12 | 北京趣拿软件科技有限公司 | 系统的测试方法、装置、存储介质及电子装置 |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9807154B2 (en) * | 2014-09-26 | 2017-10-31 | Lenovo Enterprise Solutions (Singapore) Pte, Ltd. | Scalable logging control for distributed network devices |
CN107622084A (zh) * | 2017-08-10 | 2018-01-23 | 深圳前海微众银行股份有限公司 | 日志管理方法、系统以及计算机可读存储介质 |
CN107786565A (zh) * | 2017-11-02 | 2018-03-09 | 江苏物联网研究发展中心 | 一种分布式实时入侵检测方法及检测系统 |
CN108985981B (zh) * | 2018-06-28 | 2021-04-23 | 北京奇虎科技有限公司 | 数据处理系统及方法 |
-
2019
- 2019-06-25 CN CN201910556924.2A patent/CN112134719A/zh active Pending
-
2020
- 2020-04-08 WO PCT/CN2020/083742 patent/WO2020258982A1/zh active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103838867A (zh) * | 2014-03-20 | 2014-06-04 | 网宿科技股份有限公司 | 日志处理方法和装置 |
US20160357778A1 (en) * | 2015-06-02 | 2016-12-08 | David MacKenzie | System for handling event messages for file collaboration |
US20170064027A1 (en) * | 2015-08-25 | 2017-03-02 | Box, Inc. | Data caching in a collaborative file sharing system |
CN105224445A (zh) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | 分布式跟踪系统 |
CN108647139A (zh) * | 2018-03-19 | 2018-10-12 | 北京趣拿软件科技有限公司 | 系统的测试方法、装置、存储介质及电子装置 |
Non-Patent Citations (1)
Title |
---|
ED., C. WILDES ET AL.: "NETMOD WG Internet-Draft Intended status: Standards Track", A YANG DATA MODEL FOR SYSLOG CONFIGURATION DRAFT-IETF-NETMOD-SYSLOG-MODEL-26, 15 March 2018 (2018-03-15), DOI: 20200623140536A * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113792340A (zh) * | 2021-09-09 | 2021-12-14 | 烽火通信科技股份有限公司 | 一种用于数据库逻辑日志审计的方法及装置 |
CN113792340B (zh) * | 2021-09-09 | 2023-09-05 | 烽火通信科技股份有限公司 | 一种用于数据库逻辑日志审计的方法及装置 |
CN114860774A (zh) * | 2022-05-19 | 2022-08-05 | 宁波奥克斯电气股份有限公司 | 空调器的大数据实时分析方法、系统、存储介质和空调器 |
Also Published As
Publication number | Publication date |
---|---|
CN112134719A (zh) | 2020-12-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020258982A1 (zh) | 一种分析基站安全日志的方法、系统及计算机可读存储介质 | |
US11159386B2 (en) | Enriched flow data for network analytics | |
US11601351B2 (en) | Aggregation of select network traffic statistics | |
US20200372039A1 (en) | Data processing method, apparatus, and system | |
US10177998B2 (en) | Augmenting flow data for improved network monitoring and management | |
US9866426B2 (en) | Methods and apparatus for analyzing system events | |
US8676729B1 (en) | Network traffic classification using subspace clustering techniques | |
US11863439B2 (en) | Method, apparatus and storage medium for application identification | |
US20150039749A1 (en) | Detecting traffic anomalies based on application-aware rolling baseline aggregates | |
CN109684052B (zh) | 事务分析方法、装置、设备及存储介质 | |
US11429566B2 (en) | Approach for a controllable trade-off between cost and availability of indexed data in a cloud log aggregation solution such as splunk or sumo | |
CN108200111B (zh) | 资源配置信息更新方法、装置和资源接口设备 | |
WO2018133573A1 (zh) | 业务生存性分析方法及装置 | |
US20160226736A1 (en) | Service call graphs for website performance | |
US20220286373A1 (en) | Scalable real time metrics management | |
US11297105B2 (en) | Dynamically determining a trust level of an end-to-end link | |
WO2023071761A1 (zh) | 一种异常定位方法及装置 | |
WO2021097713A1 (zh) | 分布式安全检测系统、方法、设备及存储介质 | |
WO2022001924A1 (zh) | 构建知识图谱的方法、装置及系统、计算机存储介质 | |
Zhang et al. | C-cube: Elastic continuous clustering in the cloud | |
Elsen et al. | goProbe: a scalable distributed network monitoring solution | |
US11327969B2 (en) | Term vector modeling of database workloads | |
US20130283293A1 (en) | System and method for Intelligently distributing a plurality of transactions for parallel processing | |
CN112214290B (zh) | 日志信息处理方法、边缘节点、中心节点和系统 | |
US11537491B2 (en) | Electronic apparatus and method of controlling the same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20832463 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 30.05.2022) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20832463 Country of ref document: EP Kind code of ref document: A1 |