CN112636957A - Early warning method and device based on log, server and storage medium - Google Patents
Early warning method and device based on log, server and storage medium Download PDFInfo
- Publication number
- CN112636957A CN112636957A CN202011458954.9A CN202011458954A CN112636957A CN 112636957 A CN112636957 A CN 112636957A CN 202011458954 A CN202011458954 A CN 202011458954A CN 112636957 A CN112636957 A CN 112636957A
- Authority
- CN
- China
- Prior art keywords
- log
- logs
- early warning
- risk
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a log-based early warning method, a log-based early warning device, a log-based early warning server and a log-based storage medium. The method comprises the following steps: acquiring original logs corresponding to the systems, and processing the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs; processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs; according to a preset risk level evaluation criterion, performing risk evaluation processing on each target audit log to obtain a risk level of each target audit log; and generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item. According to the technical scheme, the log audit efficiency is improved, and the technical effect of risk early warning is further improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a log-based early warning method, a log-based early warning device, a log-based early warning server and a log-based early warning storage medium.
Background
With the continuous development of internet technology, the internet system generates massive log information in the running process, and the log can record problem information and event information of software, hardware and application in the system. For these huge amounts of log information, an efficient log auditing method becomes more and more important.
The traditional log auditing method is to store logs corresponding to all systems together, but the corresponding problems of troubleshooting still take the systems as units, namely, only a certain field of a single system can be subjected to simple statistical alarm, so that the technical problems of complex and complex troubleshooting and low efficiency of the system logs still exist.
Disclosure of Invention
The embodiment of the invention provides a log-based early warning method, a log-based early warning device, a server and a storage medium, wherein original logs corresponding to various systems are converted into audit logs with uniform formats, so that the log audit efficiency is improved when the audit logs with uniform formats are processed, and the technical effect of risk early warning is further improved.
In a first aspect, an embodiment of the present invention provides a log-based early warning method, where the method includes:
acquiring original logs corresponding to the systems, and processing the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs;
processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs;
according to a preset risk level evaluation criterion, performing risk evaluation processing on each target audit log to obtain a risk level of each target audit log;
and generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
In a second aspect, an embodiment of the present invention further provides a log-based early warning apparatus, where the apparatus includes:
the log obtaining module to be processed is used for obtaining original logs corresponding to the systems and processing the corresponding original logs based on the configuration files of the systems to obtain the logs to be processed corresponding to the original logs;
the target audit log obtaining module is used for processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs;
the risk level obtaining module is used for carrying out risk evaluation processing on each target audit log according to a preset risk level evaluation criterion to obtain the risk level of each target audit log;
and the early warning information generation module is used for generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
In a third aspect, an embodiment of the present invention further provides a server, where the server includes:
one or more processors;
a storage device for storing one or more programs which, when executed by the processor, cause the processor to implement a log-based warning method as provided by any embodiment of the invention.
In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the log-based warning method provided in any of the embodiments of the present invention.
According to the technical scheme of the embodiment of the invention, the logs of each system are uniformly stored but are also checked in the checking process of the logs of a single system in the prior art, when the number of the logs is increased, the system logs are checked and counted fussy, and the system logs are obtained, The problem of complicacy and efficiency are extremely low has reached and has improved log audit efficiency, and then has improved the technological effect of risk early warning.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, a brief description is given below of the drawings used in describing the embodiments. It should be clear that the described figures are only views of some of the embodiments of the invention to be described, not all, and that for a person skilled in the art, other figures can be derived from these figures without inventive effort.
Fig. 1 is a schematic flowchart of an early warning method based on logs according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of an early warning method based on logs according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of an early warning method based on logs according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of an early warning method based on logs according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of an early warning method based on logs according to an embodiment of the present invention;
fig. 6 is a schematic flowchart of an early warning method based on logs according to a second embodiment of the present invention;
fig. 7 is a schematic diagram of a log-based early warning device module according to a third embodiment of the present invention;
fig. 8 is a schematic structural diagram of a server according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of an early warning method based on logs according to an embodiment of the present invention, where the method is applicable to a situation where original logs of various systems are processed to obtain risk levels corresponding to the original logs, and the method may be executed by an early warning device based on logs, where the early warning device based on logs may be implemented in a software and/or hardware manner, and the early warning device based on logs may be integrated in an electronic device such as a computer or a server.
As shown in fig. 1, the method of the present embodiment includes:
s110, obtaining original logs corresponding to the systems, and processing the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs.
The original log may be understood as an event record generated when each system operates, and the content of the original log may include, but is not limited to, the following information: date, time, user and action. One of the systems may correspond to one raw log or a plurality of raw logs. The configuration file may be understood as a file configured for the original log corresponding to each system, and may be preset according to requirements, and the configuration file may be a file used for performing screening, marking, and format conversion processing on the original log. The log to be processed may be a log obtained by performing operations of screening, marking, and converting a format on an original log. The screening can be filtering invalid logs in the original log or extracting target fields in the original log. The marking can be adding identification to each field of the original log according to requirements. The conversion format may be understood as converting data in the original log into data in a preset format, and the preset format may be a json (JavaScript, object notation) format.
Specifically, the log file of each system is obtained through the log collection system, that is, the original log corresponding to each system is obtained. The method comprises the steps of filtering invalid original logs in each system by configuring log screening rules of each system to obtain valid logs, marking data in the valid logs to obtain marked logs, and converting formats of the data in the marked logs to obtain logs to be processed corresponding to the original logs. It should be noted that the log collection system in this embodiment is not limited, as long as the log collection function can be implemented.
Illustratively, the system a corresponds to a configuration file 1, and the content of the configuration file 1 includes: the method comprises the steps of presetting a screening rule of the log, presetting an identification rule corresponding to a field in the log and presetting a converted data format rule. The method comprises the steps of obtaining an original log 1 and an original log 2 corresponding to a system A, filtering the original log 1 and the original log 2 in the system A according to a preset log screening rule in a configuration file 1, obtaining that the original log 1 is an invalid log when detecting that data corresponding to a preset field in the original log 1 is empty, and filtering the original log 1 to obtain the valid original log 2. And adding corresponding identifiers to data corresponding to a user field, a host address field, a time field and an event field in the original log 2 according to an identifier rule corresponding to a field in a preset log in the configuration file 1 to obtain the original log 2 carrying the identifiers. And converting the data corresponding to the identification field carried in the original log 2 into data in a json format according to a preset converted data format rule in the configuration file 1.
Optionally, after obtaining the log to be processed, the method further includes: and sending the logs to be processed to a log server so that the log server can perform centralized processing on the logs to be processed corresponding to the original logs.
The log server is a server for further processing the logs to be processed in all the systems.
Specifically, the logs to be processed are sent to the log server, and the logs to be processed can be uniformly processed based on the log server.
For a clear description of the technical solution of the present embodiment, refer to the following embodiments, as shown in fig. 2, a flash (log collection system) may be set in each system, and a agent (agent module) in the flash (log collection system) is used to determine data collection manners corresponding to different data sources, so that the original logs may be collected based on the corresponding data collection manners. After the original log is obtained, a configuration file corresponding to the system may be called, and the original log may be processed, such as screening, marking, and format conversion, according to a processing rule or a processing condition set in the configuration file, to obtain an operation log. And sending the operation log to a log server while obtaining the operation log so as to uniformly store the logs to be processed by the Elasticissearch on the log server.
And S120, processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs.
The mapping rule may be understood as a corresponding relationship between a field in the log to be processed and a field in the target audit log, and the mapping rule may be a logic rule manually set for each system. The target audit log can be understood as a log with a uniform format corresponding to each system. The uniform format may be understood as a target format into which all preset original logs need to be converted, and the field names are the same.
Specifically, a mapping rule corresponding to the log to be processed of each system is configured according to the format of the log to be processed of each system, and the log to be processed corresponding to each system is processed according to the mapping rule corresponding to the log to be processed of each system, so that a target audit log is obtained.
Illustratively, the preset mapping rule is: the value in the field "cmd" is "AUTH" which represents user authentication, and the field "action" value mapped to our audit log is "user authentication"; the value of the field 'AuthUser' is a user account, the user account is mapped to the field 'username' of the audit log, and the subsequent '@' is removed; when the field "Result" has a value of "Failed", the field "status" mapped to the audit log has a value of "Failed". According to a preset mapping rule, setting the log content as cmd: AUTH; AuthUser: showrard @; and (4) Result: failed, converted to action: user authentication; the username: showrard; status: a failed audit log.
The method for converting the logs to be processed, namely the operation logs into the audit logs, can be seen in fig. 3, the operation logs of the system a are converted into the audit logs a according to the conversion rule corresponding to the system a, the operation logs of the system B are converted into the audit logs B according to the conversion rule corresponding to the system B, and the operation logs of the system C are converted into the audit logs C according to the conversion rule corresponding to the system C, wherein the log formats of the audit logs a, the audit logs B and the audit logs C are uniform. And the audit log A, the audit log B and the audit log C are centrally stored in the log server.
And S130, performing risk evaluation processing on each target audit log according to a preset risk grade evaluation criterion to obtain the risk grade of each target audit log.
The risk level evaluation criterion may be a risk level corresponding to each field, for example, the risk level of the login IP address beijing is 1, and the risk level of the login IP address shanghai is 5.
Specifically, the preset risk level evaluation criterion is used for carrying out risk evaluation processing on each target audit log according to the risk level evaluation criterion to obtain the risk level of each target audit log, so that a risk evaluation report is obtained according to the risk level of each target audit log, and early warning notification is carried out according to the risk evaluation report.
Illustratively, a risk level evaluation criterion is preset, wherein the risk level evaluation criterion comprises: the risk level corresponding to the number of triggering events 1 in the range of 1 to 3 is 1, the risk level corresponding to the number of triggering events 1 in the range of 4 to 6 is 2, and the risk level corresponding to the number of triggering events 1 in the range of 7 and 7 or more is 3. And when the occurrence frequency of the corresponding event 1 in the target audit log is detected to be 8, the risk level of the target audit log is 3.
For a clear description of the technical solution of the present embodiment, refer to the following embodiments, as shown in fig. 4, by presetting a custom rule, and configuring the custom rule in a rule engine, performing rule calculation on an audit log with a uniform format at a preset time interval by the rule engine to obtain a risk assessment report corresponding to the audit log, and determining a risk level corresponding to the audit log according to the risk assessment report, thereby obtaining the audit log after risk assessment.
And S140, generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
Wherein, the risk assessment item can be one field or a plurality of fields of the target audit log, and the risk assessment item can be set according to experience. The pre-warning information may include the content of the pre-warning, such as the specific content of the audit log, and the corresponding system. The early warning mode corresponding to the early warning information can be at least one of a character early warning mode, a voice early warning mode, a short message, a mail and an alarm, and a user can set a corresponding early warning mode according to actual requirements.
In this embodiment, the manner of generating the warning information may include at least one, and the first embodiment may be: presetting a corresponding relation between the risk level and the early warning information, acquiring audit logs in preset time, determining the risk level corresponding to each log in the preset time, and generating the early warning information corresponding to the risk level of each target audit log according to the risk level of each target audit log.
In a second implementation manner, at least one field of the audit log is set as a risk assessment item according to experience, if field 1 is set as a risk assessment item, and when data corresponding to field 1 in a target audit log is detected, an early warning corresponding to the target audit log is generated to prompt a worker to process the target audit log in time.
In order to effectively reduce the system risk, an early warning rule corresponding to the early warning information is determined on the basis of the above embodiment, and the early warning information is sent to the corresponding terminal device based on the early warning rule.
The early warning rule may be a preset rule, and the early warning rule may be a manner of sending early warning information. Different early warning rules can be set for different risk levels, for example, when the risk level value is higher, the early warning rules can send early warning information to corresponding users in three ways.
For a clear description of the technical solution of the present embodiment, refer to the following embodiments, as shown in fig. 5, early warning rules may be customized, for example, which configuration items are customized as risk assessment items; and inputting the audit log after risk evaluation into the early warning engine, and simultaneously inputting external data into the early warning engine. The early warning log and the external data within a certain time length can be periodically acquired from the early warning engine, and optionally, the certain time length is consistent with the periodic time length. After the early warning log is obtained, whether the early warning information is triggered or not can be determined according to the early warning log. If the early warning information is triggered, a corresponding early warning path can be selected, and optionally, the early warning path can comprise enterprise WeChat and other IM software, APP message push or mail, voice telephone or image text short message. After the early warning path is determined, corresponding early warning information can be sent.
According to the technical scheme of the embodiment of the invention, the logs of each system are uniformly stored but are also checked in the checking process of the logs of a single system in the prior art, when the number of the logs is increased, the system logs are checked and counted fussy, and the system logs are obtained, The problem of complicacy and efficiency are extremely low has reached and has improved log audit efficiency, and then has improved the technological effect of risk early warning.
Example two
Fig. 6 is a schematic flowchart of an early warning method based on a log according to a second embodiment of the present invention, and on the basis of the foregoing embodiment, each step in the first embodiment may be explained in detail, and a specific implementation manner thereof may refer to a technical solution of this embodiment.
The technical terms that are the same as or corresponding to the above embodiments are not repeated herein.
As shown in fig. 6, the method of the present embodiment may specifically include:
s210, aiming at each system, calling a configuration file corresponding to the current system, and extracting configuration items in the configuration file.
One system can be one configuration file or a plurality of configuration files. A configuration file may include a configuration item or a plurality of configuration items.
Illustratively, a configuration file 1 corresponding to a system a and a configuration file 2 corresponding to a system B are configured in advance, the current system is the system a, the configuration file 1 corresponding to the system a is called, and a configuration item 1 and a configuration item 2 in the configuration files are obtained.
S220, formatting the original log corresponding to the current system based on the configuration item to obtain the log to be processed.
Exemplarily, configuration items in a configuration file 1 corresponding to a current system a are a configuration item 1, a configuration item 2, and a configuration item 3, and fields in an original log corresponding to the system a include a field 1, a field 2, a field 3, and a field 4, where the configuration item 1 corresponds to the field 1, the configuration item 2 corresponds to the field 3, and the configuration item 3 converts a data format into a json format, and according to the configuration item 1 and the configuration item 2 of the system a, the obtained fields in a log to be processed of the system a are the field 1 and the field 3, and the log to be processed is the field 1: content 1; field 2: content 2.
And S230, for each system, calling a target mapping rule of the current system, and processing the to-be-processed log corresponding to the current system into a target audit log in a target format based on the target mapping rule.
And the target formats corresponding to the logs to be processed are the same. The target mapping rule may be a mapping rule of a field of the log to be processed, or may be a mapping rule of a field value corresponding to the field to be processed.
Specifically, the mapping rules of each system are configured in advance, the mapping rules of the current system are called, format processing is performed on the logs to be processed corresponding to the current system according to the mapping rules of the current system, and the audit logs corresponding to the mapping rules of the current system are obtained.
Illustratively, the system a corresponds to a mapping rule 1, where the mapping rule 1 is to map the content corresponding to the field 1 in the log to be processed into the content corresponding to the configuration item 1, map the data in the field 2 of the log to be processed into the field corresponding to the configuration item 2, and map the content corresponding to the field 2 of the log to be processed into the content corresponding to the configuration item 2. The method comprises the steps of obtaining a mapping rule 1 corresponding to a system A, processing a log to be processed corresponding to the system A according to the mapping rule 1, analyzing the log to be processed corresponding to the system A, updating a content Zusanli corresponding to a field 1 in the log to be processed into a content corresponding to a configuration item 1 according to the mapping rule 1 when the field in the log to be processed is detected to be the field 1, replacing a field 2 ' user name ' in a day to be processed with the user name ' according to the mapping rule 1 when the field in the log to be processed is detected to be the field 2, and mapping the ' Litez ' corresponding to the ' user name ' into the ' Litez ' corresponding to the ' user name '.
S240, determining the risk evaluation level of each target audit log according to the preset risk level evaluation criteria corresponding to each event type, each IP address and the operation data type.
The event type may be understood as an event type of the log, where the event type of the log may be the following event types: management, directory extraction, convention, integration, parameters, user management, translation, and the like.
In this embodiment, the preset risk level evaluation criterion may be: presetting a corresponding risk level of each event type, presetting a risk level of each IP address, presetting a risk level of an operation data type, and setting a risk weight of each event type, each IP address and the operation data type. And based on the obtained target audit log, obtaining the event type, the IP address and the operation data type of the target audit log, and obtaining the risk evaluation grade of the target audit log according to the risk weight of each event type, each IP address and the operation data type.
Illustratively, the risk weights corresponding to the preset event type 1 and the event type 2 are 0.8 and 0.2, the risk weights corresponding to the preset IP address 1 and the IP address are 0.6 and 0.4, and the risk weights corresponding to the operation data type 1 and the operation data type 2 are 0.3 and 0.7, respectively. The event type, the IP address and the operation data type of the target audit log are respectively event type 2, preset IP address 1 and risk value of operation data type 1 are 0.2 × 0.6 × 0.3 × 0.036, the relationship between the risk value of the preset audit log and the risk evaluation grade is preset, and the risk evaluation grade of the target audit log is 1-grade risk evaluation grade according to the corresponding 1-grade risk evaluation grade within the risk value range of the preset audit log from 0 to 0.1.
And S250, generating early warning information based on the risk level of each target audit log and/or a predefined risk assessment item.
Optionally, the generating of the early warning information based on the risk level of each target audit log and/or a predefined risk assessment item includes: when the risk level corresponding to the target audit log is higher than a first preset risk level threshold value, generating early warning information; or determining a risk assessment grade corresponding to a predefined risk assessment item, and generating early warning information when the risk grade corresponding to the risk assessment grade is higher than a second preset risk grade threshold value.
Wherein the first preset risk level threshold may correspond to a risk level of the audit log. The second preset risk level threshold may correspond to a risk level of the audit log.
In this embodiment, the manner of generating the warning information may include at least one, and the first implementation may be: acquiring a risk level (for example, a level 3 risk) corresponding to the target audit log, setting a preset risk level threshold as a level 2 risk, and generating alarm information corresponding to the target audit log when detecting that the risk level corresponding to the target audit log is higher than the risk level threshold, that is, the level 3 risk of the target audit log is higher than the preset level 2 risk threshold.
A second implementation manner may be that the risk assessment levels corresponding to the set risk assessment items are preset, for example, the risk assessment item 1, the risk assessment item 2, and the risk assessment item 3 correspond to the level 1 risk assessment level, the level 2 risk assessment level, and the level 3 risk assessment level, respectively, when detecting that the field 1 of the target audit log corresponds to the risk assessment item 3, according to the level 3 risk assessment level corresponding to the risk assessment item 3, it is determined that the target audit log corresponds to the level 3 assessment level, and the preset risk level threshold is a level 2 risk, and when detecting that the risk level corresponding to the target audit log is higher than the risk level threshold, that is, the level 3 risk of the target audit log is higher than the preset level 2 risk threshold, the alarm information corresponding to the target audit log is generated.
In the technical scheme of this embodiment, a configuration file corresponding to a current system is called for each system, a configuration item in the configuration file is extracted, an original log corresponding to the current system is formatted based on the configuration item to obtain a log to be processed, a target mapping rule of the current system is called for each system, the log to be processed corresponding to the current system is processed into a target audit log in a target format based on the target mapping rule, wherein the target format corresponding to each log to be processed is the same, a risk evaluation level of each target audit log is determined according to preset risk level evaluation criteria corresponding to each event type, each IP address and operation data type, and early warning information is generated based on the risk level of each target audit log and/or a predefined risk evaluation item, thereby solving the problem that logs of each system are uniformly stored in the prior art, however, when the problem is eliminated, the logs of a single system are still eliminated, and when the number of the logs is increased, the problems of complex and complicated system log elimination and statistics and extremely low efficiency exist, so that the technical effects of improving the log audit efficiency and further improving the risk early warning are achieved.
EXAMPLE III
Fig. 7 is a schematic diagram of a module of a log-based warning device according to a third embodiment of the present invention, where the invention provides a log-based warning device, the device includes:
a to-be-processed log obtaining module 310, configured to obtain original logs corresponding to the systems, and process the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs;
a target audit log obtaining module 320, configured to process the corresponding to-be-processed logs according to the mapping rule corresponding to each system, so as to obtain target audit logs corresponding to each original log;
a risk level obtaining module 330, configured to perform risk evaluation processing on each target audit log according to a preset risk level evaluation criterion, so as to obtain a risk level of each target audit log;
and the early warning information generating module 340 is configured to generate early warning information based on the risk level of each target audit log and/or a predefined risk assessment item.
In the technical scheme of this embodiment, the original logs corresponding to each system are obtained by the log obtaining module to be processed, the corresponding original logs are processed based on the configuration file of each system to obtain the logs to be processed corresponding to each original log, the corresponding logs to be processed are processed by the target audit log obtaining module according to the mapping rule corresponding to each system to obtain the target audit logs corresponding to each original log, the target audit logs are subjected to risk assessment processing by the risk level obtaining module according to the preset risk level assessment criteria to obtain the risk levels of each target audit log, the early warning information is generated by the early warning information generating module based on the risk levels of each target audit log and/or the predefined risk assessment items, thereby solving the problem that the prior art uniformly stores the logs of each system, however, when the problem is eliminated, the logs of a single system are still eliminated, and when the number of the logs is increased, the problems of complex and complicated system log elimination and statistics and extremely low efficiency exist, so that the technical effects of improving the log audit efficiency and further improving the risk early warning are achieved. Optionally, the log to be processed obtaining module 310 is configured to, for each system, invoke a configuration file corresponding to the current system, and extract a configuration item in the configuration file; and formatting the original log corresponding to the current system based on the configuration item to obtain the log to be processed.
Optionally, the to-be-processed log obtaining module 310 is configured to send the to-be-processed log to a log server, so that the log server performs centralized processing on the to-be-processed logs corresponding to the original logs.
Optionally, the target audit log obtaining module 320 is configured to, for each system, invoke a target mapping rule of a current system, and process a to-be-processed log corresponding to the current system into a target audit log in a target format based on the target mapping rule; and the target formats corresponding to the logs to be processed are the same.
Optionally, the risk level obtaining module 330 is configured to determine a risk evaluation level of each target audit log according to a preset risk level evaluation criterion corresponding to each event type, each IP address, and the operation data type.
Optionally, the early warning information generating module 340 is configured to generate early warning information when it is detected that the risk level corresponding to the target audit log is higher than a first preset risk level threshold; or determining a risk assessment grade corresponding to a predefined risk assessment item, and generating early warning information when the risk grade corresponding to the risk assessment grade is higher than a second preset risk grade threshold value.
Optionally, the apparatus further comprises: and an early warning information sending module 350, configured to determine an early warning rule corresponding to the early warning information, and send early warning information to a corresponding terminal device based on the early warning rule.
The log-based early warning device provided by the embodiment of the invention can execute the log-based early warning method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
It should be noted that, the units and modules included in the apparatus are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the embodiment of the invention.
Example four
Fig. 8 is a schematic structural diagram of a server according to a fourth embodiment of the present invention. FIG. 8 illustrates a block diagram of an exemplary server 12 suitable for use in implementing embodiments of the present invention. The server 12 shown in fig. 8 is only an example, and should not bring any limitation to the function and the scope of use of the embodiment of the present invention.
As shown in FIG. 8, the server 12 is in the form of a general purpose computing device. The components of the server 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
The server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by server 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The server 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 8, and commonly referred to as a "hard drive"). Although not shown in FIG. 8, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The server 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with the server 12, and/or with any devices (e.g., network card, modem, etc.) that enable the server 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the server 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 8, the network adapter 20 communicates with the other modules of the server 12 via the bus 18. It should be appreciated that although not shown in FIG. 8, other hardware and/or software modules may be used in conjunction with the server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, implementing a log-based warning method provided by the embodiment of the present invention, the method includes:
acquiring original logs corresponding to the systems, and processing the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs;
processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs;
according to a preset risk level evaluation criterion, performing risk evaluation processing on each target audit log to obtain a risk level of each target audit log;
and generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
Of course, those skilled in the art can understand that the processor can also implement the technical solution of the log-based early warning method provided by any embodiment of the present invention.
EXAMPLE five
A fifth embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a log-based early warning method.
The method comprises the following steps:
acquiring original logs corresponding to the systems, and processing the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs;
processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs;
according to a preset risk level evaluation criterion, performing risk evaluation processing on each target audit log to obtain a risk level of each target audit log;
and generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A log-based early warning method is characterized by comprising the following steps:
acquiring original logs corresponding to the systems, and processing the corresponding original logs based on the configuration files of the systems to obtain to-be-processed logs corresponding to the original logs;
processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs;
according to a preset risk level evaluation criterion, performing risk evaluation processing on each target audit log to obtain a risk level of each target audit log;
and generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
2. The method of claim 1, wherein the obtaining of the original logs corresponding to the systems and the processing of the corresponding original logs based on the configuration files of the systems to obtain the to-be-processed logs corresponding to the original logs comprises:
for each system, calling a configuration file corresponding to the current system, and extracting configuration items in the configuration file;
and formatting the original log corresponding to the current system based on the configuration item to obtain the log to be processed.
3. The method of claim 1, after obtaining the pending log, further comprising:
and sending the logs to be processed to a log server so that the log server can perform centralized processing on the logs to be processed corresponding to the original logs.
4. The method of claim 1, wherein processing the respective logs to be processed according to the mapping rules corresponding to the respective systems to obtain target audit logs corresponding to the respective original logs comprises:
for each system, calling a target mapping rule of a current system, and processing a log to be processed corresponding to the current system into a target audit log in a target format based on the target mapping rule;
and the target formats corresponding to the logs to be processed are the same.
5. The method of claim 1, wherein the performing risk assessment processing on each target audit log according to a preset risk level assessment criterion to obtain a risk level of each target audit log comprises:
and determining the risk evaluation level of each target audit log according to the preset risk level evaluation criteria corresponding to each event type, each IP address and the operation data type.
6. The method of claim 1, wherein generating early warning information based on the risk level of each target audit log and/or predefined risk assessment terms comprises:
when the risk level corresponding to the target audit log is higher than a first preset risk level threshold value, generating early warning information; or the like, or, alternatively,
and determining a risk evaluation grade corresponding to a predefined risk evaluation item, and generating early warning information when the risk grade corresponding to the risk evaluation grade is higher than a second preset risk grade threshold value.
7. The method of claim 1 or 6, further comprising:
and determining an early warning rule corresponding to the early warning information, and sending the early warning information to corresponding terminal equipment based on the early warning rule.
8. A log-based early warning device, comprising:
the log obtaining module to be processed is used for obtaining original logs corresponding to the systems and processing the corresponding original logs based on the configuration files of the systems to obtain the logs to be processed corresponding to the original logs;
the target audit log obtaining module is used for processing the corresponding logs to be processed according to the mapping rules corresponding to the systems to obtain target audit logs corresponding to the original logs;
the risk level obtaining module is used for carrying out risk evaluation processing on each target audit log according to a preset risk level evaluation criterion to obtain the risk level of each target audit log;
and the early warning information generation module is used for generating early warning information based on the risk level of each target audit log and/or a predefined risk evaluation item.
9. A server, characterized in that the server comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the processor, cause the processor to implement the log-based warning method of any one of claims 1-7.
10. A storage medium containing computer executable instructions for performing the log-based warning method of any one of claims 1-7 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011458954.9A CN112636957B (en) | 2020-12-11 | 2020-12-11 | Early warning method and device based on log, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011458954.9A CN112636957B (en) | 2020-12-11 | 2020-12-11 | Early warning method and device based on log, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112636957A true CN112636957A (en) | 2021-04-09 |
| CN112636957B CN112636957B (en) | 2023-02-21 |
Family
ID=75309977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011458954.9A Active CN112636957B (en) | 2020-12-11 | 2020-12-11 | Early warning method and device based on log, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112636957B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268637A (en) * | 2021-06-24 | 2021-08-17 | 京东科技控股股份有限公司 | Data processing method, system, storage medium and electronic equipment |
| CN113762914A (en) * | 2021-07-23 | 2021-12-07 | 北京国电通网络技术有限公司 | Early warning auditing method and related equipment |
| CN113986843A (en) * | 2021-11-02 | 2022-01-28 | 青岛海尔工业智能研究院有限公司 | Data risk early warning processing method and device and electronic equipment |
| CN114915488A (en) * | 2022-06-15 | 2022-08-16 | 中国联合网络通信集团有限公司 | Flow calculation monitoring method and apparatus |
| CN115021977A (en) * | 2022-05-17 | 2022-09-06 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted machine system, vehicle comprising same, early warning method and storage medium |
| CN115408344A (en) * | 2022-09-29 | 2022-11-29 | 建信金融科技有限责任公司 | Log formatting method and device, electronic equipment and storage medium |
| CN116599690A (en) * | 2023-03-28 | 2023-08-15 | 中国船舶集团有限公司综合技术经济研究院 | Ship information security event processing method and device and computer equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150381637A1 (en) * | 2010-07-21 | 2015-12-31 | Seculert Ltd. | System and methods for malware detection using log based crowdsourcing analysis |
| CN105528280A (en) * | 2015-11-30 | 2016-04-27 | 中电科华云信息技术有限公司 | Method and system capable of determining log alarm grades according to relationship between system logs and health monitoring |
| CN106169959A (en) * | 2016-07-21 | 2016-11-30 | 柳州龙辉科技有限公司 | A kind of log processing device |
| CN107818150A (en) * | 2017-10-23 | 2018-03-20 | 中国移动通信集团广东有限公司 | A kind of log audit method and device |
| CN108494727A (en) * | 2018-02-06 | 2018-09-04 | 成都清华永新网络科技有限公司 | A kind of security incident closed-loop process method for network security management |
| CN109255518A (en) * | 2018-08-01 | 2019-01-22 | 阿里巴巴集团控股有限公司 | Data application risk appraisal procedure, device and system |
| CN109379374A (en) * | 2018-11-23 | 2019-02-22 | 四川长虹电器股份有限公司 | Threat identification method for early warning and system based on event analysis |
| CN109767351A (en) * | 2018-12-24 | 2019-05-17 | 国网山西省电力公司信息通信分公司 | A kind of security postures cognitive method of power information system daily record data |
| CN110166290A (en) * | 2019-05-16 | 2019-08-23 | 平安科技（深圳）有限公司 | Alarm method and device based on journal file |
| CN111045847A (en) * | 2019-12-18 | 2020-04-21 | Oppo广东移动通信有限公司 | Event auditing method and device, terminal equipment and storage medium |
| CN111199361A (en) * | 2020-01-13 | 2020-05-26 | 国网福建省电力有限公司信息通信分公司 | Electric power information system health assessment method and system based on fuzzy reasoning theory |
| CN111831528A (en) * | 2020-07-17 | 2020-10-27 | 浪潮商用机器有限公司 | Computer system log association method and related device |
-
2020
- 2020-12-11 CN CN202011458954.9A patent/CN112636957B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150381637A1 (en) * | 2010-07-21 | 2015-12-31 | Seculert Ltd. | System and methods for malware detection using log based crowdsourcing analysis |
| CN105528280A (en) * | 2015-11-30 | 2016-04-27 | 中电科华云信息技术有限公司 | Method and system capable of determining log alarm grades according to relationship between system logs and health monitoring |
| CN106169959A (en) * | 2016-07-21 | 2016-11-30 | 柳州龙辉科技有限公司 | A kind of log processing device |
| CN107818150A (en) * | 2017-10-23 | 2018-03-20 | 中国移动通信集团广东有限公司 | A kind of log audit method and device |
| CN108494727A (en) * | 2018-02-06 | 2018-09-04 | 成都清华永新网络科技有限公司 | A kind of security incident closed-loop process method for network security management |
| CN109255518A (en) * | 2018-08-01 | 2019-01-22 | 阿里巴巴集团控股有限公司 | Data application risk appraisal procedure, device and system |
| CN109379374A (en) * | 2018-11-23 | 2019-02-22 | 四川长虹电器股份有限公司 | Threat identification method for early warning and system based on event analysis |
| CN109767351A (en) * | 2018-12-24 | 2019-05-17 | 国网山西省电力公司信息通信分公司 | A kind of security postures cognitive method of power information system daily record data |
| CN110166290A (en) * | 2019-05-16 | 2019-08-23 | 平安科技（深圳）有限公司 | Alarm method and device based on journal file |
| CN111045847A (en) * | 2019-12-18 | 2020-04-21 | Oppo广东移动通信有限公司 | Event auditing method and device, terminal equipment and storage medium |
| CN111199361A (en) * | 2020-01-13 | 2020-05-26 | 国网福建省电力有限公司信息通信分公司 | Electric power information system health assessment method and system based on fuzzy reasoning theory |
| CN111831528A (en) * | 2020-07-17 | 2020-10-27 | 浪潮商用机器有限公司 | Computer system log association method and related device |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268637A (en) * | 2021-06-24 | 2021-08-17 | 京东科技控股股份有限公司 | Data processing method, system, storage medium and electronic equipment |
| CN113762914A (en) * | 2021-07-23 | 2021-12-07 | 北京国电通网络技术有限公司 | Early warning auditing method and related equipment |
| CN113986843A (en) * | 2021-11-02 | 2022-01-28 | 青岛海尔工业智能研究院有限公司 | Data risk early warning processing method and device and electronic equipment |
| CN115021977A (en) * | 2022-05-17 | 2022-09-06 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted machine system, vehicle comprising same, early warning method and storage medium |
| CN114915488A (en) * | 2022-06-15 | 2022-08-16 | 中国联合网络通信集团有限公司 | Flow calculation monitoring method and apparatus |
| CN115408344A (en) * | 2022-09-29 | 2022-11-29 | 建信金融科技有限责任公司 | Log formatting method and device, electronic equipment and storage medium |
| CN115408344B (en) * | 2022-09-29 | 2023-12-08 | 建信金融科技有限责任公司 | Log formatting method, device, electronic equipment and storage medium |
| CN116599690A (en) * | 2023-03-28 | 2023-08-15 | 中国船舶集团有限公司综合技术经济研究院 | Ship information security event processing method and device and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112636957B (en) | 2023-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112636957B (en) | Early warning method and device based on log, server and storage medium | |
| JP7373611B2 (en) | Log auditing methods, equipment, electronic equipment, media and computer programs | |
| CN111866016B (en) | Log analysis method and system | |
| CN112162965B (en) | Log data processing method, device, computer equipment and storage medium | |
| CN110610196A (en) | Desensitization method, system, computer device and computer-readable storage medium | |
| CN109450869B (en) | Service safety protection method based on user feedback | |
| CN112084179B (en) | Data processing method, device, equipment and storage medium | |
| CN111708673A (en) | Log data compression method, device, equipment and storage medium | |
| CN110888791A (en) | Log processing method, device, equipment and storage medium | |
| CN114595765A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN114153703A (en) | Micro-service exception positioning method and device, electronic equipment and program product | |
| CN112347066B (en) | Log processing method and device, server and computer readable storage medium | |
| CN114205156A (en) | Message detection method and device for tangent plane technology, electronic equipment and medium | |
| CN112346938B (en) | Operation auditing method and device, server and computer readable storage medium | |
| CN113342619A (en) | Log monitoring method and system, electronic device and readable medium | |
| CN113806556A (en) | Method, device, equipment and medium for constructing knowledge graph based on power grid data | |
| CN113778977A (en) | Data processing method and data processing device | |
| CN112131611A (en) | Data correctness verification method, device, equipment, system and storage medium | |
| CN113269547B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN115604668B (en) | Short message sending and pushing monitoring method, device, equipment and storage medium | |
| CN112702270B (en) | Node calling method, system and storage medium based on event distribution mechanism | |
| CN109525630B (en) | Method, apparatus, medium, and electronic device for transmitting data analysis request | |
| CN109032724B (en) | Method, device, terminal and storage medium for processing user-defined bullet screen message | |
| CN115860707A (en) | Received mail associated work order method, device, electronic equipment and storage medium | |
| CN115879166A (en) | Data identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |