CN111198850A - Log message processing method and device and Internet of things platform - Google Patents
Log message processing method and device and Internet of things platform Download PDFInfo
- Publication number
- CN111198850A CN111198850A CN201911287325.1A CN201911287325A CN111198850A CN 111198850 A CN111198850 A CN 111198850A CN 201911287325 A CN201911287325 A CN 201911287325A CN 111198850 A CN111198850 A CN 111198850A
- Authority
- CN
- China
- Prior art keywords
- message
- log
- log message
- target
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Abstract
The application is applicable to the technical field of communication of the Internet of things, and provides a log message processing method, a device, an Internet of things platform and a computer readable storage medium, wherein the method comprises the following steps: the method comprises the steps of obtaining a log message to be processed, extracting a target log message of the log message to be processed, and translating the target log message according to a preset rule, wherein the target log message can be a log message which needs to be concerned by a user. According to the log message processing method and device, the log message needing to be paid attention is not required to be searched from the huge message volume of the log message by a user in a manual searching mode, the log message needing to be paid attention by the user is directly extracted and translated, and the effect that the user can conveniently check the log message needing to be paid attention is achieved.
Description
Technical Field
The application belongs to the technical field of communication of the internet of things, and particularly relates to a log message processing method and device, an internet of things platform and a computer-readable storage medium.
Background
With the development of science and technology, the application of the internet of things is more and more extensive. Generally, the internet of things platform records log messages reported by the internet of things device, so that a user, such as a developer, a tester or an operation and maintenance person, can conveniently troubleshoot the internet of things device fault and check the running state of the internet of things device according to the log messages recorded by the internet of things platform.
Because the functions and designs of different internet of things devices are greatly different, the communication protocols between different internet of things devices and the formats and contents of reported log messages also differ, and therefore a user needs to know various internet of things device protocols to acquire the log messages needing attention from the log messages.
However, the log messages reported by the internet of things device contain a huge amount of messages, so that it is time-consuming and labor-consuming for a user to search the log messages needing attention from the huge amount of messages of the log messages, and the log messages needing attention cannot be searched conveniently.
Disclosure of Invention
The embodiment of the application provides a method and a device for processing log messages, and the problem that log messages needing attention cannot be conveniently searched from the log messages reported by Internet of things equipment in the prior art can be solved.
In a first aspect, an embodiment of the present application provides a method for processing a log message, including:
acquiring a log message to be processed;
extracting a target log message of the log message to be processed;
and translating the target log message according to a preset rule.
In a possible implementation manner of the first aspect, the extracting a target log message of the to-be-processed log message includes:
detecting whether the log message to be processed corresponding to a preset mapping set exists or not;
and if so, taking the log message to be processed corresponding to the preset mapping set as the target log message.
In a possible implementation manner of the first aspect, the to-be-processed log message includes a message key and a message value corresponding to the message key, and the target log message includes a target attribute message key and a target attribute message value corresponding to the target attribute message key;
detecting whether a log message to be processed corresponding to a preset mapping set exists or not, wherein the detecting comprises the following steps:
detecting whether a message key matched with an element in the preset mapping set exists or not;
if so, taking the log message to be processed corresponding to the preset mapping set as the target log message, including:
if the message key matched with the element in the preset mapping set exists, taking the message key matched with the element in the preset mapping set as a target attribute message key;
taking the message value corresponding to the target attribute message key as a target attribute message value;
and forming the target log message according to the target attribute message key and the target attribute message value.
In a possible implementation manner of the first aspect, the preset rule includes a first preset rule and a second preset rule;
the translating the target log message according to the preset rule includes:
translating the target attribute message key according to the first preset rule;
and translating the target attribute message value according to the second preset rule.
In a possible implementation manner of the first aspect, translating the target property message key according to the first preset rule includes:
mapping the target attribute message key based on the matching relation between the target attribute message key and elements in a preset mapping group;
translating the target attribute message value according to the second preset rule, including:
and performing preset formatting treatment on the target attribute message value.
In a possible implementation manner of the first aspect, after the obtaining the log message to be processed, the method further includes:
and classifying the log messages to be processed to obtain the log types of the log messages to be processed.
In a possible implementation manner of the first aspect, classifying the log message to be processed to obtain a log type of the log message to be processed includes:
searching a characteristic log message and a value of the characteristic log message from the log message to be processed;
and marking the log type of the log message to be processed according to the characteristic log message and the value of the characteristic log message.
In a second aspect, an embodiment of the present application provides an apparatus for processing a log message, including:
the acquisition module is used for acquiring the log message to be processed;
the extraction module is used for extracting a target log message of the log message to be processed;
and the translation module is used for translating the target log message according to a preset rule.
In one possible implementation manner of the second aspect, the extraction module includes:
the detection submodule is used for detecting whether log messages to be processed corresponding to the preset mapping set exist or not;
and if so, taking the log message to be processed corresponding to the preset mapping set as the target log message.
In a possible implementation manner of the second aspect, the pending log message includes a message key and a message value corresponding to the message key, the target log message includes a target attribute message key and a target attribute message value corresponding to the target attribute message key, and the detection sub-module includes:
the detection unit is used for detecting whether a message key matched with an element in the preset mapping set exists or not;
the confirmation submodule includes:
a confirmation unit configured to take the message value corresponding to the target attribute message key as a target attribute message value;
and the forming unit is used for forming the target log message according to the target attribute message key and the target attribute message value.
In a possible implementation manner of the second aspect, the preset rule includes a first preset rule and a second preset rule, and the translation module includes:
the first translation unit is used for translating the target attribute message key according to the first preset rule;
and the second translation unit is used for translating the target attribute message value according to the second preset rule.
In one possible implementation manner of the second aspect, the first translation unit includes:
and the mapping subunit is used for mapping the target attribute message key based on the matching relationship between the target attribute message key and the elements in the preset mapping group.
The second translation unit includes:
and the formatting processing subunit is used for performing preset formatting processing on the target attribute message value.
In a possible implementation manner of the second aspect, the log message processing apparatus further includes:
and the classification module is used for classifying the log message to be processed to obtain the log type of the log message to be processed.
In one possible implementation manner of the second aspect, the classification module includes:
the searching unit is used for searching the characteristic log message and the value of the characteristic log message from the log message to be processed;
and the marking unit is used for marking the log type of the log message to be processed according to the feature log message and the value of the feature log message.
In a third aspect, an embodiment of the present application provides an internet of things platform, including: comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the method of processing log messages as described in any one of the above when executing the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method for processing log messages according to any one of the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on an internet of things platform, causes the internet of things platform to execute the method for processing log messages according to any one of the above first aspects.
It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.
Compared with the prior art, the embodiment of the application has the advantages that: according to the method and the device, the log message to be processed is obtained, the target log message of the log message to be processed is extracted, and the target log message is translated according to the preset rule, wherein the target log message can be the log message which needs to be attended by a user, the log message which needs to be attended is not required to be searched from the huge message volume of the log message by the user in a manual searching mode, and the log message which needs to be attended by the user is directly extracted and translated, so that the effect that the user can conveniently view the log message which needs to be attended is achieved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a method for processing a log message according to an embodiment of the present application;
fig. 2 is another schematic flow chart of a processing method of a log message according to a second embodiment of the present application;
fig. 3 is a schematic flowchart of a further method for processing a log message according to a third embodiment of the present application;
fig. 4 is a schematic structural diagram of a processing apparatus for log messages according to a fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of an internet of things platform provided in the embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
The technical solutions provided in the embodiments of the present application will be described below with specific embodiments.
Example one
Referring to fig. 1, a schematic flowchart of a processing method of a log message provided in an embodiment of the present application, by way of example and not limitation, the method may be applied to an internet of things platform, and the method may include the following steps:
and step S101, acquiring a log message to be processed.
The log message to be processed refers to a log message reported by the internet of things equipment;
the internet of things equipment comprises physical equipment such as a charging pile and the like but is not limited to the charging pile.
Specifically, the internet of things device actively reports the log message to be processed to the internet of things platform.
In a possible implementation manner, the to-be-processed log message reported by the internet of things device is sent to the database for temporary storage by the internet of things platform, and then the subsequent internet of things platform reads the temporary to-be-processed log message from the database, so that the effect of improving the data processing efficiency is achieved.
The database may be a database in the internet of things platform, for example, a time sequence database.
And S102, extracting a target log message of the log message to be processed.
The target log message refers to a log message that a user needs to pay attention to.
It can be understood that the internet of things platform can directly extract the target log message from the pending log message.
And step S103, translating the target log message according to a preset rule.
The preset rule may refer to a preset rule for translating the target log message, for example, mapping the target log message through a preset mapping set, and formatting the target log message through an enumeration value mapping manner.
The internet of things platform translates the target log message, so that the extracted target log message better conforms to the reading habit of the user, and the user can conveniently view the target log message, namely the log message which needs to be paid attention to by the user.
In the embodiment of the application, the internet of things platform extracts the target log message of the log message to be processed by acquiring the log message to be processed, and translates the target log message according to the preset rule, wherein the target log message can be the log message which needs to be attended by a user, the user does not need to search the log message which needs to be attended from the huge message volume of the log message in a manual searching mode, and the log message which needs to be attended by the user is directly extracted and translated, so that the user can conveniently view the log message which needs to be attended.
Example two
The purpose of this embodiment is to specifically describe the step of extracting the target log message of the log message to be processed and the step of translating the target log message according to the preset rule in the first embodiment.
Referring to fig. 2, another flow chart of a processing method of a log message provided in the second embodiment of the present application is shown, by way of example and not limitation, the method may be applied to an internet of things platform, and the method may include the following steps:
step S201, obtaining the log information to be processed.
It should be noted that step S201 is the same as step S102, and is not described herein again.
Step S202, whether log information to be processed corresponding to a preset mapping set exists is detected.
Step S203, if yes, taking the to-be-processed log message corresponding to the preset mapping set as a target log message.
The preset mapping set is a mapping set in which elements corresponding to the target log message are stored;
the target log message refers to a log message that a user needs to pay attention to.
It can be understood that, in the embodiment of the present application, the elements in the preset mapping set correspond to the log messages that the user needs to pay attention to in advance, so that the target log messages of the message logs to be processed can be detected through the preset mapping set in the following steps conveniently.
The log message to be processed comprises a message key and a message value corresponding to the message key;
the target log message includes a target attribute message key and a target attribute message value corresponding to the target attribute message key.
It can be understood that the encoding format of the log message to be processed in the embodiment of the present application may be a key-value pair encoding format, for example, a JSON encoding format, where the message key refers to a name of the message key-value pair, and the message value refers to a value of the message key-value pair;
then, the encoding format of the target log message in the embodiment of the present application is also a key-value pair encoding format, for example, a JSON encoding format, the target attribute message key refers to the name of the message key-value pair having the target attribute, and the target message value refers to the value of the message key-value pair having the target attribute.
It should be noted that the target attribute may refer to a leaf node attribute.
Specifically, the processes of step S203 and step S204 may be: and detecting whether a message key matched with an element in the preset mapping set exists or not, if so, taking the message key matched with the element in the preset mapping set as a target attribute message key, taking a message value corresponding to the target attribute message key as a target attribute message value, and forming a target log message according to the target attribute message key and the target attribute message value.
For example, the preset mapping set includes an element having a leaf node attribute message key a, the log message to be processed includes a message key a and a message value a corresponding to the message key a, if the message key a is matched with the leaf node attribute message key a of the mapping set a, the message key is used as a target attribute message key, the message value a is used as a target attribute message value, and a target log message is formed according to the target attribute message key and the target attribute message value.
And step S204, translating the target log message according to a preset rule.
The preset rule may refer to a preset rule for translating the target log message, for example, mapping a target attribute message key of the target log message through a preset mapping set, and formatting a target attribute message value of the target log message in an enumerated value mapping manner;
the internet of things platform translates the target log message, so that the extracted target log message better conforms to the reading habit of the user, and the user can conveniently view the target log message, namely the log message which needs to be paid attention to by the user.
Specifically, translating the target log message according to the preset rule may be: and translating the target attribute message key according to a first preset rule.
By way of example and not limitation, the mapping process is performed on the target attribute message key based on the matching relationship between the target attribute message key and the elements in the preset mapping group.
It can be understood that, in the embodiment of the present application, only the message key matched with the leaf attribute node message key in the preset mapping set may be used as the target attribute message key, and subsequently, the target attribute message key may also be mapped through the element of the mapping set corresponding to the leaf attribute node message key in the preset mapping set.
For example, if the acquired pending log message is:
wherein, the deviceId is a message key in a key value pair deviceId:1000, the deviceeventsignd is a message key path composed of a message key in a key value pair eventId:100 and a message key in a key value pair eventId:200, and the deviceeventsigne is a message key path composed of a message key in a key value pair time:1567157529 and a message key in a key value pair time: 1567157530;
accordingly, the deviceId message key represents a device identification code, the deviceeventid event identification code, and the deviceeventins time represents an event time.
It is understood that in a specific application, the pending log message includes a message key path composed of a message key, a message value, and a message key.
It should be noted that the attribute of the deviceId message key is a leaf node attribute, the attribute of each message key in the deviceevents.
If the preset mapping set comprises a mapping set A and a mapping set B corresponding to the mapping set A, wherein the mapping set A comprises an English deviceId message key with a target attribute being a leaf node attribute and an English time message key with a target attribute being a leaf node attribute, correspondingly, the mapping set B comprises a Chinese name equipment ID and a Chinese name time, wherein the English deviceId message key corresponds to the Chinese name equipment identification code, and the English time of the leaf node attribute corresponds to the Chinese name time.
As can be seen from the above, the deviceId message key matches with the english deviceId message key of the leaf node attribute of the mapping set a, the deviceevents.
It is understood that the deviceId message key is a deviceId target attribute message key, the message key in the deviceevents time message key path is a time target attribute message key, 1000 in the key-value pair deviceId:1000 is a target attribute message value, 1567157529 in the key-value pair time:1567157529 is a target attribute message value, and 1567157530 in the key-value pair time:1567157530 is a target attribute message value.
Then, the target log message formed by the target attribute message key and the target attribute message value corresponding to the target attribute message key is:
accordingly, since the deviceId message key matches with the english deviceId message key of the leaf node attribute of the mapping set a, the deviceevents time message key path matches with the english time message key, plus, the english deviceId message key corresponds to the device identifier code of the chinese name in the mapping set B, and the english time of the leaf node attribute corresponds to the time of the chinese name in the mapping set B. .
Then, the result of mapping the target attribute message key, that is, the target log message translated by the first preset rule for the target attribute message key, is:
{
the device identification code is 1000 a,
event time [1567157529,1567157530]
}
It should be noted that, because deviceEvents in the deviceevents.time message key path are object arrays, there is no mapping of deviceEvents object arrays in the preset mapping set, and only the mapping of the time message key in the deviceevents.time message key path, the subscript of the deviceEvents object array is transferred to the time message key, so that the data type of the time message key in the deviceevents.time message key path is changed from the original integer to an array.
Specifically, translating the target log message according to the preset rule may further be: and translating the target attribute message key according to a second preset rule.
By way of example and not limitation, the target attribute message value is pre-formatted.
The preset formatting process includes, but is not limited to, data shifting or scaling, enumerated value mapping, date and time formatting, and the like;
the data offset or scaling may refer to an adjustment in units of data values, e.g., units of weight converted from grams to kilograms, units of money converted from minutes to units, units of time converted from seconds to minutes, units of temperature converted from degrees fahrenheit to degrees celsius, etc.;
the enumerated value mapping may refer to mapping a binary number 0 to an operation success (0) or mapping a binary 1 to an operation failure (1);
the date-time formatting may refer to converting a time stamp into a formatted time.
For example: if the target log message translated by the first preset rule is:
{
event time [1567157529,1567157530]
}
Then, the result of the preset formatting process performed on the target attribute message values 1567157529 and 1567157530 in the event time [1567157529,1567157530], i.e. the result of the second preset rule process performed on the target attribute message values, is:
{
event time [ 2 ]
“2019-08-30 17:32:08”,
“2019-08-30 17:32:09”
]
}
The time and date 2019-08-3017: 32:08 corresponds to the target attribute message value 1567157529, and the time and date 2019-08-3017: 32:09 corresponds to the target attribute message value 1567157530.
It should be noted that the upper target attribute message value 1567157529 refers to a Unix timestamp, and may be formatted into a string, i.e., a time date 2019-08-3017: 32:08 through an enumerated value mapping;
the target attribute message value 1567157530 refers to a Unix timestamp, and may be formatted into a string of characters 2019-08-3017: 32:09 by way of enumerating a value map.
In some embodiments, after the step of translating the target log message according to the preset rule, the message structure of the target log message may be rearranged, so that the message structure of the target log message better conforms to the reading habit of the user.
For example, the target log message is:
{"deviceInfo":{"detail":{"temperatureInfo":{"currentTemperature":{"value":27,"unit":"centigrade"}}}}}
wherein, the effective content of the target log message is: the currentTemperature is 27 degrees celsius.
It can be seen that the message content of the target log message is very long, the number of nested layers is very large, but the effective content of the target log message is only currentTemperature of 27 ℃, and the target log message can be rearranged, so that the rearranged target log message is { "currentTemperature":27 }.
In general, the rearrangement refers to deleting or reducing the original message.
According to the method and the device, the target log message of the log message to be processed is extracted through the preset mapping set, then the first preset rule translation is carried out on the target attribute message key of the target log message, and the second preset rule translation is carried out on the target attribute message value of the target log message, so that the extracted target log message is translated, and the effect that a user can conveniently check the extracted target log message is achieved.
EXAMPLE III
The purpose of this embodiment is to classify the log type of the log message to be processed after the log message to be processed is acquired, so that a subsequent user can conveniently query and screen the log message in the log message to be processed according to the log type of the log message to be processed.
Referring to fig. 3, a further flowchart of a processing method for a log message provided in the third embodiment of the present application is shown, by way of example and not limitation, the method may be applied to an internet of things platform, and the method may include the following steps:
step S301, obtaining the log information to be processed.
It should be noted that step S301 is the same as step S101, and is not described herein again.
And step S304, classifying the log messages to be processed to obtain the log types of the log messages to be processed.
It should be noted that the encoding format of the log message to be processed in the embodiment of the present application may refer to a key-value pair encoding format, for example, a JSON encoding format.
The log message to be processed comprises a message key, a message value and a message key path formed by the message key.
Specifically, a characteristic log message and a value of the characteristic log message are searched from the log message to be processed, and the log type of the log message to be processed is marked according to the characteristic log message and the value of the characteristic log message.
The characteristic log message may refer to a message key path of the log message to be processed, for example, a content.msgid message key path, and correspondingly, the characteristic log message may refer to an attribute value of the message key path, for example, an attribute value of the content.msgid message key path;
the log types of the messages to be processed include, but are not limited to, a status reporting type, a device alarm type, an exception message type, and the like.
It can be understood that, in the embodiment of the present application, the internet of things platform configures and defines the feature log message and the log type corresponding to the value of the feature log message in advance, that is, configures and defines the message key path and the log type corresponding to the attribute value of the message key path in advance, and then, after searching the feature log message and the value of the feature log message from the message log to be processed, the log type of the log message to be processed may be marked by the feature log message and the value of the feature log message.
As an example and not by way of limitation, when a message key path is not configured and defined in advance, the corresponding log type is an abnormal message type;
presetting a defined message key path, wherein when the attribute value of the message key path is in a unique specified range, the corresponding log type is a state reporting type;
presetting and defining the existence of a message key path, and when the attribute value of the message key path is two selectable specified ranges, the corresponding log type is the equipment alarm type.
For example:
if the content.msgId message key path of the log message to be processed does not exist, marking the log type of the log message to be processed as an abnormal message type;
if a content.msgId message key path of the log message to be processed exists and the attribute value of the content.msgId message key path is in a unique specified range, such as a specified range {1}, marking the log type of the log message to be processed as a state reporting type;
and if the content.msgId message key path of the log message to be processed and the attribute value of the content.msgId message key path are in two selectable specified ranges, such as a specified range {8} or a specified range {9}, marking the log type of the log message to be processed as the device alarm type.
It should be noted that the internet of things platform in the embodiment of the present application is not limited to the above-mentioned configuration definition manner, but the information key path and the log type corresponding to the attribute value of the information key path are defined in advance by configuration.
In some possible implementations, the manner of searching the feature log message and the value of the feature log message from the log message to be processed may be: the content.msgId message key path of the log message to be processed, namely the characteristic log message, and the value of the content.msgId message key are searched in a Linux log in a mode of searching a keyword through a query command such as a tail command.
In some embodiments, after the step of classifying the log message to be processed to obtain the log type of the log message to be processed, a step of extracting a target log message of the log message to be processed and a step of translating the target log message according to a preset rule may be performed, and then the user performs manual screening or searching to confirm whether the target log message extracted from the log message to be processed is the log message that the user needs to pay attention to.
In some other embodiments, after the step of classifying the log message to be processed to obtain the log type of the log message to be processed, the user may further perform manual screening or search for the log message that needs to be paid attention to by the user in the log message to be processed.
In the embodiment of the application, the log types of the log messages to be processed are classified after the log messages to be processed are obtained, so that the effect of facilitating follow-up users to manually inquire and screen the log messages in the log messages to be processed according to the log types of the log messages to be processed is achieved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Example four
Fig. 4 shows a block diagram of a processing apparatus for a log message according to a fourth embodiment of the present application, which corresponds to the processing method for a log message described in the foregoing embodiments, and only shows portions related to the embodiments of the present application for convenience of description.
Referring to fig. 4, the apparatus includes:
an obtaining module 41, configured to obtain a log message to be processed;
an extracting module 42, configured to extract a target log message of the to-be-processed log message;
and the translation module 43 is configured to translate the target log message according to a preset rule.
In one possible implementation, the extraction module includes:
the detection submodule is used for detecting whether log messages to be processed corresponding to the preset mapping set exist or not;
and if so, taking the log message to be processed corresponding to the preset mapping set as the target log message.
In one possible implementation manner, the to-be-processed log message includes a message key and a message value corresponding to the message key, the target log message includes a target attribute message key and a target attribute message value corresponding to the target attribute message key, and the detection sub-module includes:
the detection unit is used for detecting whether a message key matched with an element in the preset mapping set exists or not;
the confirmation submodule includes:
a confirmation unit configured to take the message value corresponding to the target attribute message key as a target attribute message value;
and the forming unit is used for forming the target log message according to the target attribute message key and the target attribute message value.
In a possible implementation manner, the preset rules include a first preset rule and a second preset rule, and the translation module includes:
the first translation unit is used for translating the target attribute message key according to the first preset rule;
and the second translation unit is used for translating the target attribute message value according to the second preset rule.
In one possible implementation manner, the first translation unit includes:
and the mapping subunit is used for mapping the target attribute message key based on the matching relationship between the target attribute message key and the elements in the preset mapping group.
The second translation unit includes:
and the formatting processing subunit is used for performing preset formatting processing on the target attribute message value.
In one possible implementation manner, the log message processing apparatus further includes:
and the classification module is used for classifying the log message to be processed to obtain the log type of the log message to be processed.
In one possible implementation, the log types include a first log type, a second log type, and a third log type;
the classification module comprises:
the searching unit is used for searching the characteristic log message and the value of the characteristic log message from the log message to be processed;
and the marking unit is used for marking the log type of the log message to be processed according to the feature log message and the value of the feature log message.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
EXAMPLE five
Fig. 5 is a schematic structural diagram of an internet of things platform provided in the fifth embodiment of the present application. As shown in fig. 5, the internet of things platform 5 of this embodiment includes: at least one processor 50 (only one processor is shown in fig. 5), a memory 51, and a computer program 52 stored in the memory 51 and operable on the at least one processor 50, wherein the processor 50 implements the steps in any of the above-described embodiments of the method for processing log messages when executing the computer program 52.
The internet of things platform 5 may be a computing device such as a cloud server. The internet of things platform may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that fig. 5 is merely an example of the internet of things platform 5 and does not constitute a limitation on the internet of things platform 5.
The Processor 50 may be a Central Processing Unit (CPU), and the Processor 50 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 51 may be an internal storage unit of the internet of things platform 5 in some embodiments, for example, a hard disk or a memory of the internet of things platform 5.
It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing functional units and modules are merely divided into two parts, and in practical applications, the foregoing functional allocation may be performed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to perform all or part of the above described functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.
The embodiment of the application provides a computer program product, and when the computer program product runs on an internet of things platform, the steps in the method embodiments can be realized when the internet of things platform is executed.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to the photographing internet of things platform, recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, and software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method for processing log messages, comprising:
acquiring a log message to be processed;
extracting a target log message of the log message to be processed;
and translating the target log message according to a preset rule.
2. The method for processing the log message according to claim 1, wherein extracting the target log message of the log message to be processed comprises:
detecting whether the log message to be processed corresponding to a preset mapping set exists or not;
and if so, taking the log message to be processed corresponding to the preset mapping set as the target log message.
3. The method for processing the log message according to claim 2, wherein the log message to be processed includes a message key and a message value corresponding to the message key, and the target log message includes a target attribute message key and a target attribute message value corresponding to the target attribute message key;
detecting whether a log message to be processed corresponding to a preset mapping set exists or not, wherein the detecting comprises the following steps:
detecting whether a message key matched with an element in the preset mapping set exists or not;
if so, taking the log message to be processed corresponding to the preset mapping set as the target log message, including:
if the message key matched with the element in the preset mapping set exists, taking the message key matched with the element in the preset mapping set as a target attribute message key;
taking the message value corresponding to the target attribute message key as a target attribute message value;
and forming the target log message according to the target attribute message key and the target attribute message value.
4. The method for processing the log message according to claim 3, wherein the preset rule comprises a first preset rule and a second preset rule;
the translating the target log message according to the preset rule includes:
translating the target attribute message key according to the first preset rule;
and translating the target attribute message value according to the second preset rule.
5. The method for processing log messages according to claim 4, wherein translating the target attribute message key according to the first preset rule comprises:
mapping the target attribute message key based on the matching relation between the target attribute message key and elements in a preset mapping group;
translating the target attribute message value according to the second preset rule, including:
and performing preset formatting treatment on the target attribute message value.
6. The method for processing the log message according to any one of claims 1 to 5, wherein after acquiring the log message to be processed, the method further comprises:
and classifying the log messages to be processed to obtain the log types of the log messages to be processed.
7. The method for processing the log message according to claim 6, wherein the classifying the log message to be processed to obtain the log type of the log message to be processed comprises:
searching a characteristic log message and a value of the characteristic log message from the log message to be processed;
and marking the log type of the log message to be processed according to the characteristic log message and the value of the characteristic log message.
8. An apparatus for processing log messages, comprising:
the acquisition module is used for acquiring the log message to be processed;
the extraction module is used for extracting a target log message of the log message to be processed;
and the translation module is used for translating the target log message according to a preset rule.
9. An internet of things platform comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for processing log messages according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the log message processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911287325.1A CN111198850A (en) | 2019-12-14 | 2019-12-14 | Log message processing method and device and Internet of things platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911287325.1A CN111198850A (en) | 2019-12-14 | 2019-12-14 | Log message processing method and device and Internet of things platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111198850A true CN111198850A (en) | 2020-05-26 |
Family
ID=70746861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911287325.1A Pending CN111198850A (en) | 2019-12-14 | 2019-12-14 | Log message processing method and device and Internet of things platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111198850A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113282559A (en) * | 2021-06-04 | 2021-08-20 | 青岛海尔科技有限公司 | Computer log classification method and device, storage medium and electronic device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598825A (en) * | 2016-12-07 | 2017-04-26 | 泰康保险集团股份有限公司 | Electronic equipment, method and device for standardizing logging code output |
| US20170178026A1 (en) * | 2015-12-22 | 2017-06-22 | Sap Se | Log normalization in enterprise threat detection |
| CN107315756A (en) * | 2016-04-27 | 2017-11-03 | 中国移动通信集团安徽有限公司 | A kind of log processing method and device |
| CN109325009A (en) * | 2018-09-19 | 2019-02-12 | 亚信科技（成都）有限公司 | The method and device of log parsing |
| CN109542737A (en) * | 2018-09-29 | 2019-03-29 | 中国平安人寿保险股份有限公司 | Platform alert processing method, device, electronic device and storage medium |
| CN110309110A (en) * | 2019-05-24 | 2019-10-08 | 深圳壹账通智能科技有限公司 | A kind of big data log monitoring method and device, storage medium and computer equipment |
| CN110377531A (en) * | 2019-07-19 | 2019-10-25 | 清华大学 | Based on log-structured persistence memory storage engine apparatus and control method |
| US20210182453A1 (en) * | 2017-12-22 | 2021-06-17 | Hewlett-Packard Development Company, L.P. | Application behavior identification |
-
2019
- 2019-12-14 CN CN201911287325.1A patent/CN111198850A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170178026A1 (en) * | 2015-12-22 | 2017-06-22 | Sap Se | Log normalization in enterprise threat detection |
| CN107315756A (en) * | 2016-04-27 | 2017-11-03 | 中国移动通信集团安徽有限公司 | A kind of log processing method and device |
| CN106598825A (en) * | 2016-12-07 | 2017-04-26 | 泰康保险集团股份有限公司 | Electronic equipment, method and device for standardizing logging code output |
| US20210182453A1 (en) * | 2017-12-22 | 2021-06-17 | Hewlett-Packard Development Company, L.P. | Application behavior identification |
| CN109325009A (en) * | 2018-09-19 | 2019-02-12 | 亚信科技（成都）有限公司 | The method and device of log parsing |
| CN109542737A (en) * | 2018-09-29 | 2019-03-29 | 中国平安人寿保险股份有限公司 | Platform alert processing method, device, electronic device and storage medium |
| CN110309110A (en) * | 2019-05-24 | 2019-10-08 | 深圳壹账通智能科技有限公司 | A kind of big data log monitoring method and device, storage medium and computer equipment |
| CN110377531A (en) * | 2019-07-19 | 2019-10-25 | 清华大学 | Based on log-structured persistence memory storage engine apparatus and control method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113282559A (en) * | 2021-06-04 | 2021-08-20 | 青岛海尔科技有限公司 | Computer log classification method and device, storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10216724B2 (en) | Performing semantic analyses of user-generated textual and voice content | |
| CN108737423B (en) | Phishing website discovery method and system based on webpage key content similarity analysis | |
| US10474818B1 (en) | Methods and devices for detection of malware | |
| CN109739867B (en) | Industrial metadata management method and system | |
| CN109635120B (en) | Knowledge graph construction method and device and storage medium | |
| US20220197923A1 (en) | Apparatus and method for building big data on unstructured cyber threat information and method for analyzing unstructured cyber threat information | |
| CN112148889A (en) | Recommendation list generation method and device | |
| US11775767B1 (en) | Systems and methods for automated iterative population of responses using artificial intelligence | |
| WO2015084476A1 (en) | Non-standard and standard clause detection | |
| CN105634855A (en) | Method and device for recognizing network address abnormity | |
| CN108491715B (en) | Terminal fingerprint database generation method and device and server | |
| CN113836128A (en) | Abnormal data identification method, system, equipment and storage medium | |
| CN112307318A (en) | Content publishing method, system and device | |
| CN111198850A (en) | Log message processing method and device and Internet of things platform | |
| CN115051863B (en) | Abnormal flow detection method and device, electronic equipment and readable storage medium | |
| US20230252140A1 (en) | Methods and systems for identifying anomalous computer events to detect security incidents | |
| CN105512270B (en) | Method and device for determining related objects | |
| CN115357286B (en) | Program file comparison method and device, electronic equipment and storage medium | |
| CN116842099A (en) | Multi-source heterogeneous data processing method and system | |
| US11681966B2 (en) | Systems and methods for enhanced risk identification based on textual analysis | |
| US11657078B2 (en) | Automatic identification of document sections to generate a searchable data structure | |
| CN113688240B (en) | Threat element extraction method, threat element extraction device, threat element extraction equipment and storage medium | |
| CN115051859A (en) | Information analysis method, information analysis device, electronic apparatus, and medium | |
| CN114490246A (en) | Monitoring method, monitoring device, electronic equipment and storage medium | |
| CN111143318B (en) | Information processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200526 |