CN113420003A - Method, device, equipment and medium for processing data interaction log - Google Patents
Method, device, equipment and medium for processing data interaction log Download PDFInfo
- Publication number
- CN113420003A CN113420003A CN202110736311.4A CN202110736311A CN113420003A CN 113420003 A CN113420003 A CN 113420003A CN 202110736311 A CN202110736311 A CN 202110736311A CN 113420003 A CN113420003 A CN 113420003A
- Authority
- CN
- China
- Prior art keywords
- log
- data interaction
- data
- communication
- party
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 260
- 238000012545 processing Methods 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004891 communication Methods 0.000 claims abstract description 158
- 230000006399 behavior Effects 0.000 claims abstract description 20
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 238000005192 partition Methods 0.000 claims description 51
- 238000003672 processing method Methods 0.000 claims description 14
- 230000002452 interceptive effect Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 238000007726 management method Methods 0.000 abstract description 12
- 238000013024 troubleshooting Methods 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/164—File meta data generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application provides a method, a device, equipment and a medium for processing a data interaction log, wherein the method comprises the following steps: acquiring a data interaction log for recording data interaction behaviors with a data interaction party; determining a communication keyword of the data interaction party; determining a log file path for storing the data interaction log according to the communication keyword or the communication keyword and log generation time, wherein the log file path indicates a storage path of a log file for storing the data interaction log; detecting whether the log file exists; and storing the data interaction log based on the detection result. The data interaction log is classified and managed according to the communication keywords and the log generation time, so that the data interaction log is stored in different storage paths, and the data interaction log management method is particularly suitable for recording and managing the data interaction log by interfaces or software containing a communication module.
Description
Technical Field
The present application relates to the technical field of log processing, and in particular, to a method, an apparatus, a device, and a medium for processing a data interaction log.
Background
The software log management is one of the necessary functions of a set of software system, and is an indispensable means for ensuring the smooth operation of the system and troubleshooting system faults. The software log management mainly records various prompt messages, alarm messages and error messages in the running process of the software system.
The software log management can be designed and developed by enterprises/individuals according to business requirements and characteristics, and can also adopt internal or external mature technologies of the enterprises, such as log4j commonly used in java development. Existing log management models typically focus on logging system operating conditions and critical business processes and log logging/presentation under a unified or limited number of log files/consoles. Therefore, logs with large data interaction amount, wide sources and different contents are often stacked together and are difficult to distinguish, and potential problems cannot be found through the logs; meanwhile, when a system fails or has abnormal services, the data logs are mixed and large in quantity, so that the troubleshooting difficulty is high.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, a device, and a medium for processing a data interaction log, which solve the problems of mixed data logs, large quantity, and large difficulty in troubleshooting in the prior art by classifying and storing the logs used for recording data interaction behaviors.
In a first aspect, an embodiment of the present application provides a method for processing a data interaction log, where the method for processing the data interaction log includes: acquiring a data interaction log for recording data interaction behaviors with a data interaction party; determining a communication keyword of the data interaction party; determining a log file path for storing the data interaction log according to the communication keyword or the communication keyword and log generation time, wherein the log file path indicates a storage path of a log file for storing the data interaction log; detecting whether the log file exists; and storing the data interaction log based on the detection result.
Optionally, the data interaction party comprises a data sending party, and the data interaction behavior comprises receiving a message to be processed from the data sending party; wherein, the step of determining the communication keyword of the data interaction party comprises the following steps: analyzing the message to be processed to obtain the analyzed message to be processed; and extracting the communication keywords of the data sender from the analyzed message to be processed.
Optionally, the data interaction party comprises a data receiving party, and the data interaction behavior comprises sending data to the data receiving party; wherein, the step of determining the communication keyword of the data interaction party further comprises: and extracting the communication keywords of the data receiver from the data.
Optionally, the processing method of the data interaction log further includes: determining an interaction type for performing data interaction with a data interaction party; wherein a log file path for storing the data interaction log is determined utilizing any one of: the communication keywords; the communication keywords and the log generation time; the communication keywords and the interaction types; the communication keywords, the log generation time and the interaction type.
Optionally, the log file path comprises any one of: under the files or folders which are named by the communication keywords and are preset in the hard disk partition; in a folder of a preset hard disk partition named by the communication keyword, a file or a folder named by a preset time period in the log generation time; files named by the interactive type or folders in folders named by the communication keywords in a preset hard disk partition; and in the folders named by the communication keywords in the preset hard disk partition, the folders named by the preset time period in the log generation time are named as files or folders named by the interactive type.
Optionally, the step of storing the data interaction log based on the detection result includes: if the log file is detected to exist, writing the data interaction log into the log file; and if the log file is detected to be absent, creating the log file under the storage path, and writing the data interaction log into the created log file.
Optionally, after the data interaction log is stored based on the detection result, the processing method of the data interaction log further includes: determining log generation time of all data interaction logs in the log file; and deleting the data interaction log with the time difference between the log generation time and the current time being greater than the set time threshold.
In a second aspect, an embodiment of the present application further provides a device for processing a data interaction log, where the device includes:
the data acquisition module is used for acquiring a data interaction log for recording data interaction behaviors with a data interaction party;
the keyword extraction module is used for determining the communication keywords of the data interaction party;
the path calculation module is used for determining a log file path for storing the data interaction log according to the communication keyword or the communication keyword and the log generation time, wherein the log file path indicates a storage path of a log file for storing the data interaction log;
the detection module is used for detecting whether the log file exists or not;
and the storage module is used for storing the data interaction log based on the detection result.
In a third aspect, an embodiment of the present application further provides an electronic device, including: the data interaction log processing device comprises a processor, a memory and a bus, wherein the memory stores machine readable instructions executable by the processor, when the electronic device runs, the processor and the memory are communicated through the bus, and the processor executes the machine readable instructions to execute the steps of the data interaction log processing method.
In a fourth aspect, the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the processing method for the data interaction log.
The data interaction log processing method, device, equipment and medium provided by the embodiment of the application carry out classification management on the data interaction log according to the communication keywords and the log generation time so as to store the data interaction log in different storage paths, are particularly suitable for recording and managing the data interaction log by interfaces or software containing a communication module, and can effectively protect the smooth operation of a system and rapidly troubleshoot system faults.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a flowchart of a method for processing a data interaction log according to an embodiment of the present application;
fig. 2 is a flowchart of another data interaction log processing method according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a data interaction log processing apparatus according to an embodiment of the present application;
fig. 4 is a block diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. Every other embodiment that can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present application falls within the protection scope of the present application.
First, an application scenario to which the present application is applicable will be described. The application can be applied to software log management, and is particularly suitable for recording and managing data interaction contents by interface type or software comprising a communication module.
Research shows that the existing log management model generally focuses on recording the system running state and key business processing, records/displays logs under a unified or limited log file/console, and has no special targeted design for data interaction logs. Therefore, logs with large data interaction amount, wide sources and different contents are often stacked together and are difficult to distinguish, and potential problems cannot be found through the logs; meanwhile, when a system fails or has abnormal services, the data interaction logs are mixed and large in quantity, so that the troubleshooting difficulty is high.
Based on this, the embodiment of the application provides a method for processing a data interaction log, which can classify and store the data interaction log so as to ensure that a system runs smoothly and quickly troubleshoots system faults.
Referring to fig. 1, fig. 1 is a flowchart of a data interaction log processing method according to an embodiment of the present disclosure. As shown in fig. 1, a method for processing a data interaction log provided in an embodiment of the present application includes:
s101, acquiring a data interaction log for recording data interaction behaviors with a data interaction party.
In this embodiment, the method for processing the data interaction log may be applied to a communication service platform, and the data interaction party may refer to a plurality of communication nodes communicating with the communication service platform. In an example, the processing method can be applied to an airplane refueling scene, in which case, the communication service platform may refer to a total aviation oil service platform, and the data interaction party may refer to a plurality of airports distributed in different regions.
The data interaction log may refer to software operation information used for recording software operation procedures executed in the communication service platform for performing data interaction with each communication node, and the software operation information may include, but is not limited to, at least one of the following items by way of example: software prompt information, software alarm information, software operation error information and interaction information with the communication node.
For example, the type of interaction with the data interaction party may include data reception and data transmission.
For the data receiving situation, the data interaction party refers to a data sending party, and the data interaction behavior refers to the data receiving from the data sending party.
That is, the data sender sends data to the communication service platform, that is, the communication service platform receives data from the data sender, and the log generated by the communication service platform refers to a receiving log for recording the behavior of receiving data from the data sender.
For example, taking the communication service platform as the main aviation service platform, and taking the data sender as the "beijing capital international airport" as an example, after the "beijing capital international airport" completes the refueling work on a certain aircraft, the "beijing capital international airport" may send refueling data (such as refueling time, refueling amount, etc.) for the certain aircraft to the main aviation service platform (also called as main aviation oil headquarters data center), and at this time, the main aviation service platform generates a receiving log for recording data received from the beijing capital international airport.
Aiming at the data sending condition, the data interaction party refers to the data receiving party, and the data interaction behavior points to the data receiving party to send data.
That is to say, the communication service platform sends data to the data receiver, that is, the communication service platform sends data to the data receiver, and the log generated by the communication service platform refers to a sending log for recording a data sending behavior to the data receiver.
For example, taking the communication service platform as a general aviation service platform, taking the data receiver as an example of a "beijing capital international airport", the general aviation service platform sends fueling data to the "beijing capital international airport", and at this time, the general aviation service platform generates a sending log for recording data sending to the beijing capital international airport.
And S102, determining the communication keywords of the data interaction party.
By way of example, the communication keywords may include, but are not limited to: the identity (i.e. the ID of the correspondent node) or the correspondent address of the data interaction party. It should be understood that the present application is not limited thereto, and one or more keywords in a certain feature word may also be used as communication keywords in a customized manner.
Aiming at the condition that the data interaction party is the data sender, the message to be processed can be received from the data sender, the message to be processed is analyzed to obtain the analyzed message to be processed, and the communication keywords of the data sender are extracted from the analyzed message to be processed.
In the case that the data interaction party is the data receiving party, the communication keyword of the data receiving party can be extracted from the data sent to the data receiving party.
S103, determining a log file path for storing the data interaction log according to the communication key word or the communication key word and the log generation time, wherein the log file path indicates a storage path of a log file for storing the data interaction log.
The processing method may further include: and determining the interaction type of data interaction with the data interaction party, wherein in this case, the data interaction logs can be classified and stored based on the communication keywords, the log generation time and the interaction type.
In the application, a storage rule for the data interaction log may be predefined, for example, a hard disk partition for storing the data interaction log and a precedence order of each level of the storage directory may be specified in advance. The log file path in step S103 refers to a storage path calculated based on a predefined storage rule.
In the first embodiment, a log file path for storing a data interaction log may be determined according to the communication keyword.
At this time, the log file path may refer to a file or a folder named by a communication keyword in a preset hard disk partition. For example, a required communication keyword may also be searched from a preset field of the data interaction log to determine a log file path for storing the data interaction log.
For example, taking the data interaction party as "beijing capital international airport" as an example, the communication keyword is the capital international airport, the preset hard disk partition is f disks, and at this time, the determined log file path for storing the data interaction log may be: f \\ capital international airport.
In the above embodiment, the storage directory of each level is ordered as: hard disk partitions, files or folders named by communication keywords.
In the second embodiment, the log file path for storing the data interaction log may be determined according to the communication keyword and the log generation time.
At this time, the log file path may refer to a file or a folder named after a preset time period in the log generation time among folders named after the communication keyword of the preset hard disk partition. For example, the log file path for storing the data interaction log may also be determined by searching for the required communication keyword and the generation time from a preset field of the data interaction log.
As an example, the preset time period may include, but is not limited to: hour, day, week, month, quarter, year.
For example, taking a data interaction party as an example of a "beijing capital international airport", a communication keyword is the capital international airport, a preset hard disk partition is an f disk, the generation time of the data interaction log is 2021 year, 5 month, 10 days, 12:30, and if a preset time period is day, the determined log file path for storing the data interaction log is as follows: f \ capital international airport/2021 year 5 month 10. If the preset time period is month, the determined log file path for storing the data interaction log is as follows: f, 5 months in 2021, if the preset time period is small, the determined log file path for storing the data interaction log is as follows: f \ capital international airport/2021 year 5 month 10 day \12:00-13:00, where 12:00-13:00 refers to a time period divided by hours.
In the above embodiment, the storage directory of each level is ordered as: hard disk partitions, folders named by communication keywords, files or folders named by preset time periods in log generation time.
In the third embodiment, the log file path for storing the data interaction log can be determined according to the communication keyword and the interaction type.
At this time, the log file path may refer to a file named by the interactive type or a folder in a folder named by a communication keyword of a preset hard disk partition. For example, a log file path for storing the data interaction log may also be determined by searching a preset field of the data interaction log for a required communication keyword and interaction type. By way of example, the interaction type may include data reception and data transmission.
For example, taking the data interaction party as "beijing capital international airport" as an example, the communication keyword is the capital international airport, the preset hard disk partition is f disks, and if the interaction type is data reception, at this time, the data interaction party is a data sender, the determined log file path for storing the data interaction log is: f \\ capital international airport \ data reception. If the interaction type is data sending, at this time, the data interaction party is a data receiving party, and the determined log file path for storing the data interaction log is as follows: f \\ \ capital international airport \ data transmission.
In the above embodiment, the storage directory of each level is ordered as: hard disk partitions, folders named with communication keywords, files or folders named with interactive types.
In the fourth embodiment, the log file path for storing the data interaction log may be determined according to the communication keyword, the log generation time, and the interaction type.
At this time, the log file path may refer to a file named by the interactive type or below a folder named by a preset time period in the log generation time among folders named by the communication keyword in the preset hard disk partition. For example, a log file path for storing the data interaction log may be determined by searching for a required communication keyword, generation time, and interaction type from a preset field of the data interaction log.
For example, taking a data interaction party as an example of a "beijing capital international airport", a communication keyword is the capital international airport, a preset hard disk partition is an f disk, the generation time of the data interaction log is 2021 year, 5 month, 10 days, 12:30, and if the preset time period is day and the interaction type is data reception, the determined log file path for storing the data interaction log is as follows: f \ capital international airport \2021 year 5 month 10 day \ data reception. If the preset time period is month and the interaction type is data transmission, the determined log file path for storing the data interaction log is as follows: f \ capital international airport \2021 year 5 month \ data transmission.
In the above embodiment, the storage directory of each level is ordered as: hard disk partitions, folders named with communication keywords, folders named with a preset time period in log generation time, and files or folders named with interactive types. In addition, the following sequence is also possible: hard disk partitions, folders named with communication keywords, folders named with interaction types, files or folders named with a preset time period in log generation time.
S104, detecting whether the log file exists or not.
In this step, after the storage path of the log file is calculated, it is possible to detect whether or not the corresponding log file exists under the storage path. By way of example, the log file may refer to a file or folder at the last level of the storage path.
However, the present application is not limited thereto, and in a preferred embodiment, the processing method of the present application may further include: and detecting each level of storage directory in the determined log file path.
For example, in the case of the first embodiment, whether a preset hard disk partition exists may be detected first, if the preset hard disk partition exists, whether a file or a folder named by a communication keyword exists in the preset hard disk partition may be continuously detected, if the file or the folder named by the communication keyword exists, it is determined that a log file exists, and if the file or the folder named by the communication keyword does not exist, it is determined that the log file does not exist, and at this time, the file or the folder named by the communication keyword may be created under the preset hard disk partition.
And if the preset hard disk partition does not exist, re-designating a hard disk partition, and creating a file or a folder named by the communication keywords under the re-designated hard disk partition.
In the second embodiment, whether there is a preset hard disk partition may be detected, and if there is a preset hard disk partition, then it is continuously detected whether there is a folder named by the communication keyword under the preset hard disk partition, if the folder named by the communication key exists, whether the file or the folder named by the preset time period in the log generation time exists or not is continuously detected under the folder named by the communication key, if it is determined that there is a file or folder named with a preset time period in the log generation time, it is determined that there is a log file, if there is no file or folder named with a preset time period in the log generation time, it is determined that there is no log file, and at this time, a file or folder named with a preset time period in the log generation time may be created under the folder named with the communication keyword.
If the folder named by the communication keyword does not exist, the folder named by the communication keyword can be created under the preset hard disk partition, and the file or the folder named by the preset time period in the log generation time can be created under the folder named by the communication keyword. And if the preset hard disk partition does not exist, re-designating a hard disk partition, creating a folder named by the communication key words under the re-designated hard disk partition, and creating a file or a folder named by a preset time period in the log generation time under the folder named by the communication key words.
For the third embodiment, whether a preset hard disk partition exists may be detected first, if the preset hard disk partition exists, whether a folder named by a communication keyword exists under the preset hard disk partition is detected continuously, if the folder named by the communication keyword exists, whether a file or a folder named by an interaction type exists under the folder named by the communication keyword is detected continuously, if it is determined that the file or the folder named by the interaction type exists, it is determined that a log file exists, if the file or the folder named by the interaction type does not exist, it is determined that the log file does not exist, and at this time, the file or the folder named by the interaction type may be created under the folder named by the communication keyword.
If the folder named by the communication keyword does not exist, the folder named by the communication keyword can be created under the preset hard disk partition, and the file or the folder named by the interactive type can be created under the folder named by the communication keyword. And if the preset hard disk partition does not exist, re-designating a hard disk partition, creating a folder named by the communication key words under the re-designated hard disk partition, and creating a file or a folder named by the interaction type under the folder named by the communication key words.
In view of the situation of the fourth embodiment, it may be detected whether there is a preset hard disk partition, if there is a preset hard disk partition, it is continuously detected whether there is a folder named by the communication keyword under the preset hard disk partition, if there is a folder named by the communication keyword, it is continuously detected whether there is a file or folder named by a preset time period in the log generation time under the folder named by the communication keyword, if it is determined that there is a folder named by a preset time period in the log generation time, it is continuously detected whether there is a file or folder named by the interaction type under the folder named by the preset time period in the log generation time, if there is a file or folder named by the interaction type, it is determined that there is a log file, if there is no file or folder named by the interaction type, it is determined that there is no log file, and at this time, a file or folder named with an interactive type may be created under a folder named with a preset time period in the log generation time.
If the folder named by the communication keyword does not exist, the folder named by the communication keyword can be created under the preset hard disk partition, the folder named by the preset time period in the log generation time is created under the folder named by the communication keyword, and the file or the folder named by the interactive type is created under the folder named by the preset time period in the log generation time.
If there is no folder named with the preset time period in the log generation time, a folder named with the preset time period in the log generation time may be created under the folder named with the communication keyword, and a file or folder named with the interactive type may be created under the folder named with the preset time period in the log generation time.
And if the preset hard disk partition does not exist, re-designating a hard disk partition, creating a folder named by the communication key words under the re-designated hard disk partition, creating a folder named by the preset time period in the log generation time under the folder named by the communication key words, and creating a file or a folder named by the interactive type under the folder named by the preset time period in the log generation time.
And S105, storing the data interaction log based on the detection result.
In this step, if the presence of the log file is detected, the data interaction log is written into the log file. For example, the data interaction log may be written as a record entry in a file at the last level of the storage path, or the data interaction log may be written as a log file in a folder at the last level of the storage path.
And if the log file is detected to be not existed, creating the log file under the storage path, or creating the storage path in the above way to write the data interaction log into the created log file.
The method for processing the data interaction log solves the problems of mixed data logs, large quantity and high troubleshooting difficulty in the prior art, and achieves the effects of protecting smooth operation of a system and rapidly troubleshooting system faults.
Compared with the log management method in the prior art, the log management method aims at the problems of large data interaction amount, wide sources and different contents of the data interaction logs, and provides the data interaction log processing method which can classify and store the logs based on the communication keywords so as to ensure that the system runs smoothly and quickly troubleshoots system faults.
Referring to fig. 2, fig. 2 is a flowchart of a method for processing a data interaction log according to another embodiment of the present application. The following describes a method for processing a data interaction log in some embodiments of the present application in detail with reference to fig. 2.
In an example, as shown in the flow at the left side of fig. 2, a method for processing a data interaction log when the data interaction log is a received log is introduced, which specifically includes:
s201, receiving a message to be processed.
In the step, the communication service platform receives a message to be processed sent by a data sender.
And S202, analyzing the message to be processed.
In this step, the communication service platform resolves the message to be processed into a field that the communication service platform can recognize.
S203, extracting the communication keywords.
In this step, the log processing system of the communication service platform extracts the communication keyword from the analyzed message to be processed. By way of example, the communication keywords may include, but are not limited to: and the identity or the communication address of the data interaction party.
And S204, calculating a log file storage path.
S205, checking whether a log file exists.
The descriptions of S204 to S205 may refer to the descriptions of S103 to S104, and the same technical effects can be achieved, which are not described in detail.
S206, judging whether the log file exists or not. If so, then S208 is performed, otherwise, S207 is performed.
S207, creating a log file.
In this step, when it is determined in S206 that a log file does not exist, a corresponding log file is created. For example, the corresponding log file or storage path may be created in the manner described in step S104 above.
And S208, writing the receiving log.
In this step, the reception log is written in a log file.
Preferably, while writing in the log file, it is detected whether a log exceeding a preset time threshold exists in the log file where the log file is located (i.e., a data interaction log whose time difference between the log generation time and the current time is greater than a set time threshold), if the log exceeds the preset time threshold, the data interaction log exceeding the preset time threshold is deleted, and if the log does not exist, the content in the log file where the log file is located is not modified.
It should be understood that, in addition to performing the above detection while writing the log file, each log file may be detected in a predetermined cycle, or each level of the storage directory below the hard disk partition may be detected in a predetermined cycle, so that the deletion operation is performed for the file or folder below each level of the storage directory.
In a preferred example, as shown in the right side of fig. 2, a method for processing a data interaction log when the data interaction log is a send log is introduced, which specifically includes:
s210, acquiring data to be transmitted.
In this step, the communication service platform sends data to the data receiver, and the log processing system of the communication service platform acquires the data to be sent.
S211, extracting the communication keywords.
In this step, the log processing system of the communication service platform extracts a communication keyword from data transmitted to the data receiving side.
S212, calculating a log file storage path.
S213, checking whether the log file exists.
The descriptions of S212 to S213 may refer to the descriptions of S103 to S104, and the same technical effects can be achieved, which are not described in detail.
S214, judging whether the log file exists or not, if so, executing S216, otherwise, executing S215.
S215, creating a log file.
In this step, when it is determined in S214 that a log file does not exist, a corresponding log file is created. For example, the corresponding log file or storage path may be created in the manner described in step S104 above.
S216, writing the sending log.
This step is writing the transmission log in a log file.
The method for processing the data interaction log solves the problems of mixed data logs, large quantity and high troubleshooting difficulty in the prior art, and achieves the effects of protecting smooth operation of a system and rapidly troubleshooting system faults.
Compared with the log management method in the prior art, the method for processing the data interaction log aims at the problems of large data interaction amount, wide sources and different contents of the data interaction log, and is used for ensuring smooth operation of a system and rapidly troubleshooting system faults.
Based on the same inventive concept, the embodiment of the present application further provides a device for processing a data interaction log corresponding to the method for processing a data interaction log, and as the principle of solving the problem of the device in the embodiment of the present application is similar to the method for processing the data interaction log in the embodiment of the present application, the method can be referred to for implementation of the device, and repeated details are not repeated.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a data interaction log processing device according to an embodiment of the present application. As shown in fig. 3, the processing device 300 for the data interaction log comprises:
the data obtaining module 301 is configured to obtain a data interaction log for recording a data interaction behavior with a data interaction party.
In this embodiment, the method for processing the data interaction log may be applied to a communication service platform, and the data interaction party may refer to a plurality of communication nodes communicating with the communication service platform.
For example, the type of interaction with the data interaction party may include data reception and data transmission.
For the data receiving situation, the data interaction party refers to a data sending party, and the data interaction behavior refers to the data receiving from the data sending party.
In this case, the data sender sends data to the communication service platform, that is, the communication service platform receives data from the data sender, and the data interaction log generated by the communication service platform refers to a receiving log for recording the behavior of receiving data from the data sender.
Aiming at the data sending condition, the data interaction party refers to the data receiving party, and the data interaction behavior points to the data receiving party to send data.
In this case, the communication service platform sends data to the data receiver, that is, the communication service platform sends data to the data receiver, and the data interaction log generated by the communication service platform refers to a sending log for recording a data sending behavior to the data receiver.
And the keyword extraction module 302 is used for determining the communication keywords of the data interaction party.
By way of example, the communication keywords may include, but are not limited to: and the identity or the communication address of the data interaction party.
For the case that the data interaction party is the data sender, the keyword extraction module 302 may receive the message to be processed from the data sender, analyze the message to be processed to obtain an analyzed message to be processed, and extract the communication keywords of the data sender from the analyzed message to be processed.
For the above-mentioned case that the data interaction party is the data receiving party, the keyword extraction module 302 may extract the communication keyword of the data receiving party from the data sent to the data receiving party.
The path calculation module 303 is configured to determine a log file path for storing the data interaction log according to the communication keyword or the communication keyword and the log generation time. Here, the log file path indicates a storage path of the log file for storing the data interaction log.
Optionally, the path calculation module 303 may be further configured to: and determining the interaction type of data interaction with the data interaction party, wherein in this case, the log file path for storing the data interaction log can be determined based on the communication keywords, the log generation time and the interaction type.
In the first embodiment, the path calculation module 303 may determine a log file path for storing a data interaction log according to the communication keyword.
In the second embodiment, the path calculation module 303 may determine a log file path for storing the data interaction log according to the communication keyword and the log generation time.
In the third embodiment, the path calculation module 303 may determine a log file path for storing a data interaction log according to the communication keyword and the interaction type.
In the fourth embodiment, the path calculation module 303 may determine a log file path for storing a data interaction log according to the communication keyword, the log generation time, and the interaction type.
The detecting module 304 is configured to detect whether a log file exists.
After calculating the storage path of the log file, the detection module 304 may detect whether the corresponding log file exists under the storage path. By way of example, the log file may refer to a file or folder at the last level of the storage path.
The storage module 305 is configured to store the data interaction log based on the detection result.
If the log file is detected to exist, the storage module 305 writes the data interaction log into the log file. For example, the data interaction log may be written as a record entry in a file at the last level of the storage path, or the data interaction log may be written as a log file in a folder at the last level of the storage path.
If it is detected that no log file exists, a log file is created under the storage path, or the storage path is created in the above manner, and the storage module 305 writes the data interaction log into the created log file.
The processing device for the data interaction log solves the problems of mixed data logs, large quantity and high troubleshooting difficulty in the prior art, and achieves the effects of protecting smooth operation of a system and rapidly troubleshooting system faults.
Compared with the log management device in the prior art, the processing device of the data interaction log provided by the embodiment of the application aims at the problems of large data interaction amount, wide source and different contents of the data interaction log, and provides a processing method of the data interaction log so as to ensure that a system runs smoothly and quickly troubleshoots system faults.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 4, the electronic device 400 includes a processor 410, a memory 420, an external device 430, and a bus 440.
The memory 420 stores machine-readable instructions executable by the processor 410, when the electronic device 400 runs, the processor 410, the memory 420 and the external device 430 communicate with each other through the bus 440, and when the machine-readable instructions are executed by the processor 410, the steps of the method for processing the data interaction log in the method embodiments shown in fig. 1 and fig. 2 may be executed.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for processing a data interaction log in the method embodiments shown in fig. 1 and fig. 2 may be executed.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A processing method of a data interaction log is characterized in that the processing method of the data interaction log comprises the following steps:
acquiring a data interaction log for recording data interaction behaviors with a data interaction party;
determining a communication keyword of the data interaction party;
determining a log file path for storing the data interaction log according to the communication keyword or the communication keyword and log generation time, wherein the log file path indicates a storage path of a log file for storing the data interaction log;
detecting whether the log file exists;
and storing the data interaction log based on the detection result.
2. The method for processing the data interaction log of claim 1, wherein the data interaction party comprises a data sending party, and the data interaction behavior comprises receiving a message to be processed from the data sending party;
wherein, the step of determining the communication keyword of the data interaction party comprises the following steps:
analyzing the message to be processed to obtain the analyzed message to be processed;
and extracting the communication keywords of the data sender from the analyzed message to be processed.
3. The method for processing the data interaction log of claim 1, wherein the data interaction party comprises a data receiving party, and the data interaction behavior comprises sending data to the data receiving party;
wherein, the step of determining the communication keyword of the data interaction party further comprises:
and extracting the communication keywords of the data receiver from the data.
4. The method for processing the data interaction log of claim 1, wherein the method for processing the data interaction log further comprises: determining an interaction type for performing data interaction with a data interaction party;
wherein a log file path for storing the data interaction log is determined utilizing any one of:
the communication keywords;
the communication keywords and the log generation time;
the communication keywords and the interaction types;
the communication keywords, the log generation time and the interaction type.
5. The method for processing the data interaction log of claim 4, wherein the log file path comprises any one of the following items:
under the files or folders which are named by the communication keywords and are preset in the hard disk partition;
in a folder of a preset hard disk partition named by the communication keyword, a file or a folder named by a preset time period in the log generation time;
files named by the interactive type or folders in folders named by the communication keywords in a preset hard disk partition;
and in the folders named by the communication keywords in the preset hard disk partition, the folders named by the preset time period in the log generation time are named as files or folders named by the interactive type.
6. The method for processing the data interaction log according to claim 1, wherein the step of storing the data interaction log based on the detection result comprises:
if the log file is detected to exist, writing the data interaction log into the log file;
and if the log file is detected to be absent, creating the log file under the storage path, and writing the data interaction log into the created log file.
7. The method for processing the data interaction log according to claim 1, wherein after storing the data interaction log based on the detection result, the method for processing the data interaction log further comprises:
determining log generation time of all data interaction logs in the log file;
and deleting the data interaction log with the time difference between the log generation time and the current time being greater than the set time threshold.
8. A data interaction log processing apparatus, wherein the data interaction log processing apparatus includes:
the data acquisition module is used for acquiring a data interaction log for recording data interaction behaviors with a data interaction party;
the keyword extraction module is used for determining the communication keywords of the data interaction party;
the path calculation module is used for determining a log file path for storing the data interaction log according to the communication keyword or the communication keyword and the log generation time, wherein the log file path indicates a storage path of a log file for storing the data interaction log;
the detection module is used for detecting whether the log file exists or not;
and the storage module is used for storing the data interaction log based on the detection result.
9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110736311.4A CN113420003A (en) | 2021-06-30 | 2021-06-30 | Method, device, equipment and medium for processing data interaction log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110736311.4A CN113420003A (en) | 2021-06-30 | 2021-06-30 | Method, device, equipment and medium for processing data interaction log |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113420003A true CN113420003A (en) | 2021-09-21 |
Family
ID=77717314
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110736311.4A Pending CN113420003A (en) | 2021-06-30 | 2021-06-30 | Method, device, equipment and medium for processing data interaction log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113420003A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115129674A (en) * | 2022-07-20 | 2022-09-30 | 江苏中信博新能源科技股份有限公司 | Local log recording method and system for photovoltaic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107870842A (en) * | 2016-09-28 | 2018-04-03 | 平安科技（深圳）有限公司 | A kind of blog management method and system |
| CN108170578A (en) * | 2018-01-08 | 2018-06-15 | 武汉斗鱼网络科技有限公司 | log collecting method and device |
| CN109885545A (en) * | 2019-02-02 | 2019-06-14 | 华为技术有限公司 | It stores, the method, apparatus of inquiry log information |
| CN111352902A (en) * | 2020-02-28 | 2020-06-30 | 中国平安人寿保险股份有限公司 | Log processing method and device, terminal equipment and storage medium |
| CN112214465A (en) * | 2020-10-19 | 2021-01-12 | 珠海金山网络游戏科技有限公司 | Log storage system and method |
-
2021
- 2021-06-30 CN CN202110736311.4A patent/CN113420003A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107870842A (en) * | 2016-09-28 | 2018-04-03 | 平安科技（深圳）有限公司 | A kind of blog management method and system |
| CN108170578A (en) * | 2018-01-08 | 2018-06-15 | 武汉斗鱼网络科技有限公司 | log collecting method and device |
| CN109885545A (en) * | 2019-02-02 | 2019-06-14 | 华为技术有限公司 | It stores, the method, apparatus of inquiry log information |
| CN111352902A (en) * | 2020-02-28 | 2020-06-30 | 中国平安人寿保险股份有限公司 | Log processing method and device, terminal equipment and storage medium |
| CN112214465A (en) * | 2020-10-19 | 2021-01-12 | 珠海金山网络游戏科技有限公司 | Log storage system and method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115129674A (en) * | 2022-07-20 | 2022-09-30 | 江苏中信博新能源科技股份有限公司 | Local log recording method and system for photovoltaic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102537275B1 (en) | Obfuscation of user content in structured user data files | |
| US10482240B2 (en) | Anti-malware device, anti-malware system, anti-malware method, and recording medium in which anti-malware program is stored | |
| CN111597382A (en) | Network security auditing method and system | |
| CN114968754A (en) | Application program interface API test method and device | |
| CN109284331B (en) | Certificate making information acquisition method based on service data resources, terminal equipment and medium | |
| CN112882797A (en) | Container safety detection method based on machine learning | |
| CN109947730B (en) | Metadata recovery method, device, distributed file system and readable storage medium | |
| CN114968959A (en) | Log processing method, log processing device and storage medium | |
| CN113420003A (en) | Method, device, equipment and medium for processing data interaction log | |
| CN111143103A (en) | Incidence relation determining method, device, equipment and readable storage medium | |
| US9483535B1 (en) | Systems and methods for expanding search results | |
| CN112052259A (en) | Data processing method, device, equipment and computer storage medium | |
| CN112433888A (en) | Data processing method and device, storage medium and electronic equipment | |
| US8745001B1 (en) | Automated remediation of corrupted and tempered files | |
| CN116185785A (en) | Early warning method and device for file abnormal change | |
| US20160085638A1 (en) | Computer system and method of identifying a failure | |
| CN115186001A (en) | Patch processing method and device | |
| CN114281769A (en) | Method and device for managing files on disk, computer equipment and storage medium | |
| JP4445750B2 (en) | Causal relationship estimation program and causal relationship estimation method | |
| CN113468029A (en) | Log management method and device, electronic equipment and readable storage medium | |
| CN116545642B (en) | Terminal monitoring management system for specific environment | |
| KR102442126B1 (en) | Disk management system | |
| US20240111742A1 (en) | Management device, management method, and recording medium | |
| JP2013197601A (en) | Fault detection device, fault detection method, and program | |
| JPH11219310A (en) | Illegality detecting/notifying method in online transaction rpocessing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |