CN111367760B - Log collection method and device, computer equipment and storage medium - Google Patents
Log collection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111367760B CN111367760B CN202010126791.8A CN202010126791A CN111367760B CN 111367760 B CN111367760 B CN 111367760B CN 202010126791 A CN202010126791 A CN 202010126791A CN 111367760 B CN111367760 B CN 111367760B
- Authority
- CN
- China
- Prior art keywords
- log
- event
- service system
- collecting
- event identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Computer Hardware Design (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a log acquisition method and device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving a hypertext transfer protocol (HTTP) request sent by a service system, wherein the HTTP request is used for requesting a log system to record an operation event of the service system; converting a first event identifier which identifies the operation event in the service system into a second event identifier according to a preset adaptation rule, wherein the second event identifier is used for identifying the operation event in the log system; and collecting log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event. The invention at least solves the technical problems of high cost, low efficiency and the like of collecting the log files in the related technology.
Description
Technical Field
The invention relates to the field of computers, in particular to a log collection method and device, computer equipment and a storage medium.
Background
At present, with the development of software technology, the code line number of software becomes very large, and the work of maintaining the software becomes very complicated and difficult. Generally, log record codes of key services can be added in the process of writing codes, so that in the running process of software, program processing conditions of the key services are recorded in log files, and then corresponding processing can be performed by analyzing the log files through monitoring the log files once the software runs abnormally.
In the related technology, a point burying method is generally integrated based on a customized front-end frame, when a user triggers a series of operations such as clicking and browsing, the point burying method is triggered to collect a corresponding log system, when a subsystem needs to record user behavior logs, a complete set of log frames must be integrated, the coupling degree is too high, and the cost is very high.
In view of the above problems in the related art, no effective solution has been found at present.
Disclosure of Invention
The embodiment of the invention provides a log collection method and device, computer equipment and a storage medium, and solves the technical problems of high cost, low efficiency and the like in log file collection in the related technology.
According to an embodiment of the present invention, there is provided a log collecting method including: receiving a hypertext transfer protocol (HTTP) request sent by a service system, wherein the HTTP request is used for requesting a log system to record an operation event of the service system; converting a first event identifier which identifies the operation event in the service system into a second event identifier according to a preset adaptation rule, wherein the second event identifier is used for identifying the operation event in the log system; and collecting log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
Optionally, the converting, according to a preset adaptation rule, a first event identifier that identifies the operation event in the service system into a second event identifier includes: reading an adaptation table, wherein the adaptation table is established by the service system based on the preset adaptation rule when the operation event is executed, and the operation event at least comprises: adding a user, deleting the user and updating the user; and converting the first event identifier of the operation event into the second event identifier according to the adaptation table, and recording the second event identifier.
Optionally, the collecting log information in the service system according to the second event identifier at least includes: acquiring a behavior log of the service system executing the operation event according to the second event identifier; and acquiring the running state information of the service corresponding to the operation event when the Java program in the service system runs according to the second event identifier.
Optionally, collecting a behavior log of the service system executing the operation event according to the second event identifier includes: collecting the behavior log by a collection component of the log system based on the second event identification; pushing the behavior log to a message cache queue of the log system; and collecting the behavior logs in the message cache queue through a log analysis center of the log system and analyzing the behavior logs.
Optionally, acquiring running state information of a service corresponding to the operation event executed by the Java program in the service system according to the second event identifier includes: collecting point state information of the Java program in operation through a collection assembly of the log system; caching the point state information to a message caching queue of the log system; and collecting point state information in the cache queue through a log analysis center of the log system and analyzing the point state information.
Optionally, collecting the point state information in the cache queue and analyzing the point state information by a log analysis center of the log system includes: storing the point state information collected from the cache queue to a search engine database; and connecting the point state information in series by using a search engine to obtain the calling link information of the Java program and the change trend information of the running state of the Java program.
Optionally, acquiring log information of the operation event in the service system according to the second event identifier includes: periodically collecting the log information based on the second event identification; and classifying and storing the log information, and displaying the log information through a visual chart.
According to still another embodiment of the present invention, there is also provided a log collecting apparatus including: the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a hypertext transfer protocol (HTTP) request sent by a service system, and the HTTP request is used for requesting a log system to record an operation event of the service system; a conversion module, configured to convert, according to a preset adaptation rule, a first event identifier that identifies the operation event in the service system into a second event identifier, where the second event identifier is used to identify the operation event in the log system; and the acquisition module is used for acquiring log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
Optionally, the conversion module includes: a reading unit, configured to read an adaptation table, where the adaptation table is established by the service system based on the preset adaptation rule when the operation event is executed, and the operation event at least includes: adding a user, deleting the user and updating the user; and the conversion unit is used for converting the first event identifier of the operation event into the second event identifier according to the adaptation table and recording the second event identifier.
Optionally, the acquisition module at least includes: the first acquisition unit is used for acquiring a behavior log of the business system executing the operation event according to the second event identifier; and the second acquisition unit is used for acquiring the running state information of the service corresponding to the operation event of the Java program in the service system according to the second event identifier.
Optionally, the first collecting unit includes: the first acquisition subunit is used for acquiring the behavior log through an acquisition component of the log system based on the second event identifier; the first cache unit is used for pushing the behavior log to a message cache queue of the log system; and the first processing subunit is used for collecting the behavior logs in the message cache queue through a log analysis center of the log system and analyzing the behavior logs.
Optionally, the second collecting unit includes: the second acquisition subunit is used for acquiring point state information of the Java program during the operation through an acquisition component of the log system; the second cache unit is used for caching the point state information to a message cache queue of the log system; and the second processing subunit is used for collecting the point state information in the cache queue through a log analysis center of the log system and analyzing the point state information.
Optionally, the second processing subunit is configured to: storing the point state information collected from the cache queue to a search engine database; and connecting the point state information in series by using a search engine to obtain the calling link information of the Java program and the change trend information of the running state of the Java program.
Optionally, the collecting module includes: the third acquisition unit is used for periodically acquiring the log information based on the second event identification; and the display unit is used for classifying and storing the log information and displaying the log information through a visual chart.
According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps in any of the apparatus embodiments described above when executed.
According to yet another embodiment of the present invention, there is also provided a computer device comprising a memory having a computer program stored therein and a processor configured to execute the computer program to perform the steps of any of the above method embodiments.
According to the invention, the event identification of the operation event executed in the service system is recorded by receiving the HTTP request sent by the service system, the first event identification of the operation event in the service system is converted into the standard second event identification in the log system according to the preset adaptation rule, and the log information recorded when the operation event is executed by the service system is collected in the service system according to the second event identification, so that when the log system is in butt joint with the service system, the frame of the log system is not required to be integrated in the service system, the log information in the service system can also be collected, and the coupling degree and the load of the service system are reduced; under the advantage that each service system is not required to integrate a log frame, a plurality of service systems can be simultaneously supported, the usability of the log system is improved, and the acquisition efficiency is improved; the technical problems of high cost, low efficiency and the like of collecting log files in the related technology are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention and do not constitute a limitation of the invention. In the drawings:
fig. 1 is a block diagram of a hardware structure of a computer terminal to which a log collection method according to an embodiment of the present invention is applied;
FIG. 2 is a flow chart of a method of log collection according to an embodiment of the invention;
FIG. 3 is a schematic flow chart of log collection according to an embodiment of the present invention;
fig. 4 is a block diagram of a log collection apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Example 1
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a server, a computer terminal, or a similar computing device. Taking the operation on a computer terminal as an example, fig. 1 is a block diagram of a hardware structure of a log collection method applied to a computer terminal according to an embodiment of the present invention. As shown in fig. 1, the computer terminal may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally, a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the computer terminal. For example, the computer terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of application software, such as a computer program corresponding to the log collection method in the embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104, thereby executing various functional applications and data processing, that is, implementing the method described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 can further include memory located remotely from the processor 102, which can be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In this embodiment, a log collecting method is provided, and fig. 2 is a flowchart of a log collecting method according to an embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
step S202, receiving a hypertext transfer protocol (HTTP) request sent by a service system, wherein the HTTP request is used for requesting a log system to record an operation event of the service system;
in the service system in this embodiment, when the user executes an operation event, the request is sent by clicking in the front end js (javascript), where the request includes: adding a request, deleting a request and updating a request; and after receiving the request, the background executes the service corresponding to the operation event, and simultaneously records the operation event in a relational database of the service system.
Step S204, converting a first event identifier which identifies an operation event in the service system into a second event identifier according to a preset adaptation rule, wherein the second event identifier is used for identifying the operation event in the log system;
in this embodiment, when the log system interfaces with the relational database of the business system, the log system needs to collect the operation events recorded in the relational database, and since the log system has standard event types (for example, addition, deletion, modification and check), there may be many event types transmitted from the business system, and the business system needs to specify an adaptation rule in advance for converting the event types (i.e., the event identifiers) in the business system into the standard event types in the log system.
Step S206, collecting the log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
Through the steps, the event identification of the operation event executed in the service system is recorded by receiving the HTTP request sent by the service system, the first event identification of the operation event in the service system is converted into the standard second event identification in the log system according to the preset adaptation rule, and the log information recorded when the operation event is executed by the service system is collected in the service system according to the second event identification, so that when the log system is butted with the service system, the frame of the log system is not required to be integrated in the service system, the log information in the service system can be collected, and the coupling degree and the load of the service system are reduced; under the advantage that each service system is not required to integrate a log frame, a plurality of service systems can be simultaneously supported, the usability of the log system is improved, and the acquisition efficiency is improved; the technical problems of high cost, low efficiency and the like of collecting the log files in the related technology are solved.
In an optional embodiment, converting the first event identifier identifying the operation event in the service system into the second event identifier according to a preset adaptation rule includes: reading an adaptation table, wherein the adaptation table is established by a service system based on preset adaptation rules when executing an operation event, and the operation event at least comprises: adding a user, deleting the user and updating the user; and converting the first event identification of the operation event into a second event identification according to the adaptation table, and recording the second event identification.
In this embodiment, when a user executes an operation event once, the service system executes a service corresponding to the operation event, records a user behavior log by using an event buried point, and stores the user behavior log in a relational database in the service system. For example: the part of the role management on the page is divided into three operation events, namely a new user, a delete user and an update user, wherein the user operates one of the operation events, the corresponding JAVA application program in the service system executes the service corresponding to the operation event, and the service system records the behavior log of the executed operation event into a relational database.
In an embodiment of the present disclosure, when a function point (i.e., the execution of an operation event) is added to the service system, an HTTP request is sent to the log system, and a time type of the operation event in the service system is determined by the HTTP request; the log system records the event type (for example, user add event of the log platform corresponding to the added user) in the log system corresponding to the added function point in the service system according to the HTTP request, namely, the log system adapts according to the preset adaptation rule and records the adapted operation event;
optionally, the adding a function point to the service system includes: establishing an adaptation table, and recording the operation event of each function point of a subsequent service system into the adaptation table of a database by the subsequent service system; and the log system reads the adaptation table and converts the event type of the operation event in the received HTTP request into a standard event type in the log system according to the adaptation table. For example, an adaptation table represented in the manner of table 1.
Table 1:
| service system function point | Event group | Event type |
| Newly added user | User (User) | Add |
| Deleting a user | User (User) | Delete |
| …… | …… | …… |
In this embodiment, the users in the event group in table 1 correspond to roles on the page, and the standard event type (i.e. the event identifier) in the log system at least includes: add, Update, Delete.
In an optional embodiment, collecting log information of the operation event in the service system according to the second event identifier at least includes: acquiring a behavior log of an operation event executed by the service system according to the second event identifier; and acquiring the running state information of the service corresponding to the Java program running operation event in the service system according to the second event identifier.
In the scenario of one embodiment of the present disclosure, the collection component collects the log files generated by each network device, and a user can perform big data analysis based on the collected log files to obtain valuable data in the log. However, when the current platform collects logs, the data printed in the log file by the program is often collected, and the data can be considered by developers in advance, and the data are corresponding logs recorded by the program which may need to be recorded or may have problems, and the defects and problems of the program are often ignored, that is, the collection and monitoring of the running state of the program cannot be realized, so that the collected logs cannot be analyzed inaccurately, and other technical problems are often solved.
Through the steps, when the service system executes the operation event, the operation state information of the corresponding service of the Java program operation in the service system is collected, and collection and monitoring are carried out according to the operation state information of the program, so that the accuracy of log collection can be improved, the accuracy of log analysis is further improved, and the technical problem that the collected log analysis is inaccurate due to the fact that the defects and the problem library of the program are ignored is solved.
In an optional embodiment of the present disclosure, collecting a behavior log of an operation event executed by a service system according to a second event identifier includes: collecting a behavior log through a collecting component of a log system based on the second event identification; pushing the behavior log to a message buffer queue of a log system; and collecting the behavior logs in the message cache queue through a log analysis center of the log system and analyzing the behavior logs.
In this embodiment, the functional components for collecting log files in the log system at least include: the system comprises a log data acquisition SDK (Software Development Kit) (namely the acquisition component), a message push queue (namely the message cache queue) and a log collection and analysis center (namely a log analysis center), and realizes the acquisition and monitoring of program state data currently running in a service system and a behavior log of an execution operation event.
In one embodiment of the scheme, Java Agent and ASM (Java bytecode manipulation framework) bytecode technologies are utilized, when a class binary file is loaded by a JVM (Java virtual machine), the class file is dynamically modified and loaded by the ASM, and timer functions are added before and after a monitoring method for calculating the time consumption of the monitoring method and simultaneously putting the time consumption of the method and the internal calling condition into a processor; the processor processes the sequence of method calling by utilizing the characteristics of first in and then out of the stack; and when one request is processed, sending the time-consuming method track to a log analysis center, and monitoring the running state of the JVM through the log collection and analysis center analysis time-consuming method track.
For example, when the Java program is running, the method executed first is an entry method of the application program, i.e., a main () method of a.java class, and the developer packages the code completion into app.jar; the jar package contains a class file (the Java class file is provided with a Java virtual machine and can be directly executed), and executes a start command: Java-Java agent-jar app. jar, the agent (i.e. jar package of the agent that i wrote) will modify the a.class file by adding a before () method before instantiating the a.class file; when the class A main () method is requested externally, the before () method is executed first, the executed class information is pushed to the log center, and meanwhile, if the class execution time length is needed, only one after () method needs to be added after the main () method, and then the time length can be calculated after the after-mentioned.
Optionally, acquiring running state information of a service corresponding to a Java program running operation event in the service system according to the second event identifier includes: point state information of the Java program in operation is collected through a collection assembly of a log system; caching the point state information into a message caching queue of a log system; and collecting point state information in the cache queue and analyzing the point state information through a log analysis center of the log system.
In one embodiment of the present disclosure, the log of operations recorded by the business system is read using logstack (which is a lightweight log gathering processing framework) open source tools. The method comprises the steps that point state information of a current program in operation is read in real time in a JVM (Java Virtual Machine) environment based on Java agents entering a Java program deeply (the Java agents can only collect the point state information of the current program), and the point state information is pushed to a message queue, wherein the purpose of introducing the message queue is that a large amount of data is pushed after sdk for collecting a lot of logs is deployed, and the message queue can buffer data flow pressure; and the log collection and analysis center collects point state information from the message queue and analyzes the point state information.
Optionally, collecting point state information in the cache queue and analyzing the point state information by using a log analysis center of the log system includes: storing the point state information collected from the cache queue to a search engine database; and connecting the point state information in series by using a search engine to obtain calling link information of the Java program and change trend information of the running state of the Java program.
In this embodiment, since the Java Agent can only collect the point state information of the current program, after the log collection and analysis center collects the point state information in the message push queue, it is necessary to concatenate the point state information to obtain the whole link information and the running state change trend information, and thus, monitoring of each index data such as the call link of the Java program, the performance loss of the jvm machine, and the running state of the jvm can be achieved.
In an embodiment of the present disclosure, collecting log information of an operation event in a service system according to a second event identifier includes: periodically collecting log information based on the second event identification; and classifying and storing the log information, and displaying the log information through a visual chart.
In this embodiment, the operation log of the service system is recorded in the operation table of the response, and the log system reads data at a certain frequency; preferably, the log system reads the operation event from the relational database of the service system at regular time (for example, every 1 minute), and does not depend on the active initiation of the service system, so as to avoid performance loss to the service system; the calling link of the program, the performance loss of the machine and the running state of the JVM are displayed through a visual chart, and various indexes are monitored by counting data changes in different time periods.
Fig. 3 is a schematic flowchart of a log collection process provided according to an embodiment of the present invention, and the log collection process of the present invention is further described with reference to fig. 3 below:
the first step is as follows: when a function point is added in the service system, an http request is sent to the log system, and an adaptation table is established through the http request;
the second step is that: when a user executes an operation event, operating the service system to record a function point (menu), and storing the recorded operation in an operation table (namely the relational database) of the service system; after receiving the http request, the log system reads the operation event through the Logstash, converts the event identifier of the operation event into a standard event in the log system according to the adaptation table, and collects log data in the operation table according to the type of the standard event; the log center stores the collected data into an elastic search database, and simultaneously creates a front-end page to read corresponding link and machine state data, if the data is abnormal or the called method is wrong, the data can be displayed on the page at the first time, and the monitoring effect is achieved.
For example, the program appearance error, exception, and network response code non-200 (such as 404, 500, etc.) of the javaweb project are collectively classified as error report; the error reporting trend is influenced by whether the program is wrongly written or not and the access amount of the program, and has no fixed change trend.
And finally, performing centralized display on the user behavior logs collected by the subsystems.
Through the embodiment, when the log system is in butt joint with the service system, the operation event is read according to the functional component, then the operation event is converted into the standard event type in the log system according to the adaptation rule, and the collected log data is managed and analyzed in a centralized mode. According to the method, the running state of the current program of the log file being collected in the environment is collected jvm by the functional component in the log system, the problem that the running state data of the running program can not be recorded when the log file is collected in the related technology is solved, and the technical personnel can know the performance change of the machine and the running link of the program without logging in the machine or inquiring the state of the machine through a cloud console or pre-burying points in the program; the log system collects and displays the operation events executed by the user at regular time, so that the service system does not need to integrate the components of the log system, and the coupling degree is reduced; the operation events recorded in the relational database are not required to be actively pushed to a log system, so that the load of a service system is reduced; and a plurality of business systems can be supported, thereby improving the usability of the log system.
Example 2
The present embodiment further provides a log collecting device, which is used to implement the foregoing embodiments and preferred embodiments, and the description of the log collecting device is omitted for brevity. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware or a combination of software and hardware is also possible and contemplated.
Fig. 4 is a block diagram of a log collection apparatus according to an embodiment of the present invention, and as shown in fig. 4, the apparatus includes: a receiving module 42, configured to receive a hypertext transfer protocol HTTP request sent by a service system, where the HTTP request is used to request a log system to record an operation event of the service system; a conversion module 44, connected to the receiving module 42, configured to convert, according to a preset adaptation rule, a first event identifier, which identifies the operation event in the business system, into a second event identifier, where the second event identifier is used to identify the operation event in the log system; and an acquiring module 46, connected to the converting module 44, configured to acquire log information in the service system according to the second event identifier, where the log information is a log recorded when the service system executes the operation event.
Optionally, the conversion module 44 includes: a reading unit, configured to read an adaptation table, where the adaptation table is established by a service system based on a preset adaptation rule when an operation event is executed, and the operation event at least includes: adding a user, deleting the user and updating the user; and the conversion unit is used for converting the first event identifier of the operation event into a second event identifier according to the adaptation table and recording the second event identifier.
Optionally, the acquisition module 46 at least includes: the first acquisition unit is used for acquiring a behavior log of the operation event executed by the service system according to the second event identifier; and the second acquisition unit is used for acquiring the running state information of the service corresponding to the Java program running operation event in the service system according to the second event identifier.
Optionally, the first collecting unit includes: the first acquisition subunit is used for acquiring the behavior log through an acquisition component of the log system based on the second event identifier; the first cache unit is used for pushing the behavior log to a message cache queue of the log system; and the first processing subunit is used for collecting the behavior logs in the message cache queue through a log analysis center of the log system and analyzing the behavior logs.
Optionally, the second collecting unit includes: the second acquisition subunit is used for acquiring point state information of the Java program in operation through an acquisition component of the log system; the second cache unit is used for caching the point state information into a message cache queue of the log system; and the second processing subunit is used for collecting the point state information in the cache queue through a log analysis center of the log system and analyzing the point state information.
Optionally, the second processing subunit is configured to: storing the point state information collected from the cache queue to a search engine database; and connecting the point state information in series by using a search engine to obtain the calling link information of the Java program and the change trend information of the running state of the Java program.
Optionally, the collecting module 46 includes: the third acquisition unit is used for periodically acquiring the log information based on the second event identification; and the display unit is used for classifying and storing the log information and displaying the log information through a visual chart.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, receiving a hypertext transfer protocol (HTTP) request sent by a service system, wherein the HTTP request is used for requesting a log system to record an operation event of the service system;
s2, converting a first event identifier, which identifies the operation event, in the service system into a second event identifier according to a preset adaptation rule, wherein the second event identifier is used for identifying the operation event in the log system;
and S3, collecting log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
Optionally, in this embodiment, the storage medium may include but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, receiving a hypertext transfer protocol (HTTP) request sent by a service system, wherein the HTTP request is used for requesting a log system to record an operation event of the service system;
s2, converting a first event identifier, which identifies the operation event, in the service system into a second event identifier according to a preset adaptation rule, wherein the second event identifier is used for identifying the operation event in the log system;
and S3, collecting log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention shall be included in the protection scope of the present invention.
Claims (10)
1. A log collection method, comprising:
receiving a hypertext transfer protocol (HTTP) request sent by a service system, wherein the HTTP request is used for requesting a log system to record an operation event of the service system;
converting a first event identifier which identifies the operation event in the service system into a second event identifier according to a preset adaptation rule, wherein the second event identifier is used for identifying the operation event in the log system;
and collecting log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
2. The method of claim 1, wherein converting a first event identifier identifying the operation event in the business system into a second event identifier according to a preset adaptation rule comprises:
reading an adaptation table, wherein the adaptation table is established by the service system based on the preset adaptation rules when executing the operation event, and the operation event at least comprises: adding a user, deleting the user and updating the user;
and converting the first event identifier of the operation event into the second event identifier according to the adaptation table, and recording the second event identifier.
3. The method of claim 1, wherein the collecting log information in the business system according to the second event identifier at least comprises:
acquiring a behavior log of the service system executing the operation event according to the second event identifier;
and acquiring running state information of a service corresponding to the operation event when the Java program in the service system runs according to the second event identifier.
4. The method of claim 3, wherein collecting a behavior log of the business system executing the operational event according to the second event identification comprises:
collecting the behavior log by a collection component of the log system based on the second event identification;
pushing the behavior log to a message cache queue of the log system;
and collecting the behavior logs in the message cache queue through a log analysis center of the log system and analyzing the behavior logs.
5. The method according to claim 3, wherein collecting running state information of a service corresponding to the operation event when a Java program in the service system runs according to the second event identifier comprises:
collecting point state information of the Java program in operation through a collection assembly of the log system;
caching the point state information to a message caching queue of the log system;
and collecting point state information in the cache queue through a log analysis center of the log system and analyzing the point state information.
6. The method of claim 5, wherein collecting point state information in the cache queue and analyzing the point state information by a log analysis center of the log system comprises:
storing the point state information collected from the cache queue to a search engine database;
and connecting the point state information in series by utilizing a search engine to obtain the calling link information of the Java program and the change trend information of the running state of the Java program.
7. The method of claim 1, wherein collecting log information in the business system according to the second event identifier comprises:
periodically collecting the log information based on the second event identification;
and classifying and storing the log information, and displaying the log information through a visual chart.
8. A log collection device, comprising:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a hypertext transfer protocol (HTTP) request sent by a service system, and the HTTP request is used for requesting a log system to record an operation event of the service system;
a conversion module, configured to convert, according to a preset adaptation rule, a first event identifier that identifies the operation event in the service system into a second event identifier, where the second event identifier is used to identify the operation event in the log system;
and the acquisition module is used for acquiring log information in the service system according to the second event identifier, wherein the log information is a log recorded when the service system executes the operation event.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program performs the steps of the method according to any of claims 1 to 7.
10. A computer storage medium on which a computer program is stored, characterized in that the computer program, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010126791.8A CN111367760B (en) | 2020-02-28 | 2020-02-28 | Log collection method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010126791.8A CN111367760B (en) | 2020-02-28 | 2020-02-28 | Log collection method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111367760A CN111367760A (en) | 2020-07-03 |
| CN111367760B true CN111367760B (en) | 2022-07-19 |
Family
ID=71208297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010126791.8A Active CN111367760B (en) | 2020-02-28 | 2020-02-28 | Log collection method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111367760B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114650236A (en) * | 2020-12-17 | 2022-06-21 | 中移(苏州)软件技术有限公司 | Log processing method and device and storage medium |
| CN112612673B (en) * | 2020-12-24 | 2022-12-06 | 青岛海尔科技有限公司 | Analysis method and device for dial testing log, storage medium and electronic device |
| CN112685274A (en) * | 2020-12-29 | 2021-04-20 | 北京达佳互联信息技术有限公司 | Log generation method and device, electronic equipment and storage medium |
| CN113138967B (en) * | 2021-05-08 | 2023-03-21 | 贵州全安密灵科技有限公司 | Data information acquisition method, black box, detonator and storage medium |
| CN113938379B (en) * | 2021-09-29 | 2024-06-04 | 浪潮云信息技术股份公司 | Method for dynamically loading cloud platform log acquisition configuration |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882710A (en) * | 2011-09-12 | 2013-01-16 | 微软公司 | Cross-machine event log interrelation |
| US8578393B1 (en) * | 2008-06-18 | 2013-11-05 | Alert Logic, Inc. | Log message collection employing on-demand loading of message translation libraries |
| CN107347024A (en) * | 2017-07-19 | 2017-11-14 | 华为技术有限公司 | A kind of method and apparatus for storing Operation Log |
| CN107426023A (en) * | 2017-07-21 | 2017-12-01 | 携程旅游信息技术（上海）有限公司 | Cloud platform log collection and retransmission method, system, equipment and storage medium |
| CN109218401A (en) * | 2018-08-08 | 2019-01-15 | 平安科技（深圳）有限公司 | Log collection method, system, computer equipment and storage medium |
-
2020
- 2020-02-28 CN CN202010126791.8A patent/CN111367760B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8578393B1 (en) * | 2008-06-18 | 2013-11-05 | Alert Logic, Inc. | Log message collection employing on-demand loading of message translation libraries |
| CN102882710A (en) * | 2011-09-12 | 2013-01-16 | 微软公司 | Cross-machine event log interrelation |
| CN107347024A (en) * | 2017-07-19 | 2017-11-14 | 华为技术有限公司 | A kind of method and apparatus for storing Operation Log |
| CN107426023A (en) * | 2017-07-21 | 2017-12-01 | 携程旅游信息技术（上海）有限公司 | Cloud platform log collection and retransmission method, system, equipment and storage medium |
| CN109218401A (en) * | 2018-08-08 | 2019-01-15 | 平安科技（深圳）有限公司 | Log collection method, system, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于AOP技术的邮件日志系统研究;刘新强，韩阳峰;《电子设计工程》;20140920;第22卷(第18期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111367760A (en) | 2020-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111367760B (en) | Log collection method and device, computer equipment and storage medium | |
| CN109710615B (en) | Database access management method, system, electronic device and storage medium | |
| CN109600258B (en) | Industrial protocol message recording device and method | |
| CN111400127B (en) | Service log monitoring method and device, storage medium and computer equipment | |
| CN111309550A (en) | Data acquisition method, system, equipment and storage medium of application program | |
| CN108228322B (en) | Distributed link tracking and analyzing method, server and global scheduler | |
| CN113472858B (en) | Buried point data processing method and device and electronic equipment | |
| CN109409948B (en) | Transaction abnormity detection method, device, equipment and computer readable storage medium | |
| CN112685370A (en) | Log collection method, device, equipment and medium | |
| CN111177094A (en) | Log data processing method and device, electronic equipment and storage medium | |
| CN112631879A (en) | Data acquisition method and device, computer readable medium and electronic equipment | |
| CN110851324A (en) | Log-based routing inspection processing method and device, electronic equipment and storage medium | |
| CN108228432A (en) | A kind of distributed link tracking, analysis method and server, global scheduler | |
| CN114385551B (en) | Log time-sharing management method, device, equipment and storage medium | |
| CN112579552A (en) | Log storage and calling method, device and system | |
| CN113434742A (en) | Account screening method and device, storage medium and electronic device | |
| CN108228417A (en) | Car networking log processing method and processing unit | |
| CN112579406A (en) | Log call chain generation method and device | |
| CN114610689B (en) | Recording and analyzing method for request log in distributed environment | |
| CN115168363B (en) | Metadata processing method and device, electronic equipment and storage medium | |
| CN114817300A (en) | Log query method based on SQL (structured query language) statements and application thereof | |
| CN110971644A (en) | Page access monitoring method and device | |
| CN116170321B (en) | Data collection method, device, equipment and storage medium for link tracking | |
| CN117312104B (en) | Visual link tracking method and system based on airport production operation system | |
| CN115689462A (en) | System automatic inspection method, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220520 Address after: 518000 China Aviation Center 2901, No. 1018, Huafu Road, Huahang community, Huaqiang North Street, Futian District, Shenzhen, Guangdong Province Applicant after: Shenzhen Ping An medical and Health Technology Service Co.,Ltd. Address before: Room 12G, Area H, 666 Beijing East Road, Huangpu District, Shanghai 200001 Applicant before: PING AN MEDICAL AND HEALTHCARE MANAGEMENT Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |