CN110932918B - Log data acquisition method and device and storage medium - Google Patents
Log data acquisition method and device and storage medium Download PDFInfo
- Publication number
- CN110932918B CN110932918B CN201911368492.9A CN201911368492A CN110932918B CN 110932918 B CN110932918 B CN 110932918B CN 201911368492 A CN201911368492 A CN 201911368492A CN 110932918 B CN110932918 B CN 110932918B
- Authority
- CN
- China
- Prior art keywords
- service node
- information
- state
- log
- calling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
- H04L41/064—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0677—Localisation of faults
Abstract
The disclosure discloses a log data acquisition method, a device and a storage medium, wherein the method comprises the following steps: the method comprises the steps of receiving log data information corresponding to each service node sent by each service node, analyzing the log data information, obtaining basic information of each service node and a calling state of each service node, obtaining running state information of each service node when the calling state is an abnormal state, and storing the basic information and the running state information of each service node as log information in a storage system. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a log data acquisition method, a log data acquisition device and a storage medium.
Background
Nowadays, in the process of processing business, more and more applications need to be called, and the calling relationship between the applications is more and more complex, and a large amount of log data is generated in the process.
At present, log data generated by the operation of an application program can be collected according to a link to which a service belongs through a log collection tool, and the content of the collected log data is data related to the operation of the application program.
However, in the related art, when log data generated by a service is collected, only data of each application program running the service is collected, so that only the data of each application program running the service can be analyzed, and there is a problem in analyzing a link with a complex call relation.
Disclosure of Invention
The disclosure provides a log data acquisition method and device of Internet of things equipment and a storage medium. The technical scheme is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a log data acquisition method, where the method includes:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the current service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, when the call state of the service node is an abnormal state, before the obtaining of the running state information of the service node, the method further includes:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, after storing the basic information of each service node and the running state information as log information in a storage system, the method further includes:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
According to a second aspect of the embodiments of the present disclosure, there is provided a log data collecting apparatus, the apparatus including:
the data receiving module is used for receiving the log data information which is sent by each service node and corresponds to each service node;
the basic information acquisition module is used for analyzing the log data information and acquiring the basic information of each service node and the calling state of each service node;
a state obtaining module, configured to obtain running state information of the service node when the call state of the service node is an abnormal state, where the running state information is used to indicate a running state of a system in which the service node is currently located;
and the information storage module is used for storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, the apparatus further comprises:
and the exception determining module is used for determining that the calling state of the service node is an abnormal state when the calling state of the service node is the abnormal state and the basic information of the service node is analyzed before the running state information of the service node is acquired, and the return value information is failure information.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, the apparatus further comprises:
the log information acquisition module is used for acquiring the corresponding log information in the storage system according to query conditions, and the query conditions are input through a client;
and the log information display module is used for displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
According to a third aspect of the embodiments of the present disclosure, there is provided a log data collecting apparatus, the apparatus including:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
According to a fourth aspect of the embodiments of the present disclosure, a computer-device-readable storage medium is provided, where the computer-device-readable storage medium contains executable instructions, and the executable instructions are called and executed by a processor, so as to implement the log data collection method according to the first aspect or any one of the alternatives of the first aspect.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
receiving log data information corresponding to each service node sent by each service node, acquiring basic information of each service node and a calling state of each service node by analyzing the log data information, acquiring running state information of the service node when the calling state is an abnormal state, wherein the running state information is used for indicating the running state of a system where the service node is located at present, and storing the basic information and the running state information of each service node as log information in a storage system. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a schematic diagram of an implementation environment shown in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a method of log data collection in accordance with an exemplary embodiment;
FIG. 3 is a flow chart illustrating a method of log data collection in accordance with another exemplary embodiment;
FIG. 4 is a schematic diagram of a full link service node invocation relationship involved in the embodiment shown in FIG. 3;
FIG. 5 is a schematic diagram of a full link service node invocation failure involved in the embodiment shown in FIG. 3;
FIG. 6 is a schematic diagram of an interface for a client to input query conditions according to the embodiment shown in FIG. 3;
FIG. 7 is a block diagram illustrating a log data collection apparatus in accordance with an exemplary embodiment;
FIG. 8 is a schematic diagram illustrating a configuration of a computer device, according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
It is to be understood that reference herein to "a number" means one or more and "a plurality" means two or more. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
FIG. 1 is a schematic diagram of an implementation environment, shown in accordance with an exemplary embodiment. This implementation environment includes several devices 110, a log server 120, and a client 130.
The device 110 may be a terminal device, a computer device, or a server with data transmission capabilities.
For example, the device 110 may be a mobile portable terminal such as a smart phone, a tablet computer, and an electronic book reader, or a computer device such as a notebook computer, or the device 110 may also be an intelligent wearable device such as smart glasses and a smart watch.
The devices 110 may be connected to each other through a wireless or wired network, and the devices 110 may perform one or several services through mutual invocation.
The connection between the device 110 and the log server 120 may be via a wireless or wired network.
The log server 120 may retrieve data in the device 110 and store in a memory of the log server 120.
The log server 120 may be a server, or the log server 120 may also be a cloud computing service center, and at this time, the data acquired by the terminal 110 may be stored in a database of the cloud computing service center.
Data transmission can be performed between the client 130 and the log server 120, and the client 130 can query the data in the log server 120 and obtain the log data stored in the memory of the log server 120.
Optionally, the wireless or wired networks described above use standard communication techniques and/or protocols. The Network is typically the Internet, but may be any Network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wireline or wireless Network, a private Network, or any combination of virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including Hypertext Mark-up Language (HTML), extensible Markup Language (XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), transport Layer Security (TLS), virtual Private Network (VPN), internet Protocol Security (IPsec). In other embodiments, custom and/or dedicated data communication techniques may also be used in place of, or in addition to, the data communication techniques described above.
FIG. 2 is a flow chart illustrating a method of log data collection in accordance with an exemplary embodiment. The log data acquisition method can be applied to a log server, and the log information is stored in a storage system of the log server by acquiring the log information on the equipment. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. As shown in fig. 2, the log data collection method may include the steps of:
in step 201, receiving log data information corresponding to each service node sent by each service node;
in step 202, analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
in step 203, when the calling state of the service node is an abnormal state, obtaining running state information of the service node, where the running state information is used to indicate a running state of a system in which the service node is currently located;
in step 204, the basic information of each service node and the running state information of the service node whose calling state is the abnormal state are stored in the storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in the form of letter and number codes.
Optionally, when the call state of the service node is an abnormal state, before the obtaining of the running state information of the service node, the method further includes:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
Optionally, the operation status information includes at least one of central processing unit status information, storage system status information, network status information, input/output status information, and disk status information of the current system.
Optionally, after storing the basic information of each service node and the running state information as log information in a storage system, the method further includes:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
In summary, according to the log data collecting method provided in the embodiment of the present disclosure, log data information corresponding to each service node and sent by each service node is received, the log data information is analyzed, basic information of each service node and a call state of each service node are obtained, when the call state is an abnormal state, running state information of the service node is obtained, the running state information is used to indicate a running state of a system where the service node is currently located, and the basic information and the running state information of each service node are stored in a storage system as log information. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
Fig. 3 is a flowchart illustrating a log data collection method according to another exemplary embodiment, which may be applied to a log server for storing log information in a storage system of the log server by acquiring the log information on a device. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. As shown in fig. 3, the log data collection method may include the steps of:
in step 301, the log server receives log data information corresponding to each service node sent by each service node.
In the embodiment of the disclosure, in order to perform one or more services, each terminal or server runs an application program related to the service, and in the process of running the corresponding application program, each terminal or server generates a data packet containing data information of the running application program, and uploads the data packet generated by running the application program to a cloud platform or a cloud server of a log collection tool through the log collection tool on each terminal or server. The log server receives a data packet containing log data information through a cloud platform or a cloud server of the log collection tool.
Wherein, the log collection tool can be a Logstash tool, a Filebeat tool, a Flume tool, a fluent tool, a Loggent tool, an rsyslog tool, and a syslog-ng tool.
Alternatively, the respective service nodes may be located in the same or different servers or terminals.
Wherein, any one of the log collecting tools is installed in the same or different servers or terminals where each service node is located.
For example, a service needs to call three services, namely, a service node a, a service node B and a service node C, where the three service nodes are located in three different servers, a log collection tool installed in the server where the service node a, the service node B and the service node C are located is a logsend tool, and according to a collection principle of the logsend tool, the logsend is a transmission tool provided by the Sematext, and is used to transmit a log to logsend, and the logsend is an Application Programming Interface (API) based on the SaaS platform, because the logsend exposes the Application Programming Interface (API), the logsend can easily push data to the elaticsearch.
In step 302, the log server parses the log data information, and obtains the basic information of each service node and the call status of each service node.
In the embodiment of the disclosure, the log server downloads a data packet containing data information of each application program from a cloud platform or a cloud server of the log collection tool, and analyzes the data packet to obtain basic information of each service node and a call state of each service node.
The basic information of each service node comprises link identification information of the current service node, identification information of a source service node, return value information of the current service node and the running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a digitally encoded form.
The link identification information is used to indicate the link to which the current serving node belongs, and a complete link represents a service. At least one service node is required to be called to complete one service, the link identification of at least one service node called to complete one service is the same and is different from the link identification of service nodes called to complete other services, and the link identification can be marked and distinguished by using a numerical sequence number.
For example, a service a needs to invoke four service nodes, namely, a service a, a service B, a service C, and a service D, link identification information carried in data packets sent by the four service nodes may be 1, a service B needs to invoke two service nodes, namely, a service E and a service F, and link identification information carried in data packets sent by the two service nodes may be 2.
The current service node identification information is used for representing the identity of the current service node, and the source service node identification information is used for representing the identity of the service node calling the current service node. The current service node identification information and the source service node identification information are marked and distinguished by using a digital serial number.
For example, if service a calls service B and service B calls C, then service a is the source service node of service B, and the service node identification information of service a may be 0, then the service node identification information of service B is 1, and service B is the source service node of service C, so the service node identification information of service C is 2.
For example, please refer to fig. 4, which shows a schematic diagram of a call relationship of a full link service node according to an embodiment of the present disclosure, and as shown in fig. 4, the link starts from service a and ends at service E. Wherein: the service A calls the service B1, the service B1 calls the service B2 first, then calls the service C1, then the service C1 calls the service C2, the service C2 calls the service C3, then the service C1 calls the service D, finally the service D calls the service E, and the link calling is finished. The service node identifier of the service A is 0, and the source service node identifier is null; the service node identifier of the service B1 is 1, and the source service node identifier is 0; the service node identifier of the service B2 is 2, and the source service node identifier is 1; the service node identifier of the service C1 is 3, and the source service node identifier is 1; the service node identifier of the service C2 is 4, and the source service node identifier is 3; the service node identifier of the service C3 is 5, and the source service node identifier is 4; the service node identifier of the service D is 6, and the source service node identifier is 3; service node identification of service E is 4 and source service node identification is 3.
The return value information of the current service node is used for representing the data information returned from the next service node when the current service node calls the next service node.
Optionally, the return value information of the current service node may include call success information that a call is successful or call failure information that a call is failed.
For example, when the service a calls the service B, the service B sends a data message to the service a, and if the call is successful, the service B sends a call success message to the service a; if the call fails, service B sends a call failure message to service A.
For example, please refer to fig. 5, which illustrates a schematic diagram of a call failure of a full link service node according to an embodiment of the present disclosure, as shown in fig. 5, the link starts from service a and ends at service E. Wherein: service B is initially invoked by service a, then service C by service B, then service D by service C, and finally service E by service D. When an error occurs in the process of calling the service D by the service C, the service C fails to call the service D, and at the moment, the service D sends a return value to the service C to inform the service C of call failure information.
Optionally, the running time of the current service node is a starting time and an ending time of the current service running.
In step 303, when the log server parses the basic information of the service node and obtains that the return value information is failure information, it determines that the calling state of the service node is an abnormal state.
In the embodiment of the present disclosure, the log server analyzes the log data information to obtain the call state of each service node, and the call state of each service node is determined according to the return value information of the current service node and is represented in the form of a state code. And if the return value information is calling success information, the calling state is a normal state. And if the return value information is calling failure information, the calling state is an abnormal state.
In step 304, when the calling state is an abnormal state, the log server obtains the running state information of the service node.
In the embodiment of the present disclosure, the operation state information is used to indicate an operation state of a system in which the current service node is located.
Optionally, the running state information includes at least one of central processor state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
When the log server determines that the calling state is an abnormal state according to the obtained return value information by analyzing the basic information of the service nodes, the log server obtains the system running states of the terminals or the servers where the two service nodes are located in the process that the current service node calls the next service node, wherein the system running states include the data information of a Central Processing Unit (CPU) of the terminals or the systems of the servers where the current two service nodes are located; acquiring data information of a storage system of a terminal or a server where two current service nodes are located; acquiring network data information of terminals or servers where two current service nodes are located; acquiring data information of input/output values of terminals or servers where two current service nodes are located; and acquiring the disk running data information of the terminal or the server where the current two service nodes are located.
In step 305, the log server stores the basic information of the respective service nodes and the operation state information as log information in the storage system.
In the embodiment of the present disclosure, the log server records the acquired basic information of each service node and the running state information of the service node with abnormal call in the storage system.
The storage system may be a memory or a database.
In step 306, the log server obtains the corresponding log information in the storage system according to a query condition, where the query condition is input through the client.
In the embodiment of the present disclosure, the storage system of the log server stores the basic information of each service node and the running state information of the service node with abnormal call. The user inputs a query condition at the client, and the log server acquires the basic information of the corresponding service node and the running state information of the service node in which the abnormal call is called according to the input query condition.
Referring to fig. 6, which shows an interface schematic diagram of a client inputting query conditions according to an embodiment of the present disclosure, as shown in fig. 6, four query condition input boxes, i.e., a link identifier input box, a service identifier input box, a start time input box, and an end time input box, are arranged in a log query interface of the client. The user can select at least one query condition to input into the corresponding query condition input box, and the log server is controlled to acquire corresponding log information according to the query condition input by the user by triggering the search control.
For example, as shown in fig. 6, the user inputs 3 in the link identifier input box, and then performs a trigger operation on the search control, and the log server may query all log information of each service node with a link identifier of 3. If the user inputs 3 in the link identifier input box, then inputs 2019.01.01 in the start time input box, inputs 2019.01.03 in the end time input box, and by performing triggering operation on the search control, the log server can query all log information of each service node with the link identifier of 3 from 1 month 1 day of 2019 to 1 month 3 day of 2019.
In step 307, the log server displays the log information on the display interface of the client.
In the embodiment of the disclosure, the log server integrates and sorts the log information according to a certain sequence according to the obtained log information corresponding to the query condition, and finally displays the log information sorted according to the certain sequence on the display interface of the client.
Optionally, the log information may be presented on a display interface of the client in a list form.
For example, as shown in table 1, when the query condition is a link identifier, the service node identifiers, the service names, the states, the entries, the return values, and the source service node identifiers of all the service nodes on the link may be displayed on the client display interface, the call relationship of each service node and the states of each service node during call may be restored, and the service node with a problem and the part with an abnormality during call of the service node may be accurately found.
TABLE 1
Optionally, the log information is sorted and integrated according to a certain order, and may be adjusted according to the query condition.
For example, when the query condition is a time period, that is, the query condition is a start time and an end time, the log information may be integrated and sorted according to the time sequence, and the log information of the time period meeting the query condition is acquired and displayed on the display interface of the client. As shown in table 2, when the query condition is the start time and the end time, the time information of each service node, the log information of each service node, that is, the log information of each application program, and the running state information of the service node that failed in the call may be displayed on the display interface of the client. The service nodes are arranged according to the time sequence, and the time nodes with the calling failure problem and the service node system running state corresponding to the time nodes can be accurately found according to the arrangement, so that the service problem is positioned.
TABLE 2
In summary, in the log data collection method provided in this disclosure, log data information corresponding to each service node sent by each service node is received, the log data information is analyzed, so as to obtain basic information of each service node and a call state of each service node, when the call state is an abnormal state, operation state information of the service node is obtained, the operation state information is used to indicate an operation state of a system where the service node is currently located, and the basic information of each service node and the operation state information are stored in a storage system as log information. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
Fig. 7 is a block diagram illustrating a log data collection apparatus according to an exemplary embodiment, and as shown in fig. 7, the log data collection apparatus may be implemented as all or part of a log server in hardware or a combination of hardware and software to perform the steps shown in any one of the embodiments shown in fig. 2 or fig. 3. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. The log data acquisition device may include:
a data receiving module 710, configured to receive log data information corresponding to each service node sent by each service node;
a basic information obtaining module 720, configured to analyze the log data information, and obtain the basic information of each service node and the call state of each service node;
a state obtaining module 730, configured to obtain running state information of the service node when the call state of the service node is an abnormal state, where the running state information is used to indicate a running state of a system in which the current service node is located;
an information storage module 740, configured to store the basic information of each service node and the running state information of the service node whose calling state is the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, the apparatus further comprises:
and the exception determining module is used for determining that the calling state of the service node is an abnormal state when the calling state of the service node is the abnormal state and the return value information is failure information after analyzing the basic information of the service node before acquiring the running state information of the service node.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, the apparatus further comprises:
the log information acquisition module is used for acquiring the corresponding log information in the storage system according to query conditions, and the query conditions are input through a client;
and the log information display module is used for displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of each functional module is illustrated, and in practical applications, the above functions may be distributed by different functional modules according to actual needs, that is, the content structure of the device may be divided into different functional modules to implement all or part of the functions described above.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
An exemplary embodiment of the present disclosure provides a log data collecting apparatus, which may be implemented as all or part of a log server in a hardware or software and hardware combination manner to perform the steps shown in any one of the embodiments shown in fig. 2 or fig. 3. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. The log data acquisition device further comprises: a processor, a memory for storing processor-executable instructions;
wherein the processor is configured to:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the current service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, when the call state of the service node is an abnormal state, before the obtaining of the running state information of the service node, the method further includes:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, after the storing the basic information of each service node and the running state information as log information in a storage system, the method further includes:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
FIG. 8 is a schematic diagram illustrating a configuration of a computer device, according to an example embodiment. The computer apparatus 800 includes a Central Processing Unit (CPU) 801, a system Memory 804 including a Random Access Memory (RAM) 802 and a Read-Only Memory (ROM) 803, and a system bus 805 connecting the system Memory 804 and the CPU 801. The computer device 800 also includes a basic Input/Output (I/O) system 806 for facilitating information transfer between various devices within the computer device, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.
The basic input/output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse, keyboard, etc. for user input of information. Wherein the display 808 and the input device 809 are connected to the central processing unit 801 through an input output controller 810 connected to the system bus 805. The basic input/output system 806 may also include an input/output controller 810 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 810 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 807 is connected to the central processing unit 801 through a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer device-readable media provide non-volatile storage for the computer device 800. That is, the mass storage device 807 may include a computer device readable medium (not shown) such as a hard disk or Compact Disc-Only Memory (CD-ROM) drive.
Without loss of generality, the computer device readable media may include computer device storage media and communication media. Computer device storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer device readable instructions, data structures, program modules or other data. Computer device storage media includes RAM, ROM, erasable Programmable Read-Only Memory (EPROM), electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, digital Video Disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer device storage media is not limited to the foregoing. The system memory 804 and mass storage 807 described above may be collectively referred to as memory.
The computer device 800 may also operate as a remote computer device connected to a network through a network, such as the internet, in accordance with various embodiments of the present disclosure. That is, the computer device 800 may connect to the network 812 through a network interface unit 811 coupled to the system bus 805, or may connect to other types of networks or remote computer device systems (not shown) using the network interface unit 811.
The memory further includes one or more programs, the one or more programs are stored in the memory, and the central processing unit 801 executes the one or more programs to implement all or part of the steps of the method shown in fig. 2 or fig. 3.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in embodiments of the disclosure may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-device-readable medium. Computer device readable media includes both computer device storage media and communication media including any medium that facilitates transfer of a computer device program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer device.
The embodiment of the present disclosure further provides a computer device storage medium, which is used for storing computer device software instructions for the testing apparatus, and includes a program designed for executing the log data acquisition method.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (7)
1. A method for log data collection, the method comprising:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring basic information of each service node and a calling state of each service node, wherein the basic information of each service node comprises link identification information of a link where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node and running time of the current service node, and the return value information of the current service node comprises calling success information or calling failure information returned from the next service node when the current service node calls the next service node;
when the basic information of the service node is analyzed and the return value information is obtained as the calling failure information, determining that the calling state of the service node is an abnormal state;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located; the operating state information includes: in the process that the current service node calls the next service node, the central processing unit state information, the storage system state information, the network state information, the input/output state information and the disk state information of a terminal or a server where two service nodes are located comprise the current service node and the next service node;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
2. The method of claim 1, wherein the link identification information, the current serving node identification information, and the source serving node identification information are identified in alphanumeric codes.
3. The method according to claim 1, wherein after storing the basic information of each service node and the operating state information as log information in a storage system, the method further comprises:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
4. The method of claim 3, wherein the query condition comprises at least one of a running time period information, a link identification information, and a running time of the service node.
5. An apparatus for collecting log data, the apparatus comprising:
the data receiving module is used for receiving the log data information which is sent by each service node and corresponds to each service node;
a basic information obtaining module, configured to analyze the log data information, and obtain basic information of each service node and a call state of each service node, where the basic information of each service node includes link identifier information of a link where a current service node is located, current service node identifier information, source service node identifier information, return value information of the current service node, and running time of the current service node, and the return value information of the current service node includes call success information or call failure information returned from a next service node when the current service node calls the next service node;
when the basic information of the service node is analyzed and the return value information is obtained as the calling failure information, determining that the calling state of the service node is an abnormal state;
a state obtaining module, configured to obtain running state information of the service node when the call state is an abnormal state, where the running state information is used to indicate a running state of a system in which the service node is located; the operating state information includes: in the process that the current service node calls the next service node, the state information of central processing units of terminals or servers where two service nodes are located, the state information of a storage system, the state information of a network, the state information of input/output and the state information of a disk, wherein the two service nodes comprise the current service node and the next service node;
and the information storage module is used for storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
6. An apparatus for collecting log data, the apparatus comprising:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring basic information of each service node and a calling state of each service node, wherein the basic information of each service node comprises link identification information of a link where a current service node is located, identification information of the current service node, source service node identification information, return value information of the current service node and running time of the current service node, and the return value information of the current service node comprises calling success information or calling failure information returned from a next service node when the current service node calls the next service node;
the exception determining module is used for determining that the calling state of the service node is an abnormal state when the basic information of the service node is analyzed and the return value information is obtained as the calling failure information;
when the calling state is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located; the operating state information includes: in the process that the current service node calls the next service node, the central processing unit state information, the storage system state information, the network state information, the input/output state information and the disk state information of a terminal or a server where two service nodes are located comprise the current service node and the next service node;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
7. A computer device readable storage medium, wherein the computer device readable storage medium contains executable instructions, and the executable instructions are called and executed by a processor to implement the log data collection method of any one of the above claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368492.9A CN110932918B (en) | 2019-12-26 | 2019-12-26 | Log data acquisition method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368492.9A CN110932918B (en) | 2019-12-26 | 2019-12-26 | Log data acquisition method and device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110932918A CN110932918A (en) | 2020-03-27 |
| CN110932918B true CN110932918B (en) | 2023-01-10 |
Family
ID=69860995
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911368492.9A Active CN110932918B (en) | 2019-12-26 | 2019-12-26 | Log data acquisition method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110932918B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111740860B (en) * | 2020-06-15 | 2023-06-09 | 中国工商银行股份有限公司 | Log data transmission link monitoring method and device |
| CN112764997B (en) * | 2021-01-28 | 2024-02-20 | 抖音视界有限公司 | Log storage method and device, computer equipment and storage medium |
| CN113190583B (en) * | 2021-05-14 | 2024-02-20 | 长春理工大学 | Data acquisition system, method, electronic equipment and storage medium |
| CN114185502A (en) * | 2021-12-15 | 2022-03-15 | 平安科技（深圳）有限公司 | Log printing method, device, equipment and medium based on production line environment |
| CN114650252B (en) * | 2022-03-25 | 2023-11-21 | 华润数字科技(深圳)有限公司 | Routing method and device based on enterprise service bus and computer equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912587A (en) * | 2016-03-31 | 2016-08-31 | 乐视控股（北京）有限公司 | Data acquisition method and system |
| CN106254144A (en) * | 2016-09-06 | 2016-12-21 | 华为技术有限公司 | Fault location platform, Fault Locating Method and device |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5995981A (en) * | 1997-06-16 | 1999-11-30 | Telefonaktiebolaget Lm Ericsson | Initialization of replicated data objects |
| CN106339379B (en) * | 2015-07-07 | 2019-08-16 | 阿里巴巴集团控股有限公司 | Website running state monitoring method and device |
| US10275301B2 (en) * | 2015-09-29 | 2019-04-30 | International Business Machines Corporation | Detecting and analyzing performance anomalies of client-server based applications |
| CN106021064A (en) * | 2016-05-16 | 2016-10-12 | 浪潮电子信息产业股份有限公司 | Monitoring information collection method, baseboard management controller and system |
| CN106815115B (en) * | 2017-01-13 | 2020-02-21 | 苏州浪潮智能科技有限公司 | Server running state monitoring system |
| US10929218B2 (en) * | 2018-05-16 | 2021-02-23 | Nec Corporation | Joint semantic and format similarity for large scale log retrieval |
| CN109656778B (en) * | 2018-11-28 | 2022-07-12 | 金蝶软件（中国）有限公司 | Data acquisition method and device, computer equipment and storage medium |
| CN110113200A (en) * | 2019-04-29 | 2019-08-09 | 平安科技（深圳）有限公司 | The correlating method of chain-circuit system and log system, device and storage medium |
| CN110297746A (en) * | 2019-07-05 | 2019-10-01 | 北京慧眼智行科技有限公司 | A kind of data processing method and system |
| CN110502494B (en) * | 2019-08-30 | 2022-03-18 | 北京思维造物信息科技股份有限公司 | Log processing method and device, computer equipment and storage medium |
-
2019
- 2019-12-26 CN CN201911368492.9A patent/CN110932918B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912587A (en) * | 2016-03-31 | 2016-08-31 | 乐视控股（北京）有限公司 | Data acquisition method and system |
| CN106254144A (en) * | 2016-09-06 | 2016-12-21 | 华为技术有限公司 | Fault location platform, Fault Locating Method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110932918A (en) | 2020-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110932918B (en) | Log data acquisition method and device and storage medium | |
| US10769228B2 (en) | Systems and methods for web analytics testing and web development | |
| CN111683066B (en) | Heterogeneous system integration method, heterogeneous system integration device, computer equipment and storage medium | |
| CN107957940B (en) | Test log processing method, system and terminal | |
| KR100865015B1 (en) | Realtime unification management information data conversion and monitoring apparatus and method for thereof | |
| WO2020248375A1 (en) | Method and system for synchronizing data between databases, computer device and storage medium | |
| US11533387B2 (en) | Interface engine architecture | |
| CN111342992B (en) | Method and system for processing equipment information change record | |
| CN107133233B (en) | Processing method and device for configuration data query | |
| CN112631924A (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN110727948B (en) | Intelligent contract auditing method and device, computer equipment and storage medium | |
| CN111586022A (en) | Firewall opening verification method, electronic device, computer equipment and storage medium | |
| CN113656194A (en) | Account checking result data notification method and device, electronic device and storage medium | |
| CN113760730A (en) | Automatic testing method and device | |
| CN112181836A (en) | Test case generation method, system, device and storage medium | |
| WO2017117870A1 (en) | Processing method and apparatus for trap instruction trap | |
| CN112579406A (en) | Log call chain generation method and device | |
| CN115604343A (en) | Data transmission method, system, electronic equipment and storage medium | |
| CN113515715B (en) | Buried point event code generation method, buried point event code processing method and related equipment | |
| CN114143308A (en) | File uploading information processing method and device, computer equipment and storage medium | |
| CN109634931B (en) | Log uploading method and device | |
| CN110166636B (en) | Pressure testing device and method and storage medium | |
| CN111324654A (en) | Interface calling method, system, computer device and computer readable storage medium | |
| CN114827197B (en) | Service communication method, device, equipment and medium based on Internet of things | |
| CN112215587B (en) | Method, device and system for automatically executing business process interaction tasks by multiple devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Hu Nan Inventor after: Zhou Xiaomin Inventor after: Yu Haifeng Inventor before: Hu Nan Inventor before: Zhou Xiaomin |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |