CN112506755B - Log acquisition method, device, computer equipment and storage medium - Google Patents
Log acquisition method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112506755B CN112506755B CN202011510561.8A CN202011510561A CN112506755B CN 112506755 B CN112506755 B CN 112506755B CN 202011510561 A CN202011510561 A CN 202011510561A CN 112506755 B CN112506755 B CN 112506755B
- Authority
- CN
- China
- Prior art keywords
- service
- log
- information
- acquisition
- service instance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012544 monitoring process Methods 0.000 claims abstract description 89
- 238000004590 computer program Methods 0.000 claims description 26
- 230000014509 gene expression Effects 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims 1
- 238000012986 modification Methods 0.000 abstract description 4
- 230000004048 modification Effects 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to a log acquisition method, a log acquisition device, computer equipment and a storage medium. The method comprises the following steps: when the service instance registration information is monitored, extracting a service host address from the service instance registration information; when the log is required to be acquired according to the address of the service host, acquiring a log file path and an acquisition index; acquiring logs according to the log file path to obtain service instance logs; obtaining service index information according to the acquired index and the service instance log; and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system. By adopting the method, the service monitoring system can acquire the service index information of the required service instance log under the condition of avoiding a great deal of modification of the service monitoring system, and the log monitoring efficiency can be effectively improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a log collection method, a log collection device, a computer device, and a storage medium.
Background
With the development of computer technology, open-source service monitoring systems, such as promethaus, have emerged. The service monitoring system supports automatic pulling of service index information and active pushing of service index information by services.
In the conventional technology, the application program usually outputs error information through a log, but the service monitoring system does not support the matching monitoring of the application log, and when log monitoring is required, all the service exposure service index information in the service monitoring system is often required to be modified.
However, in the conventional method, the workload of modifying all service exposure service index information in the service monitoring system is large, and the problems of complicated modification and low log monitoring efficiency exist.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a log collection method, apparatus, computer device, and storage medium that can avoid a great deal of modification to a service monitoring system and improve log monitoring efficiency.
A log collection method, the method comprising:
when the service instance registration information is monitored, extracting a service host address from the service instance registration information;
when the log is required to be acquired according to the address of the service host, acquiring a log file path and an acquisition index;
acquiring logs according to the log file path to obtain service instance logs;
obtaining service index information according to the acquired index and the service instance log;
and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system.
In one embodiment, when the service instance registration information is monitored, before the service host address is extracted from the service instance registration information, the method further comprises:
a service discovery interface linking the service platform;
and monitoring the service platform to inform the service instance of the registration information through the service discovery interface.
In one embodiment, before linking the service discovery interface, the method further comprises:
and responding to log acquisition configuration operation, mounting application program logs in each container, and declaring an information acquisition mode, wherein the information acquisition mode is to acquire service index information of a service monitoring system.
In one embodiment, further comprising:
acquiring a local address;
comparing the local address with the address of the host;
when the local address is included in the service host address, it is determined that the log needs to be collected.
In one embodiment, when it is determined that the log needs to be collected according to the address of the service host, obtaining the log file path and the collection index includes:
when the log is determined to be required to be acquired according to the address of the service host, acquiring a service instance identifier from the service instance registration information;
sending a service instance configuration information acquisition request to a preset log monitoring management platform, wherein the service instance configuration information acquisition request carries a service instance identifier;
and receiving feedback service instance configuration information, wherein the service instance configuration information comprises a log file path and acquisition indexes.
In one embodiment, obtaining service index information from the collection index and the service instance log includes:
acquiring an acquisition regular expression corresponding to the acquisition index according to the acquisition index;
and identifying and matching the service instance log according to the acquired regular expression to obtain service index information.
In one embodiment, after obtaining the service index information according to the collection index and the service instance log, the method further comprises:
exposing the target interface, and waiting for the service monitoring system to acquire service index information through the target interface.
A log collection device, the device comprising:
the extraction module is used for extracting the address of the service host from the service instance registration information when the service instance registration information is monitored;
the acquisition module is used for acquiring a log file path and acquisition indexes when the log is determined to be acquired according to the address of the service host;
the first processing module is used for acquiring logs according to the log file path to obtain service instance logs;
the second processing module is used for obtaining service index information according to the acquisition index and the service instance log;
and the transmission module is used for transmitting the service index information to the service monitoring system when the information acquisition prompt is received.
A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:
when the service instance registration information is monitored, extracting a service host address from the service instance registration information;
when the log is required to be acquired according to the address of the service host, acquiring a log file path and an acquisition index;
acquiring logs according to the log file path to obtain service instance logs;
obtaining service index information according to the acquired index and the service instance log;
and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
when the service instance registration information is monitored, extracting a service host address from the service instance registration information;
when the log is required to be acquired according to the address of the service host, acquiring a log file path and an acquisition index;
acquiring logs according to the log file path to obtain service instance logs;
obtaining service index information according to the acquired index and the service instance log;
and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system.
According to the log acquisition method, the device, the computer equipment and the storage medium, the service node change can be perceived in real time by monitoring the service instance registration information, when the service instance registration information is monitored, the service host address is extracted from the service instance registration information, when the log to be acquired is determined according to the service host address, the log file path and the acquisition index are acquired, the log is acquired according to the log file path, the service instance log is obtained, the acquisition of the service instance log is realized, and further, the service index information is obtained according to the acquisition index and the service instance log, and when the information acquisition prompt is received, the service index information is transmitted to the service monitoring system.
Drawings
FIG. 1 is a flowchart of a log collection method according to an embodiment;
FIG. 2 is a flowchart of a log collection method according to another embodiment;
FIG. 3 is a diagram of an application environment of a log collection method in one embodiment;
FIG. 4 is a block diagram of a log collection device according to one embodiment;
fig. 5 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In one embodiment, as shown in fig. 1, a log collection method is provided, where the method is applied to a terminal to illustrate the log collection method, it is understood that the method may also be applied to a server, and may also be applied to a system including the terminal and the server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:
step 102, when the service instance registration information is monitored, the service host address is extracted from the service instance registration information.
The service instance registration information refers to information related to deployed service instances, which is issued by the service platform after the service instance deployment is completed. For example, when the service platform is kubernetes, the service instance registration information may specifically refer to information related to the deployed service instance issued by kubernetes after the service instance deployment is completed. The service instance registration information includes a container in which the service instance is located and a corresponding service host address. The service host address refers to an address of a host that can provide a service corresponding to a service instance. For example, the service host address may specifically refer to an IP (Internet Protocol ) address. The container in which the service instance is located refers to a container that can provide a service corresponding to the service instance.
Specifically, the terminal has previously accessed to a service discovery interface exposed by the service platform, monitors service instance registration information through the service discovery interface, and when the service instance is deployed, the service platform notifies the service instance registration information corresponding to the service instance through the service discovery interface, the terminal accessed to the service discovery interface monitors the service instance registration information, and when the service instance registration information is monitored, the terminal extracts a service host address from the service instance registration information, and judges whether log acquisition is needed or not through the service host address. Furthermore, a log file reading tool is arranged on the terminal, and in this embodiment, the interaction between the log file reading tool and the service platform is actually realized.
And 104, acquiring a log file path and acquisition indexes when the need to acquire the log is determined according to the address of the service host.
The log file path refers to a storage path of a service instance log corresponding to the service instance. The acquisition index refers to an index item which corresponds to the service instance and needs to be acquired. For example, the collection index may be specifically occupied memory of the application program, average interface time consumption, and the like.
Specifically, after obtaining the address of the service host, the terminal needs to determine whether to collect the log according to the address of the service host, wherein the address of the service host includes addresses of all hosts capable of providing services corresponding to the service instance, the terminal needs to obtain a local address, compares the local address with the address of the service host, determines whether the terminal belongs to the host capable of providing services corresponding to the service instance, and only when the terminal belongs to the host capable of providing services corresponding to the service instance, the terminal needs to collect the log. After determining that the log needs to be collected, the terminal needs to obtain a log file path and a collection index so as to collect the log and obtain service index information. The method comprises the steps that a log file path and an acquisition index corresponding to each service instance are stored in a preset log monitoring management platform in advance, and a terminal needs to acquire the log file path and the acquisition index through interaction with the preset log monitoring management platform.
And step 106, acquiring the logs according to the log file paths to obtain service instance logs.
The service instance log is used for recording the running condition of the service instance.
Specifically, after obtaining the log file path and the collection index, the terminal determines the storage path of the service instance log in itself according to the log file path, and collects the log according to the storage path to obtain the service instance log.
And step 108, obtaining service index information according to the acquisition index and the service instance log.
The service index information refers to index information which is acquired from the service instance log and corresponds to the acquired index. For example, when the acquisition index is the application memory occupation, the service index information may be specifically that the application memory occupation is X%. For another example, when the collection index is the average consumption of the interface, the service index information may be specifically that the average consumption of the interface is Y minutes.
Specifically, after the service instance log is obtained, the terminal further extracts service index information from the service instance log according to the acquisition index.
Step 110, when receiving the information acquisition prompt, transmitting the service index information to the service monitoring system.
The information acquisition prompt refers to a prompt sent by the service monitoring system and needing information acquisition.
Specifically, after the service index information is obtained, the terminal exposes a target interface linked with the service monitoring system, waits for the service monitoring system to collect the service index information, and transmits the service index information to the service monitoring system when receiving an information collection prompt sent by the service monitoring system, so that the service monitoring system collects the service index information of a required service instance log. Further, after the service index information is transmitted to the service monitoring system, the terminal continuously monitors the service instance registration information, and when the new service instance registration information is monitored again, the steps are repeated to obtain the service index information corresponding to the new service instance registration information, and the service monitoring system waits for acquisition.
According to the log acquisition method, the service node change can be perceived in real time by monitoring the service instance registration information, when the service instance registration information is monitored, the service host address is extracted from the service instance registration information, when the log is required to be acquired according to the service host address, the log file path and the acquisition index are acquired, the log is acquired according to the log file path, the service instance log is obtained, the acquisition of the service instance log is realized, the service index information is further obtained according to the acquisition index and the service instance log, and when the information acquisition prompt is received, the service index information is transmitted to the service monitoring system.
In one embodiment, when the service instance registration information is monitored, before the service host address is extracted from the service instance registration information, the method further comprises:
a service discovery interface linking the service platform;
and monitoring the service platform to inform the service instance of the registration information through the service discovery interface.
Specifically, the service discovery interface is an interface exposed by the service platform and used for service discovery by a log file reading tool in the terminal, the log file reading tool in the terminal can monitor service instance registration information of the service platform through linking the service discovery interface, and after service instance deployment is completed, the service platform notifies the service instance registration information through the service discovery interface, so that the log file reading tool in the terminal monitors the service instance registration information and receives the service instance registration information through the service discovery interface.
In this embodiment, the service discovery interface of the service platform is linked, and the service platform is monitored through the service discovery interface to notify the service instance registration information, so that monitoring of the service instance registration information can be achieved.
In one embodiment, before linking the service discovery interface, the method further comprises:
and responding to log acquisition configuration operation, mounting application program logs in each container, and declaring an information acquisition mode, wherein the information acquisition mode is to acquire service index information of a service monitoring system.
The log acquisition configuration operation refers to an operation of configuring an acquisition environment by a user before log acquisition is performed. For example, the log collection configuration operation may specifically be a configuration operation performed by a user before log collection is performed. A container refers to a container that can provide various services, and includes a plurality of applications therein. The information collection mode refers to a mode of collecting service index information, and in this embodiment, it is stated that the service monitoring system is required to collect service index information.
Specifically, in the log collection configuration operation, the catalogue of each container is mapped to the terminal, so that the terminal can mount the application log in each container. Meanwhile, in log acquisition configuration operation, an information acquisition mode is declared, wherein the information acquisition mode is mainly realized by pre-configuring a target interface, and when the target interface is pre-configured, the port type, the interface path and the like of the target interface are set. For example, when the target interface is preconfigured, the information collection mode may be set to be that the service monitoring system needs to pull, that is, waiting for the service monitoring system to pull the service index information, and setting the port type to be an http (hypertext transfer protocol) port, the interface path to be 4555, where the port type and the interface path form an interface address linked with the service monitoring system.
In this embodiment, by responding to the log collection configuration operation, the log of the application program in each container is mounted, and the information collection mode is declared, so that the log collection configuration can be realized.
In one embodiment, further comprising:
acquiring a local address;
comparing the local address with the address of the host;
when the local address is included in the service host address, it is determined that the log needs to be collected.
The local address refers to the address of the terminal itself. For example, the local address may specifically refer to the terminal IP address.
Specifically, after the service host address is obtained, the terminal obtains the local address in order to determine whether the log needs to be collected, compares the local address with the service host address, when the service host address includes the local address, indicates that the terminal is a host capable of providing service corresponding to the service instance, and therefore needs to collect the log, when the service host address does not include the local address, indicates that the terminal is not a host capable of providing service corresponding to the service instance, and therefore does not need to collect the log, the terminal does not perform log collection operation, continues to monitor the service instance registration information, and when new service instance registration information is monitored, judges whether the log needs to be collected according to the new service instance registration information again.
In this embodiment, by acquiring the local address, comparing the local address with the address of the service host, when the service host address includes the local address, it is determined that the log needs to be acquired, and accurate determination of whether the log needs to be acquired can be achieved through the address.
In one embodiment, when it is determined that the log needs to be collected according to the address of the service host, obtaining the log file path and the collection index includes:
when the log is determined to be required to be acquired according to the address of the service host, acquiring a service instance identifier from the service instance registration information;
sending a service instance configuration information acquisition request to a preset log monitoring management platform, wherein the service instance configuration information acquisition request carries a service instance identifier;
and receiving feedback service instance configuration information, wherein the service instance configuration information comprises a log file path and acquisition indexes.
The service instance identifiers are identifiers for distinguishing different service instances and correspond to the service instances one by one. For example, the service instance identification may specifically be a string used to characterize the service instance. The service instance configuration information refers to configuration of the service instance in advance, and comprises a log file path of the service instance, an acquisition index and the like, wherein the log file path refers to a storage path of the service instance log in the host machine, and the acquisition index refers to a service index required to be acquired for the service instance. The preset log monitoring management platform is a preset platform for managing log monitoring, service instance configuration information corresponding to each service instance is stored in the preset log monitoring management platform, and further, the service instance configuration information corresponding to each service instance can be changed by operating on the preset log monitoring management platform.
Specifically, when the need for collecting logs is determined according to the address of the service host machine, the terminal acquires service instance identifiers from the service instance registration information, generates a service instance configuration information acquisition request carrying the service instance identifiers, sends the service instance configuration information acquisition request to a preset log monitoring management platform, so that the preset log monitoring management platform can find out corresponding service instance configuration information according to the service instance identifiers carried by the service instance configuration information acquisition request after receiving the service instance configuration information acquisition request, feeds back the service instance configuration information to the terminal, and accordingly the terminal receives the fed back service instance configuration information, wherein the service instance configuration information comprises a log file path and acquisition indexes, and the terminal can acquire service index information according to the log file path and the acquisition indexes.
In this embodiment, the service instance identifier is obtained from the service instance registration information, a service instance configuration information obtaining request is sent to the preset log monitoring management platform, the service instance configuration information obtaining request carries the service instance identifier, and the feedback service instance configuration information is received, so that the acquisition of the log file path and the acquisition index can be realized.
In one embodiment, obtaining service index information from the collection index and the service instance log includes:
acquiring an acquisition regular expression corresponding to the acquisition index according to the acquisition index;
and identifying and matching the service instance log according to the acquired regular expression to obtain service index information.
The regular expression refers to a text pattern composed of characters, and describes a character string matching pattern, which can be used for checking whether a character string contains a certain sub-string, replacing the matched sub-string or extracting the sub-string meeting a certain condition from the certain character string, and the like. In this embodiment, the collecting regular expression is mainly used for extracting service index information meeting the conditions from the service instance log.
Specifically, the acquisition regular expressions corresponding to the acquisition indexes are preset and stored, and the acquisition regular expressions are not completely identical for different acquisition indexes, so that the terminal needs to acquire the acquisition regular expressions corresponding to the acquisition indexes according to the acquisition indexes, then identify and match the service instance logs according to the acquisition regular expressions, and extract the service index information from the service instance logs. The acquisition regular expressions corresponding to the acquisition indexes can be stored in a preset database in advance, and the terminal can acquire the acquisition regular expressions corresponding to the acquisition indexes from the preset database according to the acquisition indexes.
In this embodiment, by acquiring the acquisition regular expression corresponding to the acquisition index according to the acquisition index, and identifying and matching the service instance log according to the acquisition regular expression, service index information is obtained, and the acquisition of the service index information can be achieved by using the acquisition regular expression.
In one embodiment, after obtaining the service index information according to the collection index and the service instance log, the method further comprises:
exposing the target interface, and waiting for the service monitoring system to acquire service index information through the target interface.
The target interface is a preset interface which can be linked with the service monitoring system. For example, the target interface may specifically be a preset http interface.
Specifically, after the service index information is obtained, the terminal exposes the target interface, waits for the service monitoring system to collect the service index information through the target interface, and transmits the service index information to the service monitoring system when receiving an information collection prompt sent by the service monitoring system.
In this embodiment, by exposing the target interface, the service monitoring system waits for the service index information to be collected through the target interface, so that the service monitoring system can collect the service index information.
In one embodiment, as shown in fig. 2, the log collection method of the present application is illustrated by one embodiment, and includes the steps of:
step 202, responding to log acquisition configuration operation, mounting application program logs in each container, and declaring an information acquisition mode, wherein the information acquisition mode is to acquire service index information of a service monitoring system;
step 204, linking the service discovery interface of the service platform;
step 206, monitoring the service platform to inform the service instance registration information through the service discovery interface;
step 208, when the service instance registration information is monitored, extracting the address of the service host from the service instance registration information;
step 210, obtaining a local address;
step 212, comparing the local address with the host address;
step 214, when the service host address includes a local address, determining that a log needs to be collected;
step 216, when the log is determined to be required to be acquired according to the address of the service host, acquiring a service instance identifier from the service instance registration information;
step 218, sending a service instance configuration information acquisition request to a preset log monitoring management platform, wherein the service instance configuration information acquisition request carries a service instance identifier;
step 220, receiving feedback service instance configuration information, wherein the service instance configuration information comprises a log file path and acquisition indexes;
step 222, collecting logs according to the log file path to obtain a service instance log;
step 224, acquiring an acquisition regular expression corresponding to the acquisition index according to the acquisition index;
step 226, identifying and matching the service instance log according to the acquired regular expression to obtain service index information;
step 228, exposing the target interface, waiting for the service monitoring system to collect service index information through the target interface;
and 230, transmitting the service index information to the service monitoring system when the information acquisition prompt is received.
The application also provides an application scene, as shown in fig. 3, wherein the application scene applies the log acquisition method. Specifically, the log collection method is applied to the application scene as follows:
in the application scenario, the application scenario comprises a plurality of host terminals 302, a promethaus (service monitoring system) 304, kuubenes 306 and a preset log monitoring management platform 308, the host terminals respond to log acquisition configuration operation, mount application program logs in each container, and declare an information acquisition mode, the information acquisition mode is that the promethaus is required to acquire service index information, a service discovery interface linked with kuubenes, monitor kuubenes through the service discovery interface, notify service instance registration information, extract a service host address from the service instance registration information when the service instance registration information is monitored, acquire a local address, compare the local address with the service host address, determine that a log needs to be acquired when the service host address comprises the local address, acquire a service instance identifier from the service instance registration information when the log needs to be acquired according to the determination of the service host address, send a service instance configuration information acquisition request to the preset log monitoring management platform, the service instance configuration information acquisition request carries the service instance identifier, and the received service instance configuration information comprises a file path and a regular file, and the service instance configuration information is matched with the file, and the service instance index information is required to acquire the service instance index information when the regular file is required to acquire the target index information according to the acquisition log, and the request is required to acquire the acquisition log, and the regular file is required to acquire the target index information when the acquisition log is required to acquire the target index according to the acquisition log. Furthermore, the Prometheus can realize an alarm function, a user can perform alarm configuration in advance, a service index threshold value of each service index is set, an alarm is performed when the acquired service index information is larger than the corresponding service index threshold value, and the user is informed to process, wherein the alarm configuration is carried under a program catalog of the Prometheus.
It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages performed is not necessarily sequential, but may be performed alternately or alternately with at least a part of the steps or stages in other steps or other steps.
In one embodiment, as shown in fig. 4, there is provided a log collection device, including: an extraction module 402, an acquisition module 404, a first processing module 406, a second processing module 408, and a transmission module 410, wherein:
an extracting module 402, configured to extract, when the service instance registration information is monitored, a service host address from the service instance registration information;
an obtaining module 404, configured to obtain a log file path and an acquisition index when it is determined that the log needs to be acquired according to the address of the service host;
the first processing module 406 is configured to perform log collection according to the log file path to obtain a service instance log;
a second processing module 408, configured to obtain service index information according to the collection index and the service instance log;
the transmission module 410 is configured to transmit the service indicator information to the service monitoring system when the information acquisition prompt is received.
According to the log acquisition device, the service node change can be perceived in real time by monitoring the service instance registration information, when the service instance registration information is monitored, the service host address is extracted from the service instance registration information, when the log is required to be acquired according to the service host address, the log file path and the acquisition index are acquired, the log is acquired according to the log file path, the service instance log is obtained, the acquisition of the service instance log is realized, the service index information is further obtained according to the acquisition index and the service instance log, and when the information acquisition prompt is received, the service index information is transmitted to the service monitoring system.
In one embodiment, the log collection device further includes a link module, where the link module is configured to link a service discovery interface of the service platform, and monitor, through the service discovery interface, service instance registration information notified by the service platform.
In one embodiment, the log collection device further includes a configuration module, where the configuration module is configured to respond to log collection configuration operation, mount the application logs in each container, and declare an information collection mode, where the information collection mode is to collect service index information of the service monitoring system.
In one embodiment, the log collection device further includes a judging module, where the judging module is configured to obtain a local address, compare the local address with the address of the service host, and determine that the log needs to be collected when the address of the service host includes the local address.
In one embodiment, the obtaining module is further configured to obtain a service instance identifier from the service instance registration information when it is determined that the log needs to be collected according to the address of the service host, send a service instance configuration information obtaining request to the preset log monitoring management platform, where the service instance configuration information obtaining request carries the service instance identifier, receive feedback service instance configuration information, and the service instance configuration information includes a log file path and a collection index.
In one embodiment, the second processing module is further configured to obtain, according to the collection index, a collection regular expression corresponding to the collection index, and identify and match the service instance log according to the collection regular expression, so as to obtain service index information.
In one embodiment, the transmission module is further configured to expose the target interface, and wait for the service monitoring system to collect service index information through the target interface.
For specific limitations of the log collection device, reference may be made to the above limitation of the log collection method, and no further description is given here. The above-mentioned various modules in the log collection device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a log collection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 5 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
when the service instance registration information is monitored, extracting a service host address from the service instance registration information;
when the log is required to be acquired according to the address of the service host, acquiring a log file path and an acquisition index;
acquiring logs according to the log file path to obtain service instance logs;
obtaining service index information according to the acquired index and the service instance log;
and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system.
In one embodiment, the processor when executing the computer program further performs the steps of: and the service discovery interface is linked with the service platform, and the service platform is monitored through the service discovery interface to inform the service instance of registration information.
In one embodiment, the processor when executing the computer program further performs the steps of: and responding to log acquisition configuration operation, mounting application program logs in each container, and declaring an information acquisition mode, wherein the information acquisition mode is to acquire service index information of a service monitoring system.
In one embodiment, the processor when executing the computer program further performs the steps of: and acquiring the local address, comparing the local address with the address of the service host, and determining that the log needs to be acquired when the local address is included in the address of the service host.
In one embodiment, the processor when executing the computer program further performs the steps of: when the need of collecting logs is determined according to the address of the service host machine, a service instance identifier is obtained from the service instance registration information, a service instance configuration information obtaining request is sent to a preset log monitoring management platform, the service instance configuration information obtaining request carries the service instance identifier, the service instance configuration information fed back is received, and the service instance configuration information comprises a log file path and collection indexes.
In one embodiment, the processor when executing the computer program further performs the steps of: acquiring an acquisition regular expression corresponding to the acquisition index according to the acquisition index, and identifying and matching the service instance log according to the acquisition regular expression to obtain service index information.
In one embodiment, the processor when executing the computer program further performs the steps of: exposing the target interface, and waiting for the service monitoring system to acquire service index information through the target interface.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
when the service instance registration information is monitored, extracting a service host address from the service instance registration information;
when the log is required to be acquired according to the address of the service host, acquiring a log file path and an acquisition index;
acquiring logs according to the log file path to obtain service instance logs;
obtaining service index information according to the acquired index and the service instance log;
and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system.
In one embodiment, the computer program when executed by the processor further performs the steps of: and the service discovery interface is linked with the service platform, and the service platform is monitored through the service discovery interface to inform the service instance of registration information.
In one embodiment, the computer program when executed by the processor further performs the steps of: and responding to log acquisition configuration operation, mounting application program logs in each container, and declaring an information acquisition mode, wherein the information acquisition mode is to acquire service index information of a service monitoring system.
In one embodiment, the computer program when executed by the processor further performs the steps of: and acquiring the local address, comparing the local address with the address of the service host, and determining that the log needs to be acquired when the local address is included in the address of the service host.
In one embodiment, the computer program when executed by the processor further performs the steps of: when the need of collecting logs is determined according to the address of the service host machine, a service instance identifier is obtained from the service instance registration information, a service instance configuration information obtaining request is sent to a preset log monitoring management platform, the service instance configuration information obtaining request carries the service instance identifier, the service instance configuration information fed back is received, and the service instance configuration information comprises a log file path and collection indexes.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring an acquisition regular expression corresponding to the acquisition index according to the acquisition index, and identifying and matching the service instance log according to the acquisition regular expression to obtain service index information.
In one embodiment, the computer program when executed by the processor further performs the steps of: exposing the target interface, and waiting for the service monitoring system to acquire service index information through the target interface.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (10)
1. A log collection method, the method comprising:
a service discovery interface linking the service platform;
monitoring the service platform notification service instance registration information through the service discovery interface;
when the service instance registration information is monitored, extracting a service host address from the service instance registration information;
when the log is required to be acquired according to the address of the service host machine, acquiring a log file path and an acquisition index; the acquisition index comprises at least one of memory occupation of an application program and average interface time consumption;
acquiring logs according to the log file path to obtain service instance logs;
obtaining service index information according to the acquisition index and the service instance log;
and when the information acquisition prompt is received, transmitting the service index information to a service monitoring system.
2. The method of claim 1, wherein prior to linking the service discovery interface, further comprising:
and responding to log acquisition configuration operation, mounting application program logs in each container, and declaring an information acquisition mode, wherein the information acquisition mode is to acquire service index information of a service monitoring system.
3. The method as recited in claim 1, further comprising:
acquiring a local address;
comparing the local address with the service host address;
and when the service host address comprises a local address, determining that the log needs to be acquired.
4. The method of claim 1, wherein when it is determined that the log is required to be collected according to the service hosting address, the obtaining the log file path and the collection index includes:
when the log is determined to be required to be acquired according to the address of the service host, acquiring a service instance identifier from the service instance registration information;
sending a service instance configuration information acquisition request to a preset log monitoring management platform, wherein the service instance configuration information acquisition request carries the service instance identifier;
and receiving feedback service instance configuration information, wherein the service instance configuration information comprises a log file path and acquisition indexes.
5. The method of claim 1, wherein obtaining service indicator information from the collection indicator and the service instance log comprises:
acquiring an acquisition regular expression corresponding to the acquisition index according to the acquisition index;
and identifying and matching the service instance log according to the acquisition regular expression to obtain service index information.
6. The method of claim 1, wherein after obtaining service index information according to the collection index and the service instance log, further comprises:
exposing a target interface, and waiting for a service monitoring system to acquire the service index information through the target interface.
7. A log collection device, the device comprising:
the link module is used for linking a service discovery interface of the service platform, and monitoring the service platform notification service instance registration information through the service discovery interface;
the extraction module is used for extracting the address of the service host from the service instance registration information when the service instance registration information is monitored;
the acquisition module is used for acquiring a log file path and acquisition indexes when the log is determined to be acquired according to the address of the service host; the acquisition index comprises at least one of memory occupation of an application program and average interface time consumption;
the first processing module is used for acquiring logs according to the log file path to obtain service instance logs;
the second processing module is used for obtaining service index information according to the acquisition index and the service instance log;
and the transmission module is used for transmitting the service index information to a service monitoring system when receiving the information acquisition prompt.
8. The apparatus of claim 7, further comprising a configuration module configured to mount the application log in each container in response to a log collection configuration operation and declare an information collection manner, wherein the information collection manner is a manner in which service indicator information is collected by a service monitoring system.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011510561.8A CN112506755B (en) | 2020-12-18 | 2020-12-18 | Log acquisition method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011510561.8A CN112506755B (en) | 2020-12-18 | 2020-12-18 | Log acquisition method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112506755A CN112506755A (en) | 2021-03-16 |
| CN112506755B true CN112506755B (en) | 2023-11-03 |
Family
ID=74922863
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011510561.8A Active CN112506755B (en) | 2020-12-18 | 2020-12-18 | Log acquisition method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112506755B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114363403B (en) * | 2021-12-28 | 2024-09-06 | 金医慧通科技有限公司 | Service access method, system, computer device and storage medium |
| CN114500254B (en) * | 2022-02-16 | 2023-08-22 | 平安科技（深圳）有限公司 | Historical log collection method, device and storage medium |
| CN116225854A (en) * | 2023-05-05 | 2023-06-06 | 北京明易达科技股份有限公司 | Method, system, medium and equipment for automatically collecting server log |
| CN116560949B (en) * | 2023-07-10 | 2023-09-29 | 南瑞轨道交通技术有限公司 | Method, system, equipment and storage medium for realizing data acquisition and reception |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108845917A (en) * | 2018-04-09 | 2018-11-20 | 东峡大通（北京）管理咨询有限公司 | File journalization carry module, system and method in container |
| CN109669848A (en) * | 2017-10-16 | 2019-04-23 | 比亚迪股份有限公司 | Log generation method, application server, log server and log generating system |
| CN109684038A (en) * | 2018-12-18 | 2019-04-26 | 网易（杭州）网络有限公司 | Processing method, device and the electronic equipment of Docker service container log |
| CN110990178A (en) * | 2019-11-27 | 2020-04-10 | 深圳前海微众银行股份有限公司 | Keyword collection monitoring method, device and equipment and computer storage medium |
| CN111258722A (en) * | 2020-02-14 | 2020-06-09 | 苏州浪潮智能科技有限公司 | Cluster log acquisition method, system, device and medium |
| CN111367615A (en) * | 2018-12-26 | 2020-07-03 | 卓望数码技术（深圳）有限公司 | Method, apparatus, device and storage medium facilitating container instance scheduling |
| CN111475468A (en) * | 2020-03-05 | 2020-07-31 | 中国平安人寿保险股份有限公司 | Log access method, device, equipment and storage medium of newly added system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020099813A1 (en) * | 2000-12-04 | 2002-07-25 | Jason Winshell | Method for collecting statistics about Web site usage |
-
2020
- 2020-12-18 CN CN202011510561.8A patent/CN112506755B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109669848A (en) * | 2017-10-16 | 2019-04-23 | 比亚迪股份有限公司 | Log generation method, application server, log server and log generating system |
| CN108845917A (en) * | 2018-04-09 | 2018-11-20 | 东峡大通（北京）管理咨询有限公司 | File journalization carry module, system and method in container |
| CN109684038A (en) * | 2018-12-18 | 2019-04-26 | 网易（杭州）网络有限公司 | Processing method, device and the electronic equipment of Docker service container log |
| CN111367615A (en) * | 2018-12-26 | 2020-07-03 | 卓望数码技术（深圳）有限公司 | Method, apparatus, device and storage medium facilitating container instance scheduling |
| CN110990178A (en) * | 2019-11-27 | 2020-04-10 | 深圳前海微众银行股份有限公司 | Keyword collection monitoring method, device and equipment and computer storage medium |
| CN111258722A (en) * | 2020-02-14 | 2020-06-09 | 苏州浪潮智能科技有限公司 | Cluster log acquisition method, system, device and medium |
| CN111475468A (en) * | 2020-03-05 | 2020-07-31 | 中国平安人寿保险股份有限公司 | Log access method, device, equipment and storage medium of newly added system |
Non-Patent Citations (1)
| Title |
|---|
| "移动服务端运行环境搭建的研发与应用";冯小宸;《中国优秀硕士学位论文全文数据库 信息科技辑》(第02期);I138-1922 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112506755A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112506755B (en) | Log acquisition method, device, computer equipment and storage medium | |
| CN111522922B (en) | Log information query method and device, storage medium and computer equipment | |
| CN106489251B (en) | The methods, devices and systems of applied topology relationship discovery | |
| CN108959385B (en) | Database deployment method, device, computer equipment and storage medium | |
| CN109587258B (en) | Service activity detection method and device | |
| CN105262608B (en) | Monitoring method and device for network service | |
| CN109039787B (en) | Log processing method and device and big data cluster | |
| CN110784374A (en) | Method, device, equipment and system for monitoring operation state of service system | |
| CN103095821B (en) | Continuous auditing system based on virtual machine migration recognition | |
| CN112965879A (en) | Data processing method and device, electronic equipment and readable storage medium | |
| CN114745295A (en) | Data acquisition method, device, equipment and readable storage medium | |
| CN106961361A (en) | From detection/processing method, apparatus and system | |
| CN112866348A (en) | Database access method and device, computer equipment and storage medium | |
| KR20190021848A (en) | Method for monitoring server, apparatus and system for executing the method | |
| CN113849356A (en) | Equipment testing method and device, electronic equipment and storage medium | |
| CN115705190A (en) | Method and device for determining dependence degree | |
| CN112579406B (en) | Log call chain generation method and device | |
| CN112667512A (en) | Data drive test method, device, equipment and computer readable storage medium | |
| CN105988917B (en) | Abnormal information acquisition method and device | |
| CN113778709A (en) | Interface calling method, device, server and storage medium | |
| CN113297149A (en) | Method and device for monitoring data processing request | |
| CN109684158B (en) | State monitoring method, device, equipment and storage medium of distributed coordination system | |
| CN111782428A (en) | Data calling system and method | |
| CN114884807B (en) | Link log generation method and device, internet of things platform and storage medium | |
| CN115529250B (en) | Flow playback method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |