CN114153703A - Micro-service exception positioning method and device, electronic equipment and program product - Google Patents

Micro-service exception positioning method and device, electronic equipment and program product Download PDF

Info

Publication number
CN114153703A
CN114153703A CN202111494911.0A CN202111494911A CN114153703A CN 114153703 A CN114153703 A CN 114153703A CN 202111494911 A CN202111494911 A CN 202111494911A CN 114153703 A CN114153703 A CN 114153703A
Authority
CN
China
Prior art keywords
log file
call
microservice
call request
log
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111494911.0A
Other languages
Chinese (zh)
Inventor
周洋
李玉
方航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
Original Assignee
China Construction Bank Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp filed Critical China Construction Bank Corp
Priority to CN202111494911.0A priority Critical patent/CN114153703A/en
Publication of CN114153703A publication Critical patent/CN114153703A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • G06F16/134Distributed indices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The present disclosure provides a method for locating an anomaly of a microservice, the method comprising: acquiring a log file corresponding to a call request based on the call request sent by at least one microservice, wherein the log file comprises call information of the call request; establishing an index library in real time based on the log file; obtaining the log file with the indexed repository query to locate a calling exception for the at least one microservice based on the calling information. The disclosure also provides a microservice anomaly locating device, an electronic device and a program product.

Description

Micro-service exception positioning method and device, electronic equipment and program product
Technical Field
The present disclosure relates to the field of micro-service technology or the field of financial technology, and more particularly, to a method and an apparatus for locating an anomaly of a micro-service, an electronic device, and a program product.
Background
With the rapid development of services, more and more applications are changed from a single-body architecture to a micro-service architecture. In a microservice architecture, a user request often involves the cooperation of multiple modules, multiple middleware, and multiple machines to be fulfilled by one or more microservices issuing a series of call requests. In the related technology, a log is collected in the micro-service calling process, a log file is firstly stored in a disk, and then the stored log file is obtained from the disk to analyze the performance of a micro-service framework or perform abnormal positioning on the micro-service.
In the course of implementing the disclosed concept, the inventors found that there are at least the following problems in the prior art:
when the micro-service is abnormal, the log file is stored in a disk and then is processed and analyzed, so that the log file cannot be accurately obtained at the first time, the abnormal positioning speed is low, and the efficiency is low.
Disclosure of Invention
In view of the above, the present disclosure provides a microservice anomaly locating method, apparatus, electronic device, and program product that improve microservice anomaly locating efficiency.
One aspect of the embodiments of the present disclosure provides a method for locating an exception of a microservice, including: acquiring a log file corresponding to a call request based on the call request sent by at least one microservice, wherein the log file comprises call information of the call request; establishing an index library in real time based on the log file; obtaining the log file with the indexed repository query to locate a calling exception for the at least one microservice based on the calling information.
According to an embodiment of the present disclosure, before the creating the index library in real time based on the log file, the storing the log file further includes: processing the log file to obtain a corresponding log message; storing the log message to a message queue; storing the log file by consuming log messages in the message queue.
According to an embodiment of the present disclosure, the obtaining the log file corresponding to the call request includes: intercepting a call request sent by the at least one microservice to generate a call record number; and writing the calling information of the calling request and the calling record number into the log file in a correlated manner.
According to an embodiment of the present disclosure, the intercepting a call request issued by the at least one microservice includes: intercepting a call request issued by the at least one microservice in a non-intrusive code based manner.
According to an embodiment of the present disclosure, the intercepting a call request issued by the at least one microservice to generate a call record number includes: intercepting at least one call request under the same call chain, wherein the call chain comprises a call relation executed by the at least one microservice in response to a user request; and generating the call record number based on the call chain.
According to an embodiment of the present disclosure, the call information includes at least one of: caller, callee, call timestamp, response timestamp, server address, interface name.
According to an embodiment of the present disclosure, after the obtaining the log file corresponding to the invocation request, the method further includes: calculating the time consumed corresponding to the interface name based on the calling timestamp and the response timestamp; and when the consumed time is greater than or equal to a preset threshold value, sending a prompt message.
According to an embodiment of the present disclosure, the prompt information includes the call record number, and the obtaining the log file by querying the index repository includes: matching with index information in the index library based on the calling record number; and acquiring the log file based on the index information successfully matched with the call record number.
Another aspect of the embodiments of the present disclosure provides an exception positioning apparatus for micro services, including a file obtaining module, an index base module, and an exception positioning module. The file acquisition module is used for acquiring a log file corresponding to a call request based on the call request sent by at least one microservice, wherein the log file comprises call information of the call request; the index library module is used for establishing an index library in real time based on the log file; the exception positioning module is used for utilizing the index library to inquire to obtain the log file so as to position the calling exception of the at least one micro service based on the calling information.
Another aspect of the present disclosure provides an electronic device including: one or more processors; a storage device to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method as described above.
Yet another aspect of the disclosed embodiments provides a computer program product comprising a computer program that when executed by a processor implements the method as described above.
One or more of the embodiments described above have the following advantages or benefits: compared with the mode of firstly storing, then processing and analyzing the log files of the micro-service in a disk in the related art, the embodiment of the disclosure establishes the index base in real time based on the acquired log files, so that the index base can be used for providing a function of retrieving the log files in real time, the time difference consumed between the storage of the log files in the disk and the processing and analyzing can be reduced, redundant log processing steps can be omitted, system resources can be saved, the exception handling efficiency can be improved, and the problems of low exception positioning speed and low efficiency of the micro-service can be at least partially solved.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:
fig. 1 schematically illustrates an application scenario diagram of an anomaly location method of a microservice according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a method of anomaly location for microservices, in accordance with an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart for storing the log file according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates an architecture diagram for storing the log file, according to an embodiment of the disclosure;
FIG. 5 schematically shows a flowchart for obtaining a log file corresponding to the invocation request according to an embodiment of the present disclosure;
FIG. 6 schematically shows a flow diagram for generating a call record number according to an embodiment of the disclosure;
FIG. 7 schematically illustrates a flow chart of a method of anomaly location of a microservice according to another embodiment of the present disclosure;
FIG. 8 is a flow chart that schematically illustrates retrieving the log file using the index repository, in accordance with an embodiment of the present disclosure;
FIG. 9 schematically illustrates a block diagram of an anomaly locating device for microservice according to an embodiment of the present disclosure;
FIG. 10 schematically illustrates a block diagram of an electronic device adapted to implement a microservice anomaly location method in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the related art, the log file collected based on the micro service architecture may have a large data volume. In the process of storing the log file to the disk and then taking the log file out of the disk for analysis, on one hand, taking the log file out of the disk may be accompanied by a large amount of data reading operations, which consumes much time. Moreover, the processes of scanning log files stored in a disk out of order according to the classification rules, searching for associated files, and analyzing also take time, so that a user cannot use the log files at the first time. On the other hand, in order to realize the function of extracting the log file from the disk for analysis, a corresponding thread is usually allocated to call a system resource for calculation processing, and the operation of allocating the thread and calculating the processing wastes the system resource.
The embodiment of the disclosure provides a microservice abnormity positioning method, a microservice abnormity positioning device, electronic equipment and a program product, wherein the method comprises the following steps: and acquiring a log file corresponding to the call request based on the call request sent by at least one microservice, wherein the log file comprises the call information of the call request. And establishing an index library in real time based on the log file. Obtaining a log file with an indexed repository query to locate a calling exception for at least one microservice based on the calling information.
Compared with the mode of firstly storing, then processing and analyzing the log files of the micro-services in a disk in the related art, the embodiment of the disclosure establishes the index base in real time based on the acquired log files, so that the function of retrieving the log files in real time can be provided by using the index base, the time difference consumed between the storage of the log files in the disk and the processing and analyzing can be reduced, redundant log processing steps can be omitted, system resources can be saved, the exception handling efficiency can be improved, and the problems of low exception positioning speed and low efficiency of the micro-services can be at least partially solved.
Fig. 1 schematically illustrates an application scenario diagram of an anomaly location method for a microservice according to an embodiment of the present disclosure.
As shown in fig. 1, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, a first server 105 and a second server 106. The first server 105 may include servers 1051, 1052 … … 105N, among others. The network 104 is used to provide a medium of communication links between the terminal devices 101, 102, 103 and the first server 105. In some embodiments, the first server 105 and the second server 106 may also be provided with a network 104 to provide the communication link. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the first server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The first server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device. The second server 106 may be a database server to provide the functionality of building an index base and to provide real-time retrieval functionality. The second server 106 may be the same as the first server 105.
According to an embodiment of the present disclosure, a micro service architecture may be provided in the first server 105, for example, one or more micro services may be deployed at the servers 1051, 1052 … … 105N, respectively, and may be invoked across servers between the respective micro services. A user may send a user request to the first server 105 via the terminal device 101, 102, 103, and the micro service architecture may issue one or more invocation requests in response to the user request to meet the user's requirements. In addition, log information corresponding to each call request may be obtained and stored in the second server 106 to establish an index library. And, the user may interact with the second server 106 through the terminal devices 101, 102, 103 to obtain the target log file based on the index repository query. In some embodiments of the present disclosure, the second server 106 may not be provided, and any server included in the first server 105 may provide a function of establishing an index library, and the present disclosure is not particularly limited.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The method for locating an anomaly of a microservice according to an embodiment of the present disclosure will be described in detail below with reference to fig. 2 to 8 based on the scenario described in fig. 1.
FIG. 2 schematically illustrates a flow chart of a method of anomaly location for microservices according to an embodiment of the disclosure.
As shown in fig. 2, the method for locating an anomaly of a microservice of this embodiment includes operations S210 to S230.
In operation S210, a log file corresponding to a call request is obtained based on the call request issued by at least one microservice, where the log file includes call information of the call request.
Referring to FIG. 1, microservice A may be deployed on server 1051 and microservice B, C may be deployed on server 1052. Upon receiving the user request, a call request a → B may be first sent by microservice a in response to the user request to microservice B to call microservice B to perform the relevant operation. Then, the microservice B may send a call request B → C to the microservice C when processing data, and finally return the processed data. In this case, the log file 1 may be obtained corresponding to the call request a → B, the log file 2 may be obtained corresponding to the call request B → C, or the call information of the call request a → B, the call request B → C may be written in the same log file. In addition, microservice a may invoke microservice B and, at the same time, microservice D on server 1053. The series of call requests are serial and parallel, and the call information can determine which applications, modules, server nodes and call sequence are called behind a certain call request.
In operation S220, an index repository is built in real time based on the log file.
The real-time establishment of the index library comprises, for example, when incremental log files are detected in real time, performing word segmentation on contents in the log files, establishing a Henry word segmentation result and an index of the log files, and finally storing the index into the index library. The index library may be a full-text index library, and the database may include a MySql database, an Elasticsearch database, and the like.
In operation S230, a log file is obtained using an index repository query to locate a calling exception for at least one microservice based on the calling information.
The calling exception can be the exception condition of long calling time, calling failure and the like, can be found through response information returned by the micro service framework which cannot meet the service requirement, and can also be found through data analysis of a log file acquired by an index library. Querying the log file may include the steps of: first, search content transmitted by a user may be received. Then, corresponding index information is determined from the index library based on the retrieval content. And finally, acquiring a corresponding source log file based on the index information.
According to an embodiment of the present disclosure, the call information includes at least one of: caller, callee, call timestamp, response timestamp, server address, interface name. The caller may be a micro-service which sends a call request, the callee may be a micro-service which accepts the call, the call timestamp may be the time when the call request is sent, the response timestamp may be the time when the callee returns response data, the server address may include the IP of the server where the caller and/or callee is located, and the interface name may be the name of an interface exposed by the callee, where the interface is used to receive the call request and return the response data. Taking the above-mentioned call request a → B as an example, the call information may include the caller micro-service a, the callee micro-service B, the address of the server 1051, the address of the server 1052, the time when the call request a → B is issued, the time when the micro-service B responds, and the name of the interface exposed by the micro-service B for interacting with the micro-service a.
According to the embodiment of the disclosure, after the server IP is obtained, the server IP address can be encrypted based on an encryption algorithm, such as an RSA encryption algorithm, and then the ciphertext information is written into the log file, so that the security problem caused by the server IP leakage can be prevented.
Compared with the mode of firstly storing, then processing and analyzing the log files of the micro-services in a disk in the related art, the embodiment of the disclosure establishes the index base in real time based on the acquired log files, so that the function of retrieving the log files in real time can be provided by using the index base, the time difference consumed between the storage of the log files in the disk and the processing and analyzing can be reduced, redundant log processing steps can be omitted, system resources can be saved, the exception handling efficiency can be improved, and the problems of low exception positioning speed and low efficiency of the micro-services can be at least partially solved.
FIG. 3 schematically shows a flow diagram for storing a log file according to an embodiment of the disclosure. FIG. 4 schematically illustrates an architecture diagram of a storage log file according to an embodiment of the disclosure.
As shown in fig. 3, before the operation S220 of creating the index repository in real time based on the log file, the method further includes storing the log file, and specifically may include operations S310 to S330.
In operation S310, the log file is processed to obtain a corresponding log message.
In operation S320, the log message is deposited to the message queue.
Referring to fig. 1 and 4, call information of call request a → B, call request B → C, for example, is written in the same log file. Wherein the log file may be first assigned to a log collection thread pool that may be executed at any of the first servers 105. Then, the log file is processed into a preset format by the thread in the log collection thread pool, so as to obtain the log message. At this point, the log collection thread pool may be taken as a producer. Finally, the log message is deposited by the thread into a message queue.
In operation S330, a log file is stored by consuming the log message in the message queue.
Referring to fig. 1 and 4, message middleware may be provided between a producer and a consumer, the message middleware being used to provide a message queue function. The log messages in the consumption message queue may be one or more log messages that are taken out from the message queue by the second server 106 for processing, for example, the log messages are processed in a reverse format (for example only) to obtain a log file, and then stored in the database and indexed in real time. The log file of the real-time detection increment can comprise log messages newly added in a real-time detection message queue. At this time, the second server 106 may be considered a consumer.
In some embodiments of the present disclosure, the log message in the producer and consumer mode includes content in a log file, and the corresponding file can be obtained by formatting the log message. In some embodiments of the disclosure, the log message may be a log file association message, for example, the log file is first stored in a log server, and the second server 106 retrieves the file from the log server by consuming the association message.
According to the embodiment of the disclosure, the message middleware is arranged to store the log file in an asynchronous mode, so that the coupling between the production log file of the micro-service architecture and the database storage file can be reduced. In addition, the call request in the micro-service architecture is often accompanied by a high concurrency condition, the generated log file also has a large data volume, and the asynchronous processing mode can prevent the database from being crashed due to the fact that the data volume exceeds the processing bottleneck so as to achieve the effect of stable storage.
Fig. 5 schematically shows a flowchart of acquiring a log file corresponding to the call request in operation S210 according to an embodiment of the present disclosure.
As shown in fig. 5, acquiring the log file corresponding to the call request in operation S210 includes operations S510 to S520.
In operation S510, a call request issued by at least one microservice is intercepted to generate a call record number.
According to an embodiment of the present disclosure, intercepting a call request issued by at least one microservice comprises: intercepting a call request issued by at least one microservice in a non-intrusive code based manner.
The microservice architecture can respond to user requests by a series of invocation requests to meet the user's needs. On the basis of executing the series of call requests, the function of intercepting the call requests is added, for example, to code content of the micro service architecture is further added. The non-intrusive code mode represents low intrusiveness to the code of the micro-service architecture, and reduces the complexity of service development. For example, probes may be implanted into the code of the microservice architecture without modification to the code to be monitored. The non-intrusive code approach may be implemented by a Java agent, a Java Proxy agent, or a Cglib (code Generation library) dynamic agent. In other embodiments of the present disclosure, the call request may also be intercepted by an intrusion code, so as to achieve the purpose of collecting the log file as well.
In operation S520, a write log file associating the call information of the call request with the call record number is written.
According to embodiments of the present disclosure, a decorator mode (e.g., Python decorator function) may be employed to associate the call information with the call record number written to the log file. The decorator mode enables an existing function to dynamically add additional functions without any code changes, so that call information generated in the microservice architecture can be printed in a non-intrusive code manner.
According to the embodiment of the disclosure, since one micro service may issue a plurality of call requests, the collected log files may be unordered, and even invalid log files unrelated to the call requests may be collected. Therefore, the log file can be analyzed by calling the record number.
Fig. 6 schematically shows a flowchart of generating a call record number in operation S510 according to an embodiment of the present disclosure.
As shown in fig. 6, intercepting a call request issued by at least one microservice in operation S510 to generate a call record number may include operations S610 to S620.
In operation S610, at least one call request under the same call chain is intercepted, wherein the call chain includes call relations executed by at least one microservice in response to a user request.
Referring to fig. 1, for example, a user request sent by a user is received, and a call request a → B, a call request B → C may be executed to return response data. The call chain responding to the user request comprises a call request A → B and a call request B → C, namely, the call chain comprises a micro service A calling a micro service B, and the micro service B calling the calling relation of the micro service C.
In operation S620, a call record number is generated based on the call chain.
Specifically, generating the call record number based on the call chain may include the steps of:
first, in response to a user request, microservice a is invoked, generating an invocation record number. The call record number may be, for example, a UUID number generated using the UUID algorithm.
Second, microservice a issues a call request a → B in response to a user request. At this time, the call request is intercepted in a Cglib-based dynamic proxy manner. In addition, the write log file 3 associates the call information of the call request a → B with the UUID number. Wherein the call request A → B is sent by microservice A to the exposed interface of microservice B in the form of a URL. Before sending, the UUID number may be assigned to the URL.
Then, the microservice B receives the URL of the call request a → B, can obtain the UUID number therein, and sends the call request B → C to the microservice C in response to the URL. The call request is intercepted, also in a Cglib-based dynamic proxy manner, and the call information and the UUID number are written to the log file 3. And, the UUID number is assigned to the URL of call request B → C and passed to the interface exposed by microservice C.
Then, the microserver C receives the URL of the call request B → C, can acquire the UUID number therein, processes data in response to the URL, and returns first response information. The time at which the first response information is returned may be associated with the call request B → C in the write log file 3 based on the UUID number. When the microservice B returns the second response information to the microservice a based on the first response information, the time at which the second response information is returned is associated with the call request a → B in the write log file 3 also based on the UUID number.
Finally, a log file 3 can be obtained that records log information for the call chain. The contents of the log file 3 are shown in table 1 (for example only).
TABLE 1
Figure BDA0003398689570000111
According to the embodiment of the disclosure, since the same user request may be executed by micro-services located in different servers, divergence of log files may be caused, resulting in difficulty in searching. By generating the call record number based on the call chain, the information such as specific user requests, service types, micro-services and the like related to a series of call requests can be quickly positioned based on the call record number.
FIG. 7 schematically illustrates a flow chart of a method for anomaly location of microservices according to another embodiment of the present disclosure.
As shown in fig. 7, the method for locating an anomaly of a microservice according to this embodiment includes operations S210 to S230, and operations S710 to S720.
In operation S210, a log file corresponding to a call request is obtained based on the call request issued by at least one microservice, where the log file includes call information of the call request.
In operation S220, an index repository is built in real time based on the log file.
In operation S710, a time consumed corresponding to the interface name is calculated based on the call timestamp and the response timestamp.
In operation S720, when the elapsed time is greater than or equal to the preset threshold, the prompt message is transmitted.
Referring to table 1, log information belonging to the same call chain can be extracted by UUID number. And calculates the elapsed time for each piece of call information (e.g., each line of information in table 1). For example, the time consumed by the interface B1 is 6 seconds, and if the preset threshold is 5 seconds, the time consumed by the interface B1 exceeds the preset threshold, and at this time, the prompt message may be sent. Therefore, the interface service which possibly has performance problems can be found in time, and the alarm function of the time-consuming interface service is completed.
In some embodiments, the prompt message may be sent in at least one of a text form or a voice form. When the text form is adopted, a mail server or a short message server can be called, and mails or short messages are sent based on a reserved mailbox or number. And the message pushing service can be called to push messages to the application reserved by the staff. When the voice form is adopted, for example, an outbound call system can be called to make a telephone call to a reserved number, and the voice content is automatically played after the telephone is connected. Warning sounds such as beeps, drip sounds, or long beeps may also be automatically played. The text content or the voice content may be general content, such as "find abnormal interface", or may be detailed information, such as "time consumed by interface B1 exceeds a preset threshold". It should be understood that the text content or the voice content can be set by self, and the disclosure is not limited in particular.
In operation S230, a log file is obtained using an index repository query to locate a calling exception for at least one microservice based on the calling information.
Although the various operations of the methods are described above in a particular order, embodiments of the disclosure are not so limited, and the operations described above may be performed in other orders as desired. For example, operation S220 may be performed after step S710. In some embodiments, the log file may be printed out after the elapsed time is calculated, and then the log file may be stored and the index repository may be built in real time.
According to the embodiment of the disclosure, referring to fig. 1 and 4, in addition to storing the log file into the Elasticsearch database and establishing the index library, messages can be consumed from the message queue through the executable script, the call record number can be extracted according to the time dimension, and after the interface time consumption is calculated, the cache disk (NAS) is set in the form of a file and the log snapshot is stored at regular time, so as to play a backup role. In addition, the NAS may store log files within a certain date, and for an expired log file, the NAS may perform further backup by batching data in the non-relational database hbase (for example only) in units of request count/storage block size and deleting the data in the NAS.
Fig. 8 schematically shows a flowchart of obtaining a log file using index retrieval in operation S230 according to an embodiment of the present disclosure.
As shown in fig. 8, retrieving the log file using the index in operation S230 may include operations S810 to S820. Wherein the prompt message includes a call record number.
In operation S810, matching is performed with index information in an index repository based on the call record number.
In operation S820, a log file is acquired based on the index information successfully matched with the call record number.
Referring to table 1, after the elapsed time of interface B1 exceeds the preset threshold, a prompt message may be issued, where the prompt message includes 123X. Then 123X may be retrieved as a term. After the search term 123X is successfully matched with the index information 123X, the corresponding log file 3 is acquired based on the index information 123X. Micro-services A, B and C can then be located based on the call information and troubleshoot the problems one by one, thereby completing the exception location operation quickly.
Based on the micro-service abnormity positioning method, the disclosure also provides a micro-service abnormity positioning device. The apparatus will be described in detail below with reference to fig. 9.
Fig. 9 schematically shows a block diagram of an anomaly locating device for microservice according to an embodiment of the present disclosure.
As shown in fig. 9, the microservice exception locator 900 of this embodiment includes a file acquisition module 910, an index library module 920, and an exception locator module 930.
The file obtaining module 910 may be configured to perform operation S210, for example, to obtain a log file corresponding to a call request based on the call request issued by at least one microservice, where the log file includes call information of the call request.
The index library module 920 may be used to perform operation S220, for example, for building an index library in real time based on the log file.
The exception locating module 930 may be configured to perform operation S230, for example, for obtaining a log file using an index repository query to locate a calling exception for at least one microservice based on the calling information.
According to the embodiment of the present disclosure, any plurality of modules of the file obtaining module 910, the index library module 920 and the exception positioning module 930 may be combined and implemented in one module, or any one of the modules may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of file retrieving module 910, index base module 920, and exception locator module 930 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware. Alternatively, at least one of file acquisition module 910, index library module 920, and exception locator module 930 may be implemented, at least in part, as a computer program module that, when executed, may perform corresponding functions.
FIG. 10 schematically illustrates a block diagram of an electronic device adapted to implement a microservice anomaly location method in accordance with an embodiment of the present disclosure.
As shown in fig. 10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. Processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the present disclosure.
In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus 1000 are stored. The processor 1001, ROM 1002, and RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1002 and/or the RAM 1003. Note that the program may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in one or more memories.
Electronic device 1000 may also include an input/output (I/O) interface 1005, the input/output (I/O) interface 1005 also being connected to bus 1004, according to an embodiment of the present disclosure. Electronic device 1000 may also include one or more of the following components connected to I/O interface 1005: an input section 1006 including a keyboard, mouse, and the like. Including an output portion 1007 such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker and the like. A storage section 1008 including a hard disk and the like. And a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.
The present disclosure also provides a computer-readable storage medium, which may be embodied in the devices/apparatuses/systems described in the above embodiments. Or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 1002 and/or the RAM 1003 described above and/or one or more memories other than the ROM 1002 and the RAM 1003.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. The program code is for causing a computer system to carry out the method according to the embodiments of the disclosure, when the computer program product is run on the computer system.
The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 1001. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication part 1009, and/or installed from the removable medium 1011. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. The computer program performs the above-described functions defined in the system of the embodiment of the present disclosure when executed by the processor 1001. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims (11)

1. A method for locating an anomaly of a microservice, comprising:
acquiring a log file corresponding to a call request based on the call request sent by at least one microservice, wherein the log file comprises call information of the call request;
establishing an index library in real time based on the log file;
obtaining the log file with the indexed repository query to locate a calling exception for the at least one microservice based on the calling information.
2. The method according to claim 1, wherein before the creating the index repository in real time based on the log file, further comprising storing the log file, specifically comprising:
processing the log file to obtain a corresponding log message;
storing the log message to a message queue;
storing the log file by consuming log messages in the message queue.
3. The method of claim 2, wherein the obtaining a log file corresponding to the invocation request comprises:
intercepting a call request sent by the at least one microservice to generate a call record number;
and writing the calling information of the calling request and the calling record number into the log file in a correlated manner.
4. The method of claim 3, wherein the intercepting a call request issued by the at least one microservice comprises:
intercepting a call request issued by the at least one microservice in a non-intrusive code based manner.
5. The method of claim 3, wherein intercepting a call request issued by the at least one microservice to generate a call record number comprises:
intercepting at least one call request under the same call chain, wherein the call chain comprises a call relation executed by the at least one microservice in response to a user request;
and generating the call record number based on the call chain.
6. The method of claim 3, wherein the invocation information includes at least one of:
caller, callee, call timestamp, response timestamp, server address, interface name.
7. The method of claim 6, wherein after the obtaining the log file corresponding to the invocation request, the method further comprises:
calculating the time consumed corresponding to the interface name based on the calling timestamp and the response timestamp;
and when the consumed time is greater than or equal to a preset threshold value, sending a prompt message.
8. The method of claim 7, wherein the hint information includes the call record number, and wherein obtaining the log file using the indexed repository query comprises:
matching with index information in the index library based on the calling record number;
and acquiring the log file based on the index information successfully matched with the call record number.
9. An anomaly locating device for microservices, comprising:
the file acquisition module is used for acquiring a log file corresponding to a call request based on the call request sent by at least one microservice, wherein the log file comprises call information of the call request;
the index library module is used for establishing an index library in real time based on the log file;
and the exception positioning module is used for utilizing the index library to inquire and obtain the log file so as to position the calling exception of the at least one micro service based on the calling information.
10. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.
11. A computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of claims 1 to 8.
CN202111494911.0A 2021-12-08 2021-12-08 Micro-service exception positioning method and device, electronic equipment and program product Pending CN114153703A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111494911.0A CN114153703A (en) 2021-12-08 2021-12-08 Micro-service exception positioning method and device, electronic equipment and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111494911.0A CN114153703A (en) 2021-12-08 2021-12-08 Micro-service exception positioning method and device, electronic equipment and program product

Publications (1)

Publication Number Publication Date
CN114153703A true CN114153703A (en) 2022-03-08

Family

ID=80454030

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111494911.0A Pending CN114153703A (en) 2021-12-08 2021-12-08 Micro-service exception positioning method and device, electronic equipment and program product

Country Status (1)

Country Link
CN (1) CN114153703A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114844771A (en) * 2022-05-05 2022-08-02 亚信科技(中国)有限公司 Monitoring method, device, storage medium and program product for micro-service system
CN117370052A (en) * 2023-09-14 2024-01-09 广州宇中网络科技有限公司 Microservice fault analysis method, device, equipment and storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114844771A (en) * 2022-05-05 2022-08-02 亚信科技(中国)有限公司 Monitoring method, device, storage medium and program product for micro-service system
CN114844771B (en) * 2022-05-05 2024-03-08 亚信科技(中国)有限公司 Method, device, storage medium and program product for monitoring micro service system
CN117370052A (en) * 2023-09-14 2024-01-09 广州宇中网络科技有限公司 Microservice fault analysis method, device, equipment and storage medium
CN117370052B (en) * 2023-09-14 2024-04-26 广州宇中网络科技有限公司 Microservice fault analysis method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN113987074A (en) Distributed service full-link monitoring method and device, electronic equipment and storage medium
CN112162965B (en) Log data processing method, device, computer equipment and storage medium
CN113254466B (en) Data processing method and device, electronic equipment and storage medium
CN112148674A (en) Log data processing method and device, computer equipment and storage medium
US20160323160A1 (en) Detection of node.js memory leaks
CN114153703A (en) Micro-service exception positioning method and device, electronic equipment and program product
CN115587575A (en) Data table creation method, target data query method, device and equipment
CN115357761A (en) Link tracking method and device, electronic equipment and storage medium
CN112817831A (en) Application performance monitoring method, device, computer system and readable storage medium
CN114490272A (en) Data processing method and device, electronic equipment and computer readable storage medium
CN112433921A (en) Method and apparatus for dynamic point burying
US20160127217A1 (en) Business transaction context for call graph
JP2021163475A (en) Log-based mashup code generation
CN117215867A (en) Service monitoring method, device, computer equipment and storage medium
CN111241048A (en) Web terminal log management method, device, medium and electronic equipment
CN114168607A (en) Global serial number generation method, device, equipment, medium and product
CN114218283A (en) Abnormality detection method, apparatus, device, and medium
CN113760568A (en) Data processing method and device
CN113342619A (en) Log monitoring method and system, electronic device and readable medium
CN113762910A (en) Document monitoring method and device
CN113778777A (en) Log playback method and device
CN112699116A (en) Data processing method and system
CN110554942A (en) method and device for monitoring code execution
CN114844957B (en) Link message conversion method, device, equipment, storage medium and program product
CN115408461A (en) Data information acquisition method and device, electronic equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination