CN113342769A - Unified log recording tool, method, storage medium and equipment - Google Patents
Unified log recording tool, method, storage medium and equipment Download PDFInfo
- Publication number
- CN113342769A CN113342769A CN202110612217.8A CN202110612217A CN113342769A CN 113342769 A CN113342769 A CN 113342769A CN 202110612217 A CN202110612217 A CN 202110612217A CN 113342769 A CN113342769 A CN 113342769A
- Authority
- CN
- China
- Prior art keywords
- log information
- log
- module
- interface
- message queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000012790 confirmation Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 29
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000003032 molecular docking Methods 0.000 claims description 2
- 230000002159 abnormal effect Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 4
- 239000008186 active pharmaceutical agent Substances 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Abstract
The invention discloses a unified log recording tool, a unified log recording method, a unified log recording storage medium and unified log recording equipment, and belongs to the field of software development. The invention mainly comprises a log information pushing module which is decoupled with various services, and when a configuration switch is opened, a log information sequence is pushed to a log message queue receiving module through section-oriented programming and/or a log pushing interface; the log message queue receiving module is used for receiving and providing a log information sequence and confirming the successful reading and writing of the log information sequence through a confirmation mode; the log information reading and storing module is decoupled with various services and used for reading and analyzing a log information sequence from the log message queue receiving module to obtain log information and storing the log information in a database; and the log query analysis module is used for performing query analysis on the log information in the database according to the log information type. The invention adopts a unified log record framework, and enhances the universality, flexibility, accuracy and uniformity of the system.
Description
Technical Field
The invention relates to the field of software development, in particular to a unified log recording tool, a unified log recording method, a unified log recording storage medium and unified log recording equipment.
Background
Java log record subassembly on the market at present is more in kind, but mainly also just towards project level's log record, and the log interface of realization is or realize differently, mainly has:
apache common Logging, called Jakarta common Logging for short JCL, is a general log API provided by Apache, and can enable an application program not to depend on a specific log implementation tool any more. Other Log tools, including Log4J, Avalon Log kit, JUL and the like, are simply packaged in the common-logging package, so that the logs dotted by the JCL API can be directly adapted to the corresponding Log implementation tools when the application program runs.
Log4j, Log4j is an open source code item for Apache, and by using Log4j, we can control Log information delivery to console, file, database, etc.; we can also control the output format of each log; by defining the level of each piece of log information, the generation process of the log can be controlled more finely.
SLF4J, entitled simply Logger Facade for Java that is The Simple Logger solution, but provides some Java Logger APIs through a Facade schema, similar to JCL.
The above is a main stream Java log framework, but all are oriented to single item log record, and the support for item domain level log record and unified and convenient log information query and viewing is weak.
In the traditional software development, log information such as debug, info, exception and the like is respectively output to logs by developers of respective sub-applications and modules, problems that the formats are not unified, the output levels are not unified, log file information is stored scattered and the like can be caused, further integration of multiple applications of products is achieved, and unified log information viewing is provided.
Disclosure of Invention
The invention mainly provides a unified log recording tool, a method, a storage medium and equipment, aiming at the problems of non-unified output format and level, scattered storage and difficult analysis of log information in the prior art.
In order to achieve the above purpose, the invention adopts a technical scheme that: a unified logging tool is provided, comprising: the system comprises a log information pushing module, a log message queue receiving module, a log information reading and storing module and a log query and analysis module, wherein the log information pushing module is decoupled with various services, and pushes a log information sequence into the log message queue receiving module through section-oriented programming when a configuration switch is turned on, and or pushes the log information sequence into the log message queue receiving module through a log pushing interface; the log message queue receiving module is used for receiving and providing a log information sequence and confirming the successful reading and writing of the log information sequence through a confirmation mode of a message queue; the log information reading and storing module comprises log recording application, wherein the log recording application is decoupled from various services and used for reading and analyzing a log information sequence from the log message queue receiving module to obtain log information and storing the log information in a database; and the log query analysis module is used for performing query analysis on the log information in the database according to the log information types, wherein the log information types comprise domains, items, control layers and/or methods.
The invention adopts another technical scheme that: a unified logging method is provided, comprising: the method comprises the steps that a log information pushing module is decoupled from various services, when a configuration switch is turned on, a log information sequence is pushed into a log message queue receiving module through section-oriented programming, and/or the log information sequence is pushed into the log message queue receiving module through a log pushing interface; receiving and providing a log information sequence through a log information queue receiving module, and confirming the successful reading and writing of the log information sequence through a confirmation mode of a message queue; the log information reading and storing module is decoupled with various services, and a log information sequence is read and analyzed from the log information queue receiving module to obtain log information and store the log information in a database; and performing query analysis on the log information in the database according to the log information types through a log query analysis module, wherein the log information types comprise domains, items, control layers and/or methods.
The invention adopts another technical scheme that: a computer-readable storage medium is provided that stores computer instructions operable to perform the unified logging method of scheme two.
The invention adopts another technical scheme that: there is provided a computer apparatus, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; the storage stores computer instructions executable by at least one processor, and the at least one processor operates the computer instructions to execute the unified logging method in the second scheme.
The technical scheme of the invention can achieve the following beneficial effects: the invention designs a unified log recording tool, a unified log recording method, a unified log recording storage medium and unified log recording equipment. The invention adopts a unified log recording framework, has very obvious help for the daily debugging, the running stability analysis, the specific interface execution efficiency analysis and other works of the system, performs unified processing and storage on the log information, and has great advantages and convenience for the query, display, timeliness, analysis and comparison of the log information.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of one embodiment of a unified logging tool of the present invention;
FIG. 2 is a schematic diagram of one embodiment of a unified logging tool of the present invention;
FIG. 3 is a schematic diagram of an embodiment of a unified logging tool of the present invention;
FIG. 4 is a diagram illustrating one embodiment of a unified logging method of the present invention;
FIG. 5 is a diagram of an embodiment of a unified logging method of the present invention;
fig. 6 is a schematic diagram of another embodiment of a unified logging method of the present invention.
Specific embodiments of the present invention have been shown by way of example in the drawings and will be described in more detail below. The drawings and the description are not intended to limit the scope of the inventive concept in any way, but rather to illustrate it by those skilled in the art with reference to specific embodiments.
Detailed Description
The following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In traditional software development, log information such as debug, info, exception and the like is output to logs by developers of respective sub-applications and modules, problems that formats are not unified, output levels are not unified, log file information is stored scattered and the like can be caused, further integration of multiple applications of products is achieved, and unified log information viewing is provided.
Aiming at the defects of blocking log recording and troubleshooting in practical application, in order to standardize the recording of log information, reduce repeated codes, reduce development and maintenance cost and improve the development efficiency of software, a set of unified log recording tool is developed, and the unified log recording tool has the following characteristics:
independent log record application is decoupled and separated, unified receiving of a plurality of system logs is achieved through the RabbitMQ component, and different log record domains and record queues can be divided according to needs.
The method has the advantages that the log record configuration is unified, convenient and flexible, the log pushing component is integrated in a company basic framework, and the log record can be flexibly configured to be started, whether the log record of a control layer controller is started or not, and whether the log record of a method is started or not can be flexibly configured.
The independent log recording application is independently deployed and decoupled from other services, which does not need to care which services need to record logs, and only needs to read from the log information queue and store the log information queue to a database according to the schedule.
The unified log query display interface can filter and query log records according to different project domains, different projects, different modules, different control layers and even different methods. Under the environment of a distributed micro-service architecture, the abnormal information of the service in the projects in the same time period and the same domain is inquired, and the method is greatly helpful for judging and positioning the abnormal condition when the service interfaces are subjected to inter-modulation.
The application scenarios of the unified log recording tool provided by the invention are as follows: under the distributed, micro-service and multi-service application environments, a uniform log recording interface tool is packaged, and a uniform log recording interface can be provided for each sub-service application to record logs.
The invention has the following inventive concept: the invention integrates the log information pushing into a basic code frame, and the basic code frame is a development frame which is independently researched, developed and integrated and has independent intellectual property rights; by adopting the combination of AOP section interception programming and autonomous calling, the generation, processing mechanism and realization of execution log information irrelevant to service are met; the RabbitMQ is adopted to process the log and is decoupled from the log information, so that the overall architecture thought and realization of log information generation, decoupled queue processing and unified query are embodied.
The following describes the technical solution of the present invention and how to solve the above technical problems with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
Fig. 1 shows an embodiment of a unified logging tool of the present invention, in which the unified logging tool mainly includes:
the module 101 is a log information pushing module, which is decoupled from various services, and when the configuration switch is turned on, the log information sequence is pushed to the log message queue receiving module through a profile-oriented programming, and/or the log information sequence is pushed to the log message queue receiving module through a log pushing interface.
The module 102 is a log message queue receiving module, which is used for receiving and providing a log information sequence, and confirming the successful reading and writing of the log information sequence through a confirmation mode of a message queue.
The module 103 is a log information reading and storing module, which includes a log recording application, the log recording application is decoupled from various services, and is used for reading and analyzing a log information sequence from the log message queue receiving module to obtain log information and storing the log information in a database.
The module 104 is a log query analysis module, and is configured to perform query analysis on log information in the database according to log information types, where the log information types include a domain, an item, a control layer, and/or a method.
In this embodiment, the unified log recording tool mainly includes 4 modules, which are respectively a log information pushing module, a log information Queue receiving module, a log information reading and storing module, and a log query and analysis module integrated into a basic code frame, wherein the log information pushing module pushes a log information sequence to a Message Queue (MQ for short) in the log information Queue receiving module; the log message queue receiving module queues the log information sequence and adopts a confirmation mode to ensure the correctness and the integrity of each read-write of the log information, wherein the confirmation mode is a confirm mode; a log record application in the log information reading and storing module reads and analyzes a log information sequence, deserializes the log information sequence into log information, and stores the log information into a database corresponding to the log record application; the log query analysis module provides log information which is queried and stored in a corresponding database according to domains, projects, control layers and/or methods, and has great advantages and convenience for query, display, timeliness, analysis and comparison of the log information.
In a specific embodiment of the present invention, in the log information pushing module, the log information is sorted and serialized to obtain a log information sequence, and the log information sequence is pushed to the log information queue receiving module.
In this embodiment, the generated log information is sorted, for example, data sorting such as calculating a task and a method, calculating time consumed for calling an external interface, whether the log information is abnormal or not, intercepting processing when the log information is abnormal and too long, and peer IP address information when the log information is called is performed. The sorted log information is serialized into sending format data, namely a log information sequence, and is pushed to a message queue MQ in a log message queue receiving module, and the log information is pushed to the message queue MQ according to a uniform format, so that the butt joint with a new service system can be completed, and the universality of a uniform log recording tool is enhanced.
In a specific embodiment of the invention, in the log query analysis module, log query analysis is performed through a query interface of a log record application; and or performing log query analysis on the query interface through a service system corresponding to a plurality of services.
In this embodiment, the log recording application can provide a basic log information query interface, and the service systems corresponding to various services are connected to the log information query interface to perform query analysis on the log information; the log recording application can also configure a new data source according to a domain, a project, a control layer and/or a method to directly query log information in a database, analyze the log information and display the obtained data; unified log information query embodies the extension of recording and viewing the traditional log information and further extraction and application.
Fig. 2 shows a specific embodiment of the unified logging tool of the present invention, the applications requiring logging include non-post, warehousing and marketing, the log information is stored in the corresponding database, and the database can be divided into three categories, which respectively store the log calling the external interface, the log calling the autonomous interface, and the redundant backup.
In the embodiment, the log information pushing module is integrated in a basic framework, is decoupled from various services, opens or closes log records through a configuration switch, performs section processing on various services through Aspect Oriented Programming (AOP for short) after the configuration switch is opened, and automatically pushes contents such as called interface method parameters, success or failure, abnormal information in abnormal conditions and the like to a message queue MQ; by using the message-oriented middleware (RabbitMQ), the method is decoupled from the service, and the log record docking can be completed only by configuring the log record queue address and the connection key no matter what service system needs log record. For calling an external third-party interface, a call log pushing interface needs to be manually set, and information such as a call address, parameters and return content is pushed to a message queue MQ. The log message queue receiving module is responsible for receiving or providing a log information sequence required to be read, and in order to ensure that the log information can be correctly and completely completed by reading and writing each time, a confirm mode, namely a confirm mode, is adopted for reading and writing the message queue MQ. The log information reading and storing module is an independently deployed application, is decoupled from a service, and is only responsible for reading a log information sequence from a message queue, deserializing the log information sequence into log information, and storing the log information into a corresponding database, wherein the log information is mainly called an external interface log and a local interface called log, and parameters are different. The log query analysis module provides log information which is queried and stored in the database according to domains, projects, control layers and/or methods, and can query in a log record application query interface and also can perform log query display on a corresponding service system butt-joint query interface.
FIG. 3 illustrates one embodiment of a unified logging tool of the present invention.
In the specific example shown in fig. 3, the log information record mainly includes log information generation, log information sorting, pushing to a message queue, reading from the message queue for parsing, storing to a database, performing efficiency analysis, and log query application, wherein the log information generation is divided into three categories according to a scene of information generation, and the first category is generated by AOP processing when a default annotation interface function (LogAsp) annotation category or an annotation method is called; the second type is generated by automatically calling a log information interface after a processing center executes a processing task; the third type is generated by automatically calling a log information interface after the call of an interface needing to record the log is completed when an external interface is connected.
For log information arrangement, before the log information is pushed, some processing and arranging operations are performed, such as calculating tasks and methods, calculating time consumed by calling an external interface, judging whether the log information is abnormal or not, intercepting processing when the log information is abnormal and too long, and data arrangement of opposite-end IP address information when the log information is called.
And for the push to message queue, converting the sorted log information into a JSON format, namely a posting format, obtaining the log information sequence, and pushing the log information sequence to a message queue MQ in a log message queue module.
For reading and parsing from the message queue, the logging application reads the log information sequence from the message queue and deserializes its mail format data into a log information object.
And for the log information types stored in the database, storing the log information types into the corresponding database and/or data table according to different log information types, wherein the log information types comprise domains, items, control layers and/or methods.
For the execution efficiency analysis, the log information of the processing center is subjected to some analysis on timing, such as data processing of average task execution time, average waiting time, average daily task execution quantity and the like.
For the log query application, the log record application provides a basic log information query interface, and other applications can also configure a new data source to directly query log information in a database and analyze data for display if necessary.
In a specific embodiment of the present invention, the log information generating module is configured to perform facet processing on multiple services through facet-oriented programming to obtain log information; and/or after the processing task is executed by butting an external third-party interface or a processing center, calling a log information interface to obtain log information; the log information comprises called interface parameters, called method parameters, success parameters, exception parameters, calling addresses and/or return contents.
In this embodiment, there are three main types of reasons for generating log information, and first, the log information is generated after performing section processing on multiple services through AOP; after the processing center executes the processing task, automatically calling a log information interface to generate; and thirdly, automatically calling a log information interface to generate after an external third-party interface is connected.
In one embodiment of the present invention, the functional modules in a unified logging tool of the present invention can be directly in hardware, in a software module executed by a processor, or in a combination of both.
A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium.
The Processor may be a Central Processing Unit (CPU), other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other Programmable logic devices, discrete Gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
The unified log recording tool provided by the invention realizes unified reception of logs in a plurality of service systems through the RabbitMQ component, and can also divide different log recording domains and recording queues according to needs. The log pushing component is integrated in the basic framework, and can flexibly configure and open log records, and whether to open the log records of a control layer controller or a method. The independent log recording application is independently deployed and decoupled from other services, which does not need to care which services need to record logs, and only needs to read from a log message queue and store the log message queue to a database by force. The unified log query display interface can filter and query log records according to different project domains, different projects, different modules, different control layer controllers and even different methods. Under the environment of a distributed micro-service architecture, the abnormal information of the service in the projects in the same time period and the same domain is inquired, and the method is greatly helpful for judging and positioning the abnormal condition when the service interfaces are subjected to inter-modulation.
Fig. 4 shows a specific embodiment of a unified logging method according to the present invention, in which the unified logging method mainly includes:
step S401, the log information pushing module is decoupled from various services, when the configuration switch is turned on, the log information sequence is pushed to the log message queue receiving module through section-oriented programming, and/or the log information sequence is pushed to the log message queue receiving module through the log pushing interface.
Step S402, receiving and providing the log information sequence through the log information queue receiving module, and confirming the successful read-write of the log information sequence through the confirmation mode of the message queue.
Step S403, the log information reading and storing module is decoupled with various services, and the log information sequence is read and analyzed from the log information queue receiving module to obtain the log information and store the log information in the database.
Step S404, the log information in the database is queried and analyzed according to the log information type through the log query analysis module, wherein the log information type comprises a domain, an item, a control layer and/or a method.
In this embodiment, the unified log recording method pushes the log information sequence to a Message Queue (MQ for short) in the log Message Queue receiving module through the log information pushing module; queuing the log information sequence through a log information queue receiving module, and ensuring the correctness and integrity of each read-write of the log information by adopting a confirmation mode, wherein the confirmation mode is a confirm mode; reading and analyzing a log information sequence through a log record application in a log information reading and storing module, deserializing the log information sequence into log information, and storing the log information into a database corresponding to the log record application; and providing log information which is inquired and stored in a corresponding database according to domains, projects, control layers and/or methods through a log inquiry analysis module.
In a specific embodiment of the present invention, the log information sequence is pushed to the log message queue receiving module through the log pushing interface, including executing a processing task by butting an external third party interface or a processing center, and calling the log information interface to obtain the log information; and processing the log information to obtain a log information sequence, and pushing the log information sequence to a log message queue receiving module through a log pushing interface.
In this embodiment, the external third-party interface and the processing center have substantially the same recording method for the log information, and after the external third-party interface or the processing center executes any service, the log information interface is actively called to record the related log information.
Fig. 5 illustrates a specific example of a unified logging method of the present invention.
In the specific example shown in fig. 5, the external third-party interface is the same as the log information recording process of the processing center, and after a certain kind of task action is performed, the log information interface is actively called to record the relevant log information. The log information recording process comprises four steps of executing related tasks, waiting for the return of an execution result, sorting log information and pushing the log information to a message queue.
For executing related tasks, after an asynchronous task or a queue task is processed and completed by calling an external third-party interface or a processing center, when log information needs to be recorded, a call is made to a log information interface.
And for the execution waiting result, after the task is finished, acquiring the execution result or capturing the execution exception.
For the log information arrangement, if an external third-party interface is called, calling parameters and returning results need to be obtained; other tasks can clear up whether the execution result is successful or failed.
And for the push to the message queue, the sorted log information is serialized into a mail format to obtain a log information sequence, and the log information sequence is pushed to the message queue in the log message queue module.
In a specific embodiment of the present invention, when the configuration switch is turned on, the log information sequence is pushed to the log message queue receiving module by the profile-oriented programming, including that, after the configuration switch is turned on, the profile processing is performed on a plurality of services by the profile-oriented programming to obtain the log information; obtaining an annotation interface through a reflection mechanism, and judging whether the log information sets an ignoring attribute through the annotation interface; and processing the log information without the ignoring attribute to obtain a log information sequence and pushing the log information sequence to a log message queue receiving module, wherein the processing comprises sorting and serialization.
In this embodiment, by using the facet-oriented programming technology, when a log needs to be recorded, the AOP component in the Spring Freework performs log recording on all control layers and all methods in a project by default, and when a method interface is called, records a call parameter, returns a parameter, and returns exception information. A reflection mechanism and a custom annotation technology in Java are used, when log records are opened by default, logs are recorded for all control layers and methods by default, and when some control layers or some methods do not need to be recorded, specified @ LogAsp (ignore) annotation can be added to exclude the recorded logs. By combining AOP section interception programming and autonomous calling, the generation, processing mechanism and implementation of execution log messages irrelevant to services are met.
Fig. 6 illustrates a specific example of a unified logging method of the present invention.
In the specific example shown in fig. 6, the general log record is a log information record when the internal interface is called, and is mainly used for checking an abnormal error, and is also used for analyzing the execution time and efficiency of the interface. The general log interface execution flow comprises the steps of opening a configuration switch of a log, judging whether to set an ignoring attribute or not through annotation, arranging log information when recording is needed, and pushing the log information to a message queue.
For the configuration switch for opening the log, whether the main switch for recording the log is set in the configuration, namely the configuration switch, is arranged, and after the configuration switch is opened, the log information in the application is processed and recorded.
And judging whether the ignoring attribute is set for the annotation, acquiring annotation LogAsp (log Asp) by reflection when the interface method is called to judge whether the ignoring attribute is set, skipping the log information recording process for the log with the ignoring attribute, and carrying out the next processing on the log information for the log without the ignoring attribute, wherein the ignoring attribute is a non-recording attribute.
The log information is sorted when the log information needs to be recorded, and the log information mainly comprises the execution result, success or failure, execution time, exception or failure, exception information when exception occurs and the like of the called interface.
And for the pushed log information to the message queue, converting the sorted log information into a JSON format and pushing the JSON format to the message queue MQ.
The unified log recording method provided by the invention does not need to pay attention to service logic, and can complete the recording of log information only by turning on or turning off a configuration switch of the log or automatically and manually transferring the parameters. Meanwhile, the message queue MQ is adopted for decoupling, a new service system needs to record, for a code frame of any company, the code frame can be butted with the service system only by pushing the code frame into the message queue MQ according to the unified format of the log information, and the universality of a log recording tool in the scheme is ensured. According to the scheme, logs are recorded on all control layer controllers and interface methods by default, meanwhile, exclusion recording on a certain control layer controller or an interface method under a certain control layer controller can be achieved, a certain control layer controller can be set not to record, but a certain method under the control layer controller records logs, so that the method is flexible and meets daily log recording requirements. The reading and writing of the message queue MQ adopt a confirm mode, and confirm is carried out after the operation of a reading party or a writing party is finished, so that the accuracy of reading and writing of the log information is ensured. The log information is processed and stored uniformly, and the method has great advantages and convenience for query, display, timeliness, analysis and comparison of the log information.
In another embodiment of the present invention, a computer-readable storage medium stores computer instructions operable to perform the unified logging method described in any of the embodiments.
In another embodiment of the invention, a computer device includes at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform the unified logging method as described in any of the embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent structural changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A unified logging tool, comprising: a log information pushing module, a log message queue receiving module, a log information reading and storing module and a log query and analysis module, wherein,
the log information pushing module is decoupled from various services, and when the configuration switch is turned on, through section-oriented programming, a log information sequence is pushed into the log information queue receiving module, and/or the log information sequence is pushed into the log information queue receiving module through a log pushing interface;
the log message queue receiving module is used for receiving and providing the log information sequence and confirming the successful reading and writing of the log information sequence through a confirmation mode of a message queue;
the log information reading and storing module comprises a log recording application, and the log recording application is decoupled with the multiple services and is used for reading and analyzing the log information sequence from the log message queue receiving module to obtain log information and storing the log information into a database; and
the log query analysis module is used for performing query analysis on the log information in the database according to log information types, wherein the log information types comprise domains, items, control layers and/or methods.
2. The unified logging tool of claim 1, further comprising:
the log information generating module is used for carrying out section processing on the multiple services through the section-oriented programming to obtain the log information; and/or
The log information interface is used for calling the log information interface to obtain the log information after an external third-party interface or a processing center is butted to execute a processing task; the log information comprises called interface parameters, called method parameters, success parameters, exception parameters, calling addresses and/or return contents.
3. The unified logging tool of claim 1, wherein in the log information pushing module, the log information is sorted and serialized to obtain the log information sequence, and the log information sequence is pushed to the log message queue receiving module.
4. The unified logging tool of claim 1, wherein in the log query analysis module,
performing log query analysis through a query interface of the log record application; and/or
And performing log query analysis through the service system docking query interface corresponding to the multiple services.
5. A unified logging method, comprising:
the method comprises the steps that a log information pushing module is decoupled from various services, when a configuration switch is turned on, a log information sequence is pushed into a log message queue receiving module through section-oriented programming, and/or the log information sequence is pushed into the log message queue receiving module through a log pushing interface;
receiving and providing the log information sequence through the log information queue receiving module, and confirming the successful reading and writing of the log information sequence through a confirmation mode of the message queue;
the log information reading and storing module is decoupled with the multiple services, and the log information sequence is read and analyzed from the log information queue receiving module to obtain log information and store the log information in a database; and
and performing query analysis on the log information in the database according to log information types through a log query analysis module, wherein the log information types comprise domains, items, control layers and/or methods.
6. The unified logging method of claim 5, wherein said pushing the log information sequence into the log message queue receiving module by profile-oriented programming when the configuration switch is open comprises:
after the configuration switch is turned on, performing section processing on the multiple services through the section-oriented programming to obtain the log information;
obtaining an annotation interface through a reflection mechanism, and judging whether the log information is set with an ignoring attribute through the annotation interface; and
and processing the log information without the ignoring attribute to obtain a log information sequence and pushing the log information sequence to the log information queue receiving module, wherein the processing comprises sorting and serialization.
7. The unified logging method of claim 6, wherein said pushing said sequence of log information into said log message queue receiving module through a log push interface comprises:
executing a processing task by butting an external third-party interface or a processing center, and calling a log information interface to obtain the log information;
and processing the log information to obtain the log information sequence, and pushing the log information sequence to the log message queue receiving module through the log pushing interface.
8. A computer readable storage medium storing computer instructions, wherein the computer instructions are operable to perform the unified logging method of any of claims 5-7.
9. A computer device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor;
wherein the memory stores computer instructions executable by the at least one processor, the at least one processor operating the computer instructions to perform the unified logging method of any of claims 5-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110612217.8A CN113342769A (en) | 2021-06-02 | 2021-06-02 | Unified log recording tool, method, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110612217.8A CN113342769A (en) | 2021-06-02 | 2021-06-02 | Unified log recording tool, method, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113342769A true CN113342769A (en) | 2021-09-03 |
Family
ID=77474993
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110612217.8A Pending CN113342769A (en) | 2021-06-02 | 2021-06-02 | Unified log recording tool, method, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113342769A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113806182A (en) * | 2021-09-30 | 2021-12-17 | 北京蓝海医信科技有限公司 | Interactive process monitoring method based on information integration platform and computer equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111221664A (en) * | 2019-12-24 | 2020-06-02 | 视联动力信息技术股份有限公司 | Operation log processing system |
| CN111241049A (en) * | 2020-01-06 | 2020-06-05 | 深圳市同洲电子股份有限公司 | Distributed operation log implementation system based on micro-service architecture |
| CN111708749A (en) * | 2020-07-24 | 2020-09-25 | 深圳市富之富信息科技有限公司 | Operation log recording method and device, computer equipment and storage medium |
-
2021
- 2021-06-02 CN CN202110612217.8A patent/CN113342769A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111221664A (en) * | 2019-12-24 | 2020-06-02 | 视联动力信息技术股份有限公司 | Operation log processing system |
| CN111241049A (en) * | 2020-01-06 | 2020-06-05 | 深圳市同洲电子股份有限公司 | Distributed operation log implementation system based on micro-service architecture |
| CN111708749A (en) * | 2020-07-24 | 2020-09-25 | 深圳市富之富信息科技有限公司 | Operation log recording method and device, computer equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113806182A (en) * | 2021-09-30 | 2021-12-17 | 北京蓝海医信科技有限公司 | Interactive process monitoring method based on information integration platform and computer equipment |
| CN113806182B (en) * | 2021-09-30 | 2024-04-05 | 北京蓝海医信科技有限公司 | Interactive flow monitoring method based on information integration platform and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20060085361A1 (en) | Anomaly detector in a health care system using adapter | |
| US9697104B2 (en) | End-to end tracing and logging | |
| US7912938B2 (en) | Correlation of web service interactions in composite web services | |
| US20040167912A1 (en) | Unified logging service for distributed applications | |
| US9442822B2 (en) | Providing a visual representation of a sub-set of a visual program | |
| US20040117435A1 (en) | Common persistence layer | |
| US20040205136A1 (en) | Document message state management engine | |
| US20090063395A1 (en) | Mapping log sets between different log analysis tools in a problem determination environment | |
| US8997116B2 (en) | Event logging application programming interface (API) | |
| CN113342769A (en) | Unified log recording tool, method, storage medium and equipment | |
| US7802235B2 (en) | System and method for tracing and/or evaluating the exchange of information | |
| US20120284167A1 (en) | Performance Testing Tool for Financial Applications | |
| US20060136933A1 (en) | Server-side eventing for managed server applications | |
| US7937433B1 (en) | Queuing connector to promote message servicing | |
| CN112765246A (en) | Task processing method and device, electronic equipment and storage medium | |
| CN111652580B (en) | Method and device for processing data of nodes | |
| Davies et al. | Websphere mq v6 fundamentals | |
| CN111881025A (en) | Automatic test task scheduling method, device and system | |
| KR101870369B1 (en) | System for linking between inside channel and outside channel based web oprating enviroment and method thereof | |
| Mengistu | Distributed Microservice Tracing Systems: Open-source tracing implementation for distributed Microservices build in Spring framework | |
| US7305583B2 (en) | Command initiated logical dumping facility | |
| CN113808723B (en) | Service system information integration platform | |
| US11645137B2 (en) | Exception management in heterogenous computing environment | |
| US20240061729A1 (en) | Multitenancy cross-tenant collaboration driven by event proxy | |
| CN117708182A (en) | Method and related device for constructing data blood margin |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |