CN115328755A - Concurrent log recording method and system - Google Patents
Concurrent log recording method and system Download PDFInfo
- Publication number
- CN115328755A CN115328755A CN202211241042.5A CN202211241042A CN115328755A CN 115328755 A CN115328755 A CN 115328755A CN 202211241042 A CN202211241042 A CN 202211241042A CN 115328755 A CN115328755 A CN 115328755A
- Authority
- CN
- China
- Prior art keywords
- log
- thread
- printing
- current
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 238000012545 processing Methods 0.000 claims description 20
- 230000002159 abnormal effect Effects 0.000 claims description 12
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 abstract description 4
- 210000001503 joint Anatomy 0.000 abstract description 2
- 230000005856 abnormality Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3055—Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a concurrent log recording method and a concurrent log recording system, and belongs to the technical field of networks. The method comprises the following steps: the log recording monitor monitors log printing requests submitted by all the service threads and judges whether the log printing requests contain a final termination state or not; and the log record monitor receives the thread id pushed by each service thread, queues log records according to the thread id, and performs queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record. According to the invention, the plurality of log output frames are integrated, so that the threshold of a developer is reduced, the developer can conveniently butt joint a plurality of log acquisition systems through the exposed abstract interface, the order of log output is ensured through log integration among threads, and the readability of the log and the efficiency of positioning the log problem are improved.
Description
Technical Field
The invention belongs to the technical field of networks, and particularly relates to a concurrent log recording method and system.
Background
When the program is in error or abnormal, the log is used as an important troubleshooting tool, the basis of the problem of the positioning system can be accelerated, and convenience can be provided for normal maintenance of the system. In general, when the log is stored in a concurrent manner, the log is in an unordered state as a whole when a plurality of threads are sequentially input. But still a complete log request can be found gradually from top to bottom to handle the corresponding problem.
In order to avoid the situation that logs are scattered when multiple application servers are deployed in a multi-point deployment mode or distributed system nodes are deployed in a multiple application servers, a database or other storage engines are generally adopted for log collection to store the summary logs of multiple services, and therefore the records of the logs can be mixed with the logs between different threads of different servers.
And printing the logs to ensure that the logs are output in order at a single node and are output in order at the summary server. The method for collecting logs provided by the chinese patent CN105278996A solves the problem of reference conflict caused by the need of introducing a third-party component in the existing log collection process, and the technical scheme monitors all class files in the range to be collected when log collection is implemented, and then collects and processes the files in a unified manner, but does not consider the sequential performance of the logs executed in a single thread, and is still the same as that of the old log collection system, and the logs are collected in a centralized manner according to the output sequence of the logs, so that the collected files are still in a time-ordered and concurrent unordered storage state.
Disclosure of Invention
The invention mainly aims to overcome the defects of the prior art and provide a concurrent log recording method and a concurrent log recording system, which ensure the orderliness of log output and improve the readability of logs and the efficiency of positioning log problems through log integration among threads.
According to one aspect of the present invention, there is provided a concurrent logging method, the method comprising the steps of:
the log recording monitor monitors log printing requests submitted by all the service threads and judges whether the log printing requests contain a final termination state or not;
and the log record monitor receives the thread id pushed by each service thread, queues the log records according to the thread id, and performs queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record.
Preferably the method comprises: if the log printing request contains the final termination state, permanently storing and transmitting the log content;
and if the log printing request does not contain the final termination state and part of log storage functions are started, caching the log contents and storing the log contents into a specified database.
Preferably, the method comprises: and in the execution of the business method, the log records and the thread ids of the current execution thread are asynchronously pushed to the log record monitor by uniformly recording the file records and printing, and the log record monitor performs log merging, sorting, storing and transmitting.
Preferably, the method comprises: when a service processing interface requests, resetting the id of the current service thread, recording the related information of the log file, and asynchronously uploading the related information of the log file to the log record monitor; the log class file related information comprises: the method comprises the steps of current service execution time, a server ip, an interface request path, a current operation user, request parameters and return parameters.
Preferably, the method comprises: carrying out abnormal monitoring on the business method before the business method is started, then executing the business method, and recording abnormal information in time when the business method is abnormally terminated;
and after the execution of the business method is finished, asynchronously uploading the state of the current thread printing log to a log record monitor, and storing or pushing the state to a designated log collection file for processing by the log record monitor.
According to another aspect of the present invention, there is also provided a concurrent logging system, comprising a logging listener;
the log record listener is used for monitoring log printing requests submitted by all the business threads and judging whether the log printing requests contain final termination states or not;
and the log record monitor receives the thread id pushed by each service thread, queues log records according to the thread id, and performs queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record.
Preferably, if the log printing request includes the final termination state, the log content is permanently stored and transmitted;
if the log printing request does not contain the final termination state and a part of log storage functions are started, caching the log contents and storing the log contents into a specified database.
Preferably, during the execution of the business method, the log records are printed through the unified record class file records, the regular printing is carried out, the log records and the thread ids of the currently executed threads are asynchronously pushed to the log record listener, and the log record listener carries out log merging, sorting, storing and transmitting.
Preferably, the system further comprises a service processing interface, and when the service processing interface requests, the id of the current service thread is reset, the log file related information is recorded, and the log file related information is asynchronously uploaded to the log record monitor; the log class file related information comprises: the method comprises the steps of current service execution time, a server ip, an interface request path, a current operation user, request parameters and return parameters.
Preferably, the service method is monitored abnormally before the service method is started, then the service method is executed, and when the service method is terminated abnormally, abnormal information is recorded in time;
and after the execution of the business method is finished, asynchronously uploading the printing log state of the current thread to a log record monitor, and storing or pushing the printing log state to a specified log collection file by the log record monitor for processing.
Has the beneficial effects that: according to the invention, through integrating a plurality of log output frames, the threshold of a developer is reduced, the developer can conveniently butt joint a plurality of log acquisition systems through the exposed abstract interface, through the log integration among threads, the order of log output is ensured, and the readability of the log and the efficiency of locating the log problem are improved.
The features and advantages of the present invention will become apparent by reference to the following drawings and detailed description of specific embodiments of the invention.
Drawings
FIG. 1 is a flow chart of a concurrent logging method;
FIG. 2 is a schematic diagram of a concurrent logging system.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
FIG. 1 is a flow chart of a concurrent logging method. As shown in fig. 1, the present invention provides a concurrent logging method, which includes the following steps:
the log recording monitor monitors log printing requests submitted by all the service threads and judges whether the log printing requests contain a final termination state or not;
and the log record monitor receives the thread id pushed by each service thread, queues log records according to the thread id, and performs queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record.
Preferably, the method comprises:
if the log printing request contains the final termination state, permanently storing and transmitting log contents;
if the log printing request does not contain the final termination state and a part of log storage functions are started, caching the log contents and storing the log contents into a specified database.
Specifically, a log record listener is loaded when a project is started, when a business thread needs to print a log, a print log type instance is called to print the log, a print task is submitted to the log record listener by the print log type, and the log record listener uniformly integrates and processes the current thread execution log.
The log record monitor monitors log printing requests submitted by all service execution threads of the whole project, caches logs according to thread ids, queues log records are arranged according to the thread ids due to the orderliness of the printing sequence of the logs in the threads, the log records are added one by one, and when the printing log requests contain a final termination state, log contents are permanently stored and transmitted to a log acquisition device in the configuration. And when the printing log request does not contain the final state, caching and storing the data into a specified database according to whether partial log storage is started or not.
The log record monitor can arrange the logs according to the thread id pushed by each thread, the arranging process is to perform queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record, store incomplete log records according to whether partial log persistence is started or not according to configuration, judge whether the current log printing state is a final state or not, and directly store and transmit the data if the current log printing state is the final state.
Preferably, the method comprises: and in the execution of the service method, the log records and the thread ids of the current execution thread are asynchronously pushed to the log record monitor by uniformly recording the file records and printing, and the log record monitor performs log combination, sequencing, storage and transmission.
Specifically, during the execution of the service method, the log is recorded and printed through a unified recording class file, the log is normally operated, the log recording class can carry out conventional printing and asynchronous log recording monitor pushing log records and thread id of a current execution thread, the log recording monitor carries out log merging, sorting, storing, transmitting and the like, the storage position supports common databases such as mysql, mongoDB and the like, the common databases can be selected through configuration, and the transmission supports log collection frames such as logstack and the like.
Preferably, the method comprises: when a service processing interface requests, resetting the id of the current service thread, recording the related information of the log file, and asynchronously uploading the related information of the log file to the log record monitor; the log class file related information comprises: the method comprises the steps of current service execution time, a server ip, an interface request path, a current operation user, request parameters and return parameters.
Specifically, when an interface request is made, by intercepting a tangent plane before processing a service method, resetting the id of a current execution thread, recording the current service execution time, the ip of a server, an interface request path, current operation users and request parameters, returning parameters and other information, and asynchronously uploading the information to a log record monitor.
The log files of the asynchronous uploading log listener are not traditional slf4j or log4j and the like, are independently implemented for packaging various log record frames, can be configured and selected according to the used log types, but the processing files are all unified proxy files.
Preferably, the method comprises: carrying out abnormal monitoring on the business method before the business method is started, then executing the business method, and recording abnormal information in time when the business method is abnormally terminated;
and after the execution of the business method is finished, asynchronously uploading the printing log state of the current thread to a log record monitor, and storing or pushing the printing log state to a specified log collection file by the log record monitor for processing.
Specifically, the service method is monitored for abnormalities before the service method is started, then the service method is executed, and when the service method is terminated abnormally, abnormal information is recorded in time, and the abnormality is further thrown out and is handed to other processors of the system for processing.
And when the business method is executed, continuing to enter the section interception, asynchronously uploading the printing log state of the current thread to a log record monitor, and storing or pushing the printing log state to a designated log collection file by the log record monitor for processing.
In the whole log recording process, asynchronous recording is adopted, the whole service flow is not influenced, the service interface processing performance is minimally influenced, even if the whole log data are lost, normal recording is carried out on a conventional log, and the stability of the project and the integrity of log recording are ensured.
According to the embodiment, the multiple log output frames are integrated, the threshold of a developer is reduced, the developer can conveniently butt against multiple log acquisition systems through the exposed abstract interface, the order of log output is guaranteed through log integration among threads, and the readability of logs and the efficiency of log positioning are improved.
Example 2
FIG. 2 is a schematic diagram of a concurrent logging system. As shown in fig. 2, the present invention also provides a concurrent logging system, which includes a logging listener;
the log record monitor is used for monitoring log printing requests submitted by all the service threads and judging whether the log printing requests contain a final termination state or not;
the log record monitor receives the thread id pushed by each service thread, queues log records according to the thread id, and performs queue type addition on the data of the same thread id acquired before according to the thread id of the current log record.
Preferably, the first and second electrodes are formed of a metal,
if the log printing request contains the final termination state, permanently storing and transmitting log contents;
and if the log printing request does not contain the final termination state and part of log storage functions are started, caching the log contents and storing the log contents into a specified database.
Preferably, during the execution of the business method, the log records are printed through the unified record class file records, the regular printing is carried out, the log records and the thread ids of the currently executed threads are asynchronously pushed to the log record listener, and the log record listener carries out log merging, sorting, storing and transmitting.
Preferably, the system further comprises a service processing interface, and when the service processing interface requests, the id of the current service thread is reset, the log file related information is recorded, and the log file related information is asynchronously uploaded to the log record listener; the log class file related information comprises: the method comprises the steps of current service execution time, a server ip, an interface request path, a current operation user, request parameters and return parameters.
Preferably, the service method is monitored abnormally before the service method is started, then the service method is executed, and when the service method is terminated abnormally, abnormal information is recorded in time;
and after the execution of the business method is finished, asynchronously uploading the printing log state of the current thread to a log record monitor, and storing or pushing the printing log state to a specified log collection file by the log record monitor for processing.
The specific implementation process of this embodiment 2 is the same as that of embodiment 1, and is not described herein again.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A concurrent logging method, comprising:
the log record listener monitors log printing requests submitted by all the business threads and judges whether the log printing requests contain a final termination state or not;
and the log record monitor receives the thread id pushed by each service thread, queues log records according to the thread id, and performs queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record.
2. The method according to claim 1, characterized in that it comprises:
if the log printing request contains the final termination state, permanently storing and transmitting the log content;
and if the log printing request does not contain the final termination state and part of log storage functions are started, caching the log contents and storing the log contents into a specified database.
3. The method of claim 2, wherein the method comprises:
and in the execution of the business method, the log records and the thread ids of the current execution thread are asynchronously pushed to the log record monitor by uniformly recording the file records and printing, and the log record monitor performs log merging, sorting, storing and transmitting.
4. A method according to claim 3, characterized in that the method comprises:
when a service processing interface requests, resetting the id of the current service thread, recording the related information of the log file, and asynchronously uploading the related information of the log file to the log record monitor; the log file related information comprises: the method comprises the steps of current service execution time, a server ip, an interface request path, a current operation user, request parameters and return parameters.
5. The method according to claim 1, characterized in that it comprises:
carrying out abnormal monitoring on the business method before the business method is started, then executing the business method, and recording abnormal information in time when the business method is abnormally terminated;
and after the execution of the business method is finished, asynchronously uploading the state of the current thread printing log to a log record monitor, and storing or pushing the state to a designated log collection file for processing by the log record monitor.
6. A concurrent logging system, comprising a logging listener;
the log record listener monitors log printing requests submitted by all the business threads and judges whether the log printing requests contain a final termination state or not;
and the log record monitor receives the thread id pushed by each service thread, queues log records according to the thread id, and performs queue type addition on the previously acquired data with the same thread id according to the thread id of the current log record.
7. The system of claim 6,
if the log printing request contains the final termination state, permanently storing and transmitting log contents;
if the log printing request does not contain the final termination state and a part of log storage functions are started, caching the log contents and storing the log contents into a specified database.
8. The system of claim 7,
and in the execution of the service method, the log records and the thread ids of the current execution thread are asynchronously pushed to the log record monitor by uniformly recording the file records and printing, and the log record monitor performs log combination, sequencing, storage and transmission.
9. The system of claim 8,
the system also comprises a service processing interface, wherein when the service processing interface requests, the id of the current service thread is reset, the log file related information is recorded, and the log file related information is asynchronously uploaded to the log record monitor; the log file related information comprises: the method comprises the steps of current service execution time, a server ip, an interface request path, a current operation user, request parameters and return parameters.
10. The system of claim 6,
carrying out abnormal monitoring on the business method before the business method is started, then executing the business method, and recording abnormal information in time when the business method is abnormally terminated;
and after the execution of the business method is finished, asynchronously uploading the state of the current thread printing log to a log record monitor, and storing or pushing the state to a designated log collection file for processing by the log record monitor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211241042.5A CN115328755A (en) | 2022-10-11 | 2022-10-11 | Concurrent log recording method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211241042.5A CN115328755A (en) | 2022-10-11 | 2022-10-11 | Concurrent log recording method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115328755A true CN115328755A (en) | 2022-11-11 |
Family
ID=83914680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211241042.5A Pending CN115328755A (en) | 2022-10-11 | 2022-10-11 | Concurrent log recording method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115328755A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100049747A1 (en) * | 2008-08-19 | 2010-02-25 | Samsung Electronics Co., Ltd. | Apparatus and method for storing log in a thread oriented logging system |
| CN103577310A (en) * | 2012-07-20 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Method and device for recording software debugging logs |
| CN108052675A (en) * | 2017-12-28 | 2018-05-18 | 惠州Tcl家电集团有限公司 | Blog management method, system and computer readable storage medium |
| CN110175154A (en) * | 2019-05-17 | 2019-08-27 | 深圳前海微众银行股份有限公司 | A kind of processing method of log recording, server and storage medium |
| CN112231274A (en) * | 2020-10-16 | 2021-01-15 | 京东数字科技控股股份有限公司 | Log summarizing method and device, electronic equipment and storage medium |
| CN112948136A (en) * | 2021-02-02 | 2021-06-11 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Method for implementing asynchronous log record of embedded operating system |
| CN114595201A (en) * | 2022-03-23 | 2022-06-07 | 四川省明厚天信息技术股份有限公司 | Method, equipment and storage medium for inquiring acquisition record of interface access log |
-
2022
- 2022-10-11 CN CN202211241042.5A patent/CN115328755A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100049747A1 (en) * | 2008-08-19 | 2010-02-25 | Samsung Electronics Co., Ltd. | Apparatus and method for storing log in a thread oriented logging system |
| CN103577310A (en) * | 2012-07-20 | 2014-02-12 | 腾讯科技（深圳）有限公司 | Method and device for recording software debugging logs |
| CN108052675A (en) * | 2017-12-28 | 2018-05-18 | 惠州Tcl家电集团有限公司 | Blog management method, system and computer readable storage medium |
| CN110175154A (en) * | 2019-05-17 | 2019-08-27 | 深圳前海微众银行股份有限公司 | A kind of processing method of log recording, server and storage medium |
| CN112231274A (en) * | 2020-10-16 | 2021-01-15 | 京东数字科技控股股份有限公司 | Log summarizing method and device, electronic equipment and storage medium |
| CN112948136A (en) * | 2021-02-02 | 2021-06-11 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Method for implementing asynchronous log record of embedded operating system |
| CN114595201A (en) * | 2022-03-23 | 2022-06-07 | 四川省明厚天信息技术股份有限公司 | Method, equipment and storage medium for inquiring acquisition record of interface access log |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110245078B (en) | Software pressure testing method and device, storage medium and server | |
| CN110502494B (en) | Log processing method and device, computer equipment and storage medium | |
| US9286106B1 (en) | Scheduling periodic tasks with dependencies and determining improper loop dependencies between tasks placed in a waiting tasks set and in a unfinished dependent tasks set | |
| US9251032B2 (en) | Method, computer program, and information processing apparatus for analyzing performance of computer system | |
| US8135979B2 (en) | Collecting network-level packets into a data structure in response to an abnormal condition | |
| US20070263541A1 (en) | Automatic correlation of service level agreement and operating level agreement | |
| US9740991B2 (en) | Calculating in-flight metrics for non-interruptible business transactions | |
| US20050055673A1 (en) | Automatic database diagnostic monitor architecture | |
| US20100229182A1 (en) | Log information issuing device, log information issuing method, and program | |
| KR20010079612A (en) | Systems and methods for monitoring distributed applications using diagnostic information | |
| US11782609B2 (en) | Method and apparatus for auditing abnormality of block device in cloud platform, device, and storage medium | |
| EP3384391B1 (en) | Real-time change data from disparate sources | |
| US20030014507A1 (en) | Method and system for providing performance analysis for clusters | |
| CN110489317B (en) | Cloud system task operation fault diagnosis method and system based on workflow | |
| Cao et al. | Tcprt: Instrument and diagnostic analysis system for service quality of cloud databases at massive scale in real-time | |
| CN111522870B (en) | Database access method, middleware and readable storage medium | |
| US8407336B2 (en) | Supporting apparatus for analyzing computer operation status | |
| CN115328755A (en) | Concurrent log recording method and system | |
| KR101968575B1 (en) | Method for automatic real-time analysis for bottleneck and apparatus for using the same | |
| US11243857B2 (en) | Executing test scripts with respect to a server stack | |
| SG193013A1 (en) | System and method for processing similar emails | |
| CN112181759A (en) | Method for monitoring micro-service performance and diagnosing abnormity | |
| CN111209130B (en) | Fault processing method, system, equipment and medium based on MySQL master-slave replication cluster | |
| CN113411382B (en) | Real-time data acquisition system and method based on network equipment F5 | |
| WO2022161417A1 (en) | Data query method and apparatus, and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20221111 |