CN109857715B

CN109857715B - Log recording method and system for strengthening error checking support

Info

Publication number: CN109857715B
Application number: CN201910064459.0A
Authority: CN
Inventors: 张甄
Original assignee: Anhui Dingzhongshuke Information Technology Co Ltd
Current assignee: Anhui Dingzhongshuke Information Technology Co Ltd
Priority date: 2019-01-23
Filing date: 2019-01-23
Publication date: 2021-09-21
Anticipated expiration: 2039-01-23
Also published as: CN109857715A

Abstract

The invention discloses a log recording method and a log recording system for strengthening error checking support, which mainly comprise the following steps: the log record constructing module is provided for a program library called by the application program, is used for supplementing necessary information for the log content according to the requirement of the log output module when the application program calls the log, and is constructed into a structured log record; the log transmission module is used for supporting the functions of sending the constructed log record to the log record and receiving the log record, and may support different transmission modes among threads, processes and even across hosts according to the running environment and the deployment mode of the service. The invention screens logs according to conditions to carry out full or partial output by the thinking of buffering log contents and delaying output of a memory when log output conditions are triggered, can output simplified logs in a system range daily without changing other parts of an application program, and realizes two aims of saving disk resources and improving the problem analysis and troubleshooting speed.

Description

Log recording method and system for strengthening error checking support

Technical Field

The invention belongs to the field of software development, and particularly relates to a log recording method and a log recording system for strengthening error troubleshooting support.

Background

At present, a log level is designated in a code or configuration file for logging in a computer system, all log records generated in the running process after service starting are logged in a log file if the log record is higher than or equal to the designated level, and log files are not logged if the log record is lower than the designated level, generally, technicians set the log level allowed to be recorded as DEBUG in order to facilitate checking the correctness of program running and positioning ERROR reasons, performance bottlenecks and the like in a development and test environment, and adjust the level to ERROR or INFO after the program is deployed in a production running environment so as to prevent excessive logs from influencing the overall performance of the system and save the space of a magnetic disk, however, due to the complexity of the system and the use environment thereof, checking logs after ERRORs occur in the production environment often finds that necessary information cannot be acquired due to higher log level, problem analysis and positioning cannot be carried out, and even if the log level is temporarily adjusted to DEBUG in the production environment, errors cannot be repeatedly found, so that in order to meet the requirements of operation of the production environment and the requirements of quick examination and repair of production problems, the real requirement of a technician on the log record of the production system is as follows: the log records output in the normal operation process are as compact as possible, and can prove that a certain request is received and normally processed once, for example, only the log of INFO level is recorded;

when some condition of inconsistent operation and expectation occurs in the processing process of a certain request, such as throwing an exception, overtime or program crash, all logs of the request are recorded, and logs of the level below the INFO generated before an error occurs in the processing process are included.

Disclosure of Invention

In view of the above technical background, the present application provides a logging method and apparatus capable of meeting the above requirements, so as to solve the problems of normal and efficient operation of the production environment and occasional occurrence during operation, and to timely solve the contradiction between the two actual requirements.

In order to achieve the purpose, the invention adopts the technical scheme that:

a log recording method and system for strengthening error checking support mainly comprises:

the log record constructing module is provided for a program library called by the application program, is used for supplementing necessary information for the log content according to the requirement of the log output module when the application program calls the log, and is constructed into a structured log record;

the log transmission module is used for supporting the functions of sending log records to the log records and receiving the log records after the log records are constructed, and possibly supporting different transmission modes among threads, processes and even crossing hosts according to the running environment and the deployment mode of the service;

and the log output module is used for receiving the constructed log record and outputting the log to a log file according to a strategy.

Preferably, the method comprises the steps that when an application program receives a request and finishes the processing process of the request, a field suitable for identifying the request is selected according to the convention, log record content is constructed according to a specified format, the start and the end of the processing process of one request are marked, and a log record library (with the level of INFO) is called; the application program can construct log records with any content in the request processing process, a log record library is called at a required level, the log record library supplements some necessary additional information for the log records, the log records are organized into structured log records and then are sent to a log output module of the log record library, the log output module of the log record library stores the received structured log records in a memory and waits for the request of subsequent log records, and the log output module can output the related log records in the memory to a log file according to different strategies.

Preferably, when a log record of a specified format is received which indicates that a request has been completed, a determination is made based on the received log record in the request: 1.1 if the request has a log record of ERROR or higher level, outputting all log records of the request in time sequence; 1.2 if there is no ERROR or higher level log record for the request, only the INFO level log of the request is output in chronological order.

Preferably, if the log output module finds that a certain incomplete request has exceeded a certain time (for example, half of the timeout time) and does not receive a subsequent log record, it directly outputs all logs of the request and adds a log in a special format to mark that the service processing the request may have a performance problem or has crashed.

Preferably, if a log record of the sharing level is received, all log records of the request are output in time sequence.

Preferably, if a log record is received that does not meet the requirement of the agreed format or the log record for which the request processing starts cannot be found, the method reverts to the ordinary log strategy and directly outputs the log record to the log file.

Compared with the prior art, the invention has the following beneficial effects:

according to the invention, the log is screened according to the conditions to carry out full-volume or partial output when the log output conditions are triggered by the idea of internally buffering the log content and delaying output, the simplified log can be output daily in the system range without changing other parts of an application program except for possibly adjusting the log records of two times of requesting the start and the end of processing, and the aim of outputting the detailed log only when errors occur is fulfilled, meanwhile, two aims of saving disk resources and improving the problem analysis and investigation speed are fulfilled, and a better use prospect is brought.

Drawings

FIG. 1 is a basic flowchart of a main thread of a logging method and system for enhancing error checking support according to the present invention;

FIG. 2 is a basic flowchart of a timeout check thread of the log recording method and system for enhanced error checking support according to the present invention.

Detailed Description

The present invention is further described below, and the following examples are only used to more clearly illustrate the technical solutions of the present invention, but not to limit the scope of the present invention.

Example 1

Because the log record construction module and the log transmission module are realized only for realizing a single function, excessive logic judgment is not involved, and the core thought of the invention is not influenced, the following explanation is directed to the thought and the flow of the log output module;

the log output module is realized by two threads, wherein one thread is a thread for executing log output logic and is called a main thread; the other is the thread that performs the log timeout check, called here the timeout check thread.

The basic flow of the main thread is

1) The log recording method in the embodiment includes the steps of: when receiving the log record, the method enters step S301 to check whether the log record conforms to the agreed format, if not, the log is directly output, and if so, the method enters step S302;

2) step S302: if the log record mark request processing starts, caching the log record and waiting for receiving the next log record, if not, entering step S303;

3) step S303: checking whether a log record cached by the request exists in the memory, if not, directly outputting the log, and if so, entering the step S304;

4) step S304: if the log record flag request processing process is finished, step S305 is entered, otherwise, the log record is cached;

5) step S305: checking all cached log records of the request, if ERROR or higher level logs exist, outputting all cached logs of the request, and if not, outputting only INFO level logs;

6) step S306: and clearing the output logs cached in the memory.

The basic flow of the timeout checking thread is

1) Step S401: the overtime check thread checks the time of the last log record requested in all processing cached in the memory at regular intervals, if the difference between the time and the current time exceeds a specified threshold value, the step S402 is entered, otherwise, no processing is performed;

2) step S402: the timeout check thread sends an EXCEPTION level log in a manner that mimics the log record construction module, records that a timeout or crash problem may occur and triggers log output.

The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims

1. A log recording method and system for strengthening error checking support is characterized by comprising the following steps:

and the log output module is used for receiving the constructed log records, outputting the log records to a log file according to a strategy, recording the log records output in the normal operation process, proving that a certain request is received and normally processed once, and recording only the logs of the INFO level.

2. The method and system of claim 1 for logging with enhanced error troubleshooting support, wherein: includes such steps as choosing the fields suitable for marking a request according to convention when an application program receives the request and finishes the processing procedure of the request, creating log record content according to a specified format, marking the beginning and the end of the processing process of a request, calling a log record library with the level of INFO, constructing the log record of any content by an application program in the processing process of the request, and calls a logging library at a required level, the logging library supplements additional information to the log record, organized into structured log records, and then sent to a log output module of a log record library, the log output module of the log record library stores the received structured log records in a memory, and waiting for the request of subsequent log records, and outputting the related log records in the memory to a log file by the log output module according to different strategies according to the condition.

3. The method and system of claim 2 for logging with enhanced error troubleshooting support, wherein: when receiving the log record with the appointed format which marks the completion of a request, judging according to the received log record in the request: 1.1 if the request has a log record of ERROR or higher level, outputting all log records of the request in time sequence; 1.2 if there is no ERROR or higher level log record for the request, only the INFO level log of the request is output in chronological order.

4. The method and system of claim 2 for logging with enhanced error troubleshooting support, wherein: if the log output module finds that a certain incomplete request exceeds a certain time and does not receive the subsequent log record, all logs of the request are directly output, and a log with a special format is added to mark that the service for processing the request may have performance problems or be broken down.

5. The method and system of claim 2 for logging with enhanced error troubleshooting support, wherein: if a log record at the EXCEPTION level is received, all log records of the request are output in chronological order.

6. The method and system of claim 2 for logging with enhanced error troubleshooting support, wherein: and if a log record which does not meet the requirement of the agreed format is received or the log record which is requested to start processing cannot be found, returning to the common log strategy and directly outputting the log record to the log file.