WO2020010702A1 - Log search method and system, computer device, and storage medium - Google Patents

Log search method and system, computer device, and storage medium Download PDF

Info

Publication number
WO2020010702A1
WO2020010702A1 PCT/CN2018/106687 CN2018106687W WO2020010702A1 WO 2020010702 A1 WO2020010702 A1 WO 2020010702A1 CN 2018106687 W CN2018106687 W CN 2018106687W WO 2020010702 A1 WO2020010702 A1 WO 2020010702A1
Authority
WO
WIPO (PCT)
Prior art keywords
log
information
request
unique identifier
search
Prior art date
Application number
PCT/CN2018/106687
Other languages
French (fr)
Chinese (zh)
Inventor
杨天鹏
金龙
郝振亚
汪昌帅
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Publication of WO2020010702A1 publication Critical patent/WO2020010702A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication

Definitions

  • the present application relates to the technical field of log data processing, and in particular, to a log search method, system, computer device, and storage medium.
  • the log file is a log event record of network equipment, systems, and service programs that are generated during operation. Each line of the log contains descriptions of related operations such as date, time, user, and action.
  • a variety of log files are designed in various operating systems, such as application logs, security logs, system logs, Scheduler service logs, FTP logs, WWW logs, DNS server logs, and so on. When users perform some operations on the system, these log files usually record some related content of our operations, which are very useful for system security staff. For example, if the user performs an IPC probe on the system, the system will quickly record the IP, time, user name, etc. used by the probe in the security log. After the FTP probe, the IP and time will be recorded in the FTP log. , Username used for detection, etc.
  • the log data in the log file is of great value for network security. Developers can make various judgments and analysis based on the error log in the log data.
  • a log search method includes the following steps:
  • the client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
  • a log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
  • Intercepting the log information output by the log framework splitting the log information into request time, request information, unique identification and log content, and calling an application programming interface API to store the split log information;
  • a search interface is preset on the server front-end page, and a search condition field is set on the search interface.
  • the search condition field contains at least one field information corresponding to the request information.
  • a log search system includes the following units:
  • the client request interception unit is configured to intercept the client request at the server backend, obtain the request information in the client request, and generate the client request by calling a unique identification code UUID generator preset on the server backend. The corresponding unique identifier, and storing the request information and the unique identifier in a log record MDC tool;
  • the preset log frame unit is configured to preset a log frame on the back end of the server, the log frame contains configuration information including a request time, request information, and unique identifier, and the request information in the MDC tool And a unique identifier assigned to the request information and the unique identifier in the log frame;
  • the storage log information unit is configured to intercept the log information output by the log framework, split the log information into a request time, request information, unique identifier and log content, and call an application programming interface API to split the log information.
  • Log information for storage
  • the search unit is configured to preset a search interface on a server front-end page, and a search condition field is set on the search interface, and the search condition field contains at least one piece of field information corresponding to the request information. After the field information search request, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
  • a computer device includes a memory and a processor.
  • the memory stores computer-readable instructions.
  • the processor causes the processor to perform the following steps:
  • the client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
  • a log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
  • Intercept the log information output by the log framework split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
  • a search interface is preset on the server front-end page, and a search condition field is set on the search interface.
  • the search condition field contains at least one field information corresponding to the request information.
  • a storage medium storing computer-readable instructions.
  • the one or more processors execute the following steps:
  • the client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
  • a log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
  • Intercept the log information output by the log framework split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
  • a search interface is preset on the server front-end page, and a search condition field is set on the search interface.
  • the search condition field contains at least one field information corresponding to the request information.
  • the above log search method, device, computer equipment, and storage medium include intercepting a client request at the server backend, obtaining the request information in the client request, and generating a client request by calling a unique identification code UUID generator preset on the server backend
  • the corresponding unique identifier stores the request information and the unique identifier in the log record MDC tool.
  • the server backend presets a log frame.
  • the log frame contains configuration information including the request time, the request information, and the unique identifier.
  • Request information and unique identifier are assigned to the request information and unique identifier in the logging framework; intercept the log information output by the logging framework, split the log information into request time, request information, unique identifier, and log content, and call the application programming interface API
  • the split log information is stored; a search interface is preset on the server front-end page, and a search condition field is set on the search interface.
  • the search condition field contains at least one piece of field information corresponding to the requested information. After the search request, search the stored day Log information, and display the log information corresponding to the field information in the form of a list on the search interface.
  • This application collects logs based on MDC technology. It collects all logs requested by each client and records the unique identifier. Developers can directly view the logs after searching through the front-end page. This application can greatly reduce log search problems and improve Search efficiency for error logs.
  • FIG. 1 is a flowchart of a log search method according to an embodiment of the present application
  • FIG. 2 is a flowchart of step S1 in FIG. 1;
  • FIG. 3 is a structural diagram of a log search system according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a module for intercepting a client request unit in FIG. 3.
  • FIG. 1 is a flowchart of a log search method according to an embodiment of the present application. As shown in FIG. 1, the log search method includes the following steps:
  • Step S1 intercepting the client request: intercepting the client request at the server backend, obtaining the request information in the client request, and generating a unique identifier corresponding to the client request by calling a unique identification code UUID generator preset on the server backend, and The request information and unique ID are stored in the logging MDC tool.
  • the request sent by the client from the browser generally includes URL information, request parameters, request host IP and request information of the client's corresponding project name.
  • the server in this step may be a server of a log processing platform. The above request information is intercepted at the server backend. In order to distinguish client requests, the log processing platform binds a unique identifier to each request.
  • a UUID generator preset on the back end of the server is used.
  • UUID is an abbreviation of Universally Unique Identifier. It is a standard for software construction. It is also used by the Open Software Foundation for distributed computing. Part of the environmental field. The purpose is to allow all elements in the distributed system to have unique identification information, without the need to specify the identification information through the central control terminal. In this case, there is no need to consider duplicate names when the database is created problem. Therefore, in this step, a unique identifier corresponding to each client request is generated by the UUID generator to ensure the uniqueness of each request.
  • the logging MDC tool is a function that is convenient for recording logs in a multi-threaded condition, that is, a multi-threaded method is used to process requests from multiple users. For example, in a web application server, when a user visits a page, the application server creates a new thread to process the request, or reuses an existing thread from the thread pool. During a user's session, there are multiple threads that have processed the user's request. This makes it difficult to distinguish the logs corresponding to different users. It is very troublesome when you need to track the relevant log records of a user in the system. The MDC tool in this step can solve the above problems.
  • the MDC tool can be regarded as a hash table bound to the current thread, and key-value pairs can be added to it.
  • MDC tools contain content that can be accessed by code executing in the same thread.
  • the child thread of the current thread will inherit the contents of the MDC tool in its parent thread.
  • the contents of the MDC tool are saved by the program at the appropriate time. For a web application, this data is usually saved at the very beginning of the request being processed. It is ensured that the values stored in the MDC tool between processes under multiple processes have no mutual influence. This step is based on the advantages of the thread log record of the MDC tool. After obtaining the client request, the request information and the unique identifier are stored in the MDC tool for subsequent log frame output log information.
  • Step S2 presetting the log frame:
  • the log frame is preset on the server backend.
  • the log frame contains configuration information including request time, request information, and unique identifier.
  • the request information and unique identifier in the MDC tool are assigned to the log frame. Request information and unique identification.
  • the log framework preset in the server includes log frameworks such as log4j or log4j2.
  • Log4j and log4j2 are both open source projects of Apache. By using Log4j or log4j2, you can control the destination of log information delivery, as well as the output format of each log. By defining the level of each log message, you can control the log generation process in more detail.
  • the input destination, output format, level, etc. are all flexibly configured through a configuration file without modifying the application code.
  • Log4j uses the log4j.properties file as the main configuration file
  • log4j2 uses log4j2.xml or log4j2.yml or log4j2.json as the main configuration file.
  • configuration information including the request information and unique identification is added to the configuration file. So far, all output log information requested by the client will be carried with the configuration information in the configuration file.
  • the above attributes that is, the log information output by the log framework includes request information and a unique identifier.
  • Step S3 Store log information: intercept the log information output by the log framework, split the log information into request time, request information, unique identifier, and log content, and call the application programming interface API to store the split log information.
  • the output log information is intercepted and intercepted and split.
  • the log information is divided into the request time, request information, unique identifier, and log content, and the corresponding call is called. API for storage. If the database is used to store logs, the logs are inserted into the database through SQL scripts. If the search engine Elasticsearch is used for storage, the storage method of Elasticsearch is called for storage, and Elasticsearch storage engine is preferably used for storage, so that subsequent search log information is faster.
  • Step S4 Search and display log information: a search interface is preset on the server front-end page, and a search condition field is set on the search interface.
  • the search condition field contains at least one field information corresponding to the requested information. After the request, search the stored log information, and display the log information corresponding to the field information in the form of a list on the search interface.
  • This step provides developers with a search function for log information through a preset search interface.
  • the search interface searches through field information corresponding to at least one request message, so that developers can quickly query all the log information of the request and quickly locate whether there is a problem with the request and where the problem lies.
  • the specific search condition fields include URL address, request time interval, request parameters, request host IP, project name, and unique identifier. The URL address is required and the other search criteria fields are optional.
  • the search method is selected according to the storage method of step S3. If the database is used to store logs, this step uses a SQL script query to search the database. If the search engine Elasticsearch is used for storage, the query is performed by using Elasticsearch's query expression QueryDSL. Due to the large amount of data in the log information, the query speed stored in the database is slow, which is not conducive to searching. Therefore, the preferred log information is stored in the Elasticsearch search engine, which can quickly search the log data.
  • the client request is intercepted, the request information and the unique identifier are stored in the MDC tool, the unified log output format is configured, the request information collected in the MDC tool is output, and the output log information is formatted and stored in the log.
  • the front end provides a search interface, which searches for log information through the search condition field and displays it to developers. The entire log storage and search process can improve the efficiency of developers in handling online errors, reduce the developer's frequent search for logs, and frequent connection lines. On the device.
  • step S1 when intercepting a client request in step S1, the following steps are adopted:
  • Step S101 intercept and obtain the request information: when the server backend intercepts the client request, it intercepts the client request by calling the filter method of the preset spring program, and obtains the client request including the URL address, the request parameters, and requests the host IP and Request information for the project name corresponding to the client.
  • the spring program is an open-source design-level framework that solves the problem of loose coupling between the business logic layer and other layers.
  • Spring program is a lightweight Java development framework, that is, Spring program is a layered JavaSE / EE full-stack (one-stop) lightweight open source framework.
  • a spring program is set in the server backend in advance, and by calling the filter method of the spring program, all URL requests are intercepted and the request information is obtained through the chain processing of the filter.
  • Step S102 generating a unique identifier: calling a UUID generator to generate a unique identifier.
  • this step uses a UUID generator to generate a unique identifier for each request to distinguish each client request.
  • step S103 the request information and the unique identifier are stored: the request information and the unique identifier are stored in the MDC tool by using the MDC.put method.
  • the MDC tool is set in advance on the server back end, and the unique identifier generated and the obtained request information are stored in the MDC tool.
  • the specific storage method is as follows:
  • all URL requests are intercepted and the request information is obtained by calling the filter method of the spring program, and a unique identifier is generated for each request by the UUID generator to distinguish each client request, and the request information and The unique identifier is stored for the purpose of obtaining and storing all logs of each client request.
  • the log framework when the log framework outputs the log information, after the log framework obtains the request information and the unique identifier in the MDC tool according to the configuration information, the log framework outputs the log information including the request time, the request information, and the unique identifier.
  • the REQUEST_UUID, REQUEST_URL, REQUEST_IP, REQUEST_PARAMS, and APPKEY in the configuration information added in this embodiment are the request information and the unique identifier stored in the MDC tool. All the output log information for each request will bring the above-mentioned types. Attributes.
  • intercepting the log information output by the log framework, and splitting the log information into request time, request information, unique identifier and log content includes: intercepting the logs output by the log framework by calling the preset aop method of the spring program Information; the log information is split into request time, request information, unique identifier, and log content by calling the split method of the preset spring program.
  • the aop method is an important content in a spring program. It defines an entry point for an existing program, and then cuts in different execution contents before and after it. Based on the aop method, it does not destroy the original program logic, so it can be very good for business. Each part of the logic is isolated, so that the coupling between the parts of the business logic is reduced, the program is reusable, and the development efficiency is improved. In this step, the log information output by the logging framework is intercepted by calling the aop method.
  • the split method can do is split a string into substrings and return the result as an array of strings.
  • the log information is intercepted and split by calling the aop method and the split method in the spring program, and the split content is provided for subsequent display of the log information.
  • the application programming interface API when called to store the split log information, the following steps are taken: setting a logging table in the database, setting a split field in the logging table, and the split field including the request Time field, request information field, unique identification field, and log content field.
  • Setting a logging table in the database setting a split field in the logging table, and the split field including the request Time field, request information field, unique identification field, and log content field.
  • Call the API to execute the SQL script insert the split log information into the split fields in turn, and complete the storage of the log information.
  • SQL is the abbreviation of Structured Query Language, which is a database query and programming language. It is used to access data and query, update, and manage relational database systems. It is also the extension of database script files. SQL does not require the user to specify the method of storing data, nor does the user need to know the specific data storage method. Therefore, different database systems with completely different underlying structures can use the same structured query language as the interface for data input and management.
  • the query language statement can be nested, which makes it extremely flexible and powerful. In this embodiment, log information is stored by executing a SQL script.
  • the application programming interface API when the application programming interface API is called to store the split log information, the following steps are used:
  • the API uses the search engine Elasticsearch, calls Elasticsearch, and stores the split log information in Elasticsearch.
  • ElasticSearch is a Lucene-based search server. Elasticsearch provides a full-text search engine with distributed multi-user capabilities, based on a RESTful web interface. Elasticsearch is developed in Java and released as an open source under the Apache license. It is a popular enterprise search engine that can achieve real-time search, is stable, reliable, fast, and easy to install and use.
  • This application preferably uses the search engine Elasticsearch to store log information to achieve fast storage and subsequent search functions.
  • a log search system is proposed, as shown in FIG. 3, and includes the following units:
  • the client request interception unit is configured to intercept the client request on the server backend, obtain the request information in the client request, and generate a unique identifier corresponding to the client request by calling the unique identifier UUID generator preset on the server backend. Request information and unique identification are stored in the log record MDC tool;
  • the preset log frame unit is set to preset a log frame on the server backend.
  • the log frame contains configuration information including request time, request information, and unique identifier.
  • the request information and unique identifier in the MDC tool are assigned to the log frame. Request information and unique identification;
  • the storage log information unit is configured to intercept the log information output by the log framework, split the log information into request time, request information, unique identifier, and log content, and call the application programming interface API to store the split log information;
  • the search unit is set to preset a search interface on a server front-end page, and a search condition field is set on the search interface.
  • the search condition field contains at least one field information corresponding to the request information. When a search request containing field information is received, the search is performed. Store the log information, and display the log information corresponding to the field information in the form of a list on the search interface.
  • the intercepting client request unit includes:
  • the interception module is configured to intercept the client request by calling the filter method of a preset spring program, and obtain the request information including the URL address, the request parameter, the request host IP and the corresponding project name of the client in the client request;
  • Generate unique identification module set to call UUID generator to generate unique identification
  • the storage module is configured to store the request information and the unique identifier into the MDC tool by using the MDC.put method.
  • the log information unit is further configured to log information obtained from the MDC tool according to the configuration information and the unique identification, and the log frame outputs log information including the request time, the request information and the unique identification.
  • the log information storage unit includes: an interception log information module configured to intercept the log information output by the log framework by calling the aop method of a preset spring program; a split log information module configured to call The split method of the spring program is used to split the log information into request time, request information, unique identifier, and log content.
  • the log information storage unit further includes: a log record table module configured to set a log record table in a database, and a split field in the log record table.
  • the split field includes a request time field and request information. Field, unique identification field, and log content field; insert the log information module, set to call the API to execute the SQL script, and insert the split log information into the split fields in turn to complete the storage of the log information.
  • the log information storage unit further includes a search engine module, and the API is set to use the search engine Elasticsearch, call Elasticsearch, and store the split log information in Elasticsearch.
  • a computer device which includes a memory and a processor.
  • the memory stores computer-readable instructions.
  • the processor is caused to execute the log search method in the foregoing embodiments. Steps.
  • a storage medium storing computer-readable instructions.
  • the one or more processors are caused to execute the log search method in the foregoing embodiments. Steps.
  • the storage medium may be a non-volatile storage medium.
  • the program may be stored in a computer-readable storage medium.
  • the storage medium may include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Storage Device Security (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The present application relates to the technical field of log data processing, and in particular to a log search method and system, a computer device, and a storage medium. The log search method comprises: intercepting and obtaining request information at a server back-end; generating a unique identifier by calling a UUID generator preset at the server back-end; and storing the request information and the unique identifier in a log recording MDC tool. A log frame is preset at the server back-end, log information outputted by the log frame is intercepted, and the log information is split and stored. A search interface is preset on a page of a server front-end, and the log information is displayed in the form of a list. A log collection scheme based on an MDC technology in the present application can greatly reduce search problems of logs and improve search efficiency of an error log.

Description

日志搜索方法、系统、计算机设备和存储介质Log search method, system, computer equipment and storage medium
本申请要求于2018年07月11日提交中国专利局、申请号为201810756846.6、发明名称为“日志搜索方法、系统、计算机设备和存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority from a Chinese patent application filed with the Chinese Patent Office on July 11, 2018, with an application number of 201810756846.6 and an invention name of "Log Search Method, System, Computer Equipment, and Storage Medium", the entire contents of which are incorporated by reference In this application.
技术领域Technical field
本申请涉及日志数据处理技术领域,尤其涉及一种日志搜索方法、系统、计算机设备和存储介质。The present application relates to the technical field of log data processing, and in particular, to a log search method, system, computer device, and storage medium.
背景技术Background technique
日志文件是网络设备、系统及服务程序等,在运作时产生的一个log的事件记录,每一行日志都记载着日期、时间、使用者及动作等相关操作的描述。各类操作系统中都会设计各种各样的日志文件,如应用程序日志、安全日志、系统日志、Scheduler服务日志、FTP日志、WWW日志、DNS服务器日志等等。用户在系统上进行一些操作时,这些日志文件通常会记录下我们操作的一些相关内容,这些内容对系统安全工作人员相当有用。比如用户对系统进行了IPC探测,系统就会在安全日志里迅速地记下探测者探测时所用的IP、时间、用户名等,用FTP探测后,就会在FTP日志中记下IP、时间、探测所用的用户名等。日志文件中的日志数据对于实现网络安全的价值非常大,开发人员可以根据日志数据中的错误日志进行各种判断和分析。The log file is a log event record of network equipment, systems, and service programs that are generated during operation. Each line of the log contains descriptions of related operations such as date, time, user, and action. A variety of log files are designed in various operating systems, such as application logs, security logs, system logs, Scheduler service logs, FTP logs, WWW logs, DNS server logs, and so on. When users perform some operations on the system, these log files usually record some related content of our operations, which are very useful for system security staff. For example, if the user performs an IPC probe on the system, the system will quickly record the IP, time, user name, etc. used by the probe in the security log. After the FTP probe, the IP and time will be recorded in the FTP log. , Username used for detection, etc. The log data in the log file is of great value for network security. Developers can make various judgments and analysis based on the error log in the log data.
但是,当某一项目越来越大,项目的集群节点越来越多的时候,整个线上系统如果出现某些未知问题,开发人员需要到每台设备的日志文件中去找错误日志,并且错误的日志还不是连续的,可能一个问题需要反复的找几次错误日志,解决问题的效率比较低,操作也比较繁琐。However, when a certain project is getting bigger and larger, and the project has more and more cluster nodes, if some unknown problems occur in the entire online system, developers need to find the error log in the log file of each device, and The error log is not continuous. It may be necessary to repeatedly find the error log for a problem, the efficiency of solving the problem is relatively low, and the operation is tedious.
发明内容Summary of the invention
有鉴于此,有必要针对错误日志的记录不是连续的,且同一个问题查找繁 琐的问题,提供一种日志搜索方法、系统、计算机设备和存储介质。In view of this, it is necessary to provide a log search method, a system, a computer device, and a storage medium for the records of the error log that are not continuous and for the same problem to find tedious problems.
一种日志搜索方法,包括如下步骤:A log search method includes the following steps:
在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;A log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;Intercepting the log information output by the log framework, splitting the log information into request time, request information, unique identification and log content, and calling an application programming interface API to store the split log information;
在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。A search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the request information. When a search request is received that includes the field information, After that, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
一种日志搜索系统,包括如下单元:A log search system includes the following units:
拦截客户端请求单元,设置为在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request interception unit is configured to intercept the client request at the server backend, obtain the request information in the client request, and generate the client request by calling a unique identification code UUID generator preset on the server backend. The corresponding unique identifier, and storing the request information and the unique identifier in a log record MDC tool;
预设日志框架单元,设置为在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;The preset log frame unit is configured to preset a log frame on the back end of the server, the log frame contains configuration information including a request time, request information, and unique identifier, and the request information in the MDC tool And a unique identifier assigned to the request information and the unique identifier in the log frame;
存储日志信息单元,设置为拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;The storage log information unit is configured to intercept the log information output by the log framework, split the log information into a request time, request information, unique identifier and log content, and call an application programming interface API to split the log information. Log information for storage;
搜索单元,设置为在服务器前端页面预设搜索界面,所述搜索界面上设置 搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。The search unit is configured to preset a search interface on a server front-end page, and a search condition field is set on the search interface, and the search condition field contains at least one piece of field information corresponding to the request information. After the field information search request, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
一种计算机设备,包括存储器和处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被所述处理器执行时,使得所述处理器执行以下步骤:A computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor causes the processor to perform the following steps:
在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;A log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;Intercept the log information output by the log framework, split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。A search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the request information. When a search request is received that includes the field information, After that, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
一种存储有计算机可读指令的存储介质,所述计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行以下步骤:A storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:
在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;A log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;Intercept the log information output by the log framework, split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。A search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the request information. When a search request is received that includes the field information, After that, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
上述日志搜索方法、装置、计算机设备和存储介质,包括在服务器后端拦截客户端请求,获取客户端请求中的请求信息,通过调用预设在服务器后端的唯一识别码UUID生成器生成客户端请求对应的唯一标识,将请求信息和唯一标识存入日志记录MDC工具中;在服务器后端预设日志框架,日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将MDC工具中的请求信息和唯一标识赋值给日志框架中的请求信息及唯一标识;拦截日志框架输出的日志信息,将日志信息拆分成请求时间、请求信息、唯一标识及日志内容,调用应用程序编程接口API将拆分后的日志信息进行存储;在服务器前端页面预设搜索界面,搜索界面上设置搜索条件字段,搜索条件字段中含有至少一项请求信息对应的字段信息,当接收到包含有字段信息的搜索请求后,搜索存储的日志信息,并将字段信息对应的日志信息在搜索界面以列表的形式进行展示。本申请基于MDC技术的日志收集方案,把每次客户端请求的所有日志收集在一起并记录唯一标识,开发人员直接可以通过前端页面搜索后查看日志,本申请能大大减轻日志的搜索问题,提高错误日志的搜索效率。The above log search method, device, computer equipment, and storage medium include intercepting a client request at the server backend, obtaining the request information in the client request, and generating a client request by calling a unique identification code UUID generator preset on the server backend The corresponding unique identifier stores the request information and the unique identifier in the log record MDC tool. The server backend presets a log frame. The log frame contains configuration information including the request time, the request information, and the unique identifier. Request information and unique identifier are assigned to the request information and unique identifier in the logging framework; intercept the log information output by the logging framework, split the log information into request time, request information, unique identifier, and log content, and call the application programming interface API The split log information is stored; a search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one piece of field information corresponding to the requested information. After the search request, search the stored day Log information, and display the log information corresponding to the field information in the form of a list on the search interface. This application collects logs based on MDC technology. It collects all logs requested by each client and records the unique identifier. Developers can directly view the logs after searching through the front-end page. This application can greatly reduce log search problems and improve Search efficiency for error logs.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
通过阅读下文优选实施方式的详细描述,各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的,而并不认为是对本申请的限制。Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the detailed description of the preferred embodiments below. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the present application.
图1为本申请一个实施例中的日志搜索方法的流程图;FIG. 1 is a flowchart of a log search method according to an embodiment of the present application; FIG.
图2为图1中步骤S1流程图;FIG. 2 is a flowchart of step S1 in FIG. 1;
图3为本申请一个实施例中的日志搜索系统的结构图;3 is a structural diagram of a log search system according to an embodiment of the present application;
图4为图3中的拦截客户端请求单元的模块示意图。FIG. 4 is a schematic diagram of a module for intercepting a client request unit in FIG. 3.
具体实施方式detailed description
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。In order to make the purpose, technical solution, and advantages of the present application clearer, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.
本技术领域技术人员可以理解,除非特意声明,这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是,本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件,但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "the" may include plural forms. It should be further understood that the word "comprising" used in the specification of the present application refers to the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and / or groups thereof.
图1为本申请一个实施例中的日志搜索方法的流程图,如图1所示,日志搜索方法包括如下步骤:FIG. 1 is a flowchart of a log search method according to an embodiment of the present application. As shown in FIG. 1, the log search method includes the following steps:
步骤S1,拦截客户端请求:在服务器后端拦截客户端请求,获取客户端请求中的请求信息,通过调用预设在服务器后端的唯一识别码UUID生成器生成客户端请求对应的唯一标识,将请求信息和唯一标识存入日志记录MDC工具中。Step S1, intercepting the client request: intercepting the client request at the server backend, obtaining the request information in the client request, and generating a unique identifier corresponding to the client request by calling a unique identification code UUID generator preset on the server backend, and The request information and unique ID are stored in the logging MDC tool.
客户端从浏览器发送的请求一般包括URL地址、请求参数,请求主机IP及客户端对应的项目名称的请求信息。本步骤的服务器可以是日志处理平台的服务器。在服务器后端拦截上述请求信息,为了区别客户端请求,日志处理平台对每次的请求均绑定一项唯一标识。唯一标识生成时,采用调用预设在服务器后端的UUID生成器,UUID是通用唯一识别码(Universally Unique Identifier)的缩写,是一种软件建构的标准,亦为开放软件基金会组织在分布式计算环境领域的一部分。其目的,是让分布式系统中的所有元素,都能有唯一的辨识信息,而不需要通过中央控制端来做辨识信息的指定,在这样的情况下,不需考虑数据库创建时的名称重复问题。因此本步骤通过UUID生成器生成每个客户端 请求对应的唯一标识,保证每次请求的唯一性。The request sent by the client from the browser generally includes URL information, request parameters, request host IP and request information of the client's corresponding project name. The server in this step may be a server of a log processing platform. The above request information is intercepted at the server backend. In order to distinguish client requests, the log processing platform binds a unique identifier to each request. When the unique identifier is generated, a UUID generator preset on the back end of the server is used. UUID is an abbreviation of Universally Unique Identifier. It is a standard for software construction. It is also used by the Open Software Foundation for distributed computing. Part of the environmental field. The purpose is to allow all elements in the distributed system to have unique identification information, without the need to specify the identification information through the central control terminal. In this case, there is no need to consider duplicate names when the database is created problem. Therefore, in this step, a unique identifier corresponding to each client request is generated by the UUID generator to ensure the uniqueness of each request.
日志记录MDC工具是一种方便在多线程条件下记录日志的功能,即采用多线程的方式来处理多个用户的请求。例如在Web应用服务器中,当用户访问某个页面时,应用服务器会创建一个新的线程来处理该请求,或者从线程池中复用已有的线程。在一个用户的会话存续期间,存在有多个线程处理过该用户的请求。这使得比较难以区分不同用户所对应的日志。当需要追踪某个用户在系统中的相关日志记录时,就会非常麻烦。而本步骤中的MDC工具可解决上述问题,MDC工具可以看成是一个与当前线程绑定的哈希表,可以往其中添加键值对。MDC工具中包含的内容可以被同一线程中执行的代码所访问。当前线程的子线程会继承其父线程中的MDC工具的内容。当需要记录日志时,只需要从MDC工具中获取所需的信息即可。MDC工具的内容则由程序在适当的时候保存进去。对于一个Web应用来说,通常是在请求被处理的最开始保存这些数据。确保了多进程下进程之间在MDC工具存放的值是没有相互的影响。本步骤基于MDC工具的线程日志记录的优点,在获取客户端请求后将请求信息和唯一标识存入MDC工具中,以供后续的日志框架输出日志信息使用。The logging MDC tool is a function that is convenient for recording logs in a multi-threaded condition, that is, a multi-threaded method is used to process requests from multiple users. For example, in a web application server, when a user visits a page, the application server creates a new thread to process the request, or reuses an existing thread from the thread pool. During a user's session, there are multiple threads that have processed the user's request. This makes it difficult to distinguish the logs corresponding to different users. It is very troublesome when you need to track the relevant log records of a user in the system. The MDC tool in this step can solve the above problems. The MDC tool can be regarded as a hash table bound to the current thread, and key-value pairs can be added to it. MDC tools contain content that can be accessed by code executing in the same thread. The child thread of the current thread will inherit the contents of the MDC tool in its parent thread. When you need to record logs, you only need to obtain the required information from the MDC tool. The contents of the MDC tool are saved by the program at the appropriate time. For a web application, this data is usually saved at the very beginning of the request being processed. It is ensured that the values stored in the MDC tool between processes under multiple processes have no mutual influence. This step is based on the advantages of the thread log record of the MDC tool. After obtaining the client request, the request information and the unique identifier are stored in the MDC tool for subsequent log frame output log information.
步骤S2,预设日志框架:在服务器后端预设日志框架,日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将MDC工具中的请求信息和唯一标识赋值给日志框架中的请求信息及唯一标识。Step S2, presetting the log frame: The log frame is preset on the server backend. The log frame contains configuration information including request time, request information, and unique identifier. The request information and unique identifier in the MDC tool are assigned to the log frame. Request information and unique identification.
在服务器中预设的日志框架包括log4j或log4j2等日志框架。Log4j和log4j2均是Apache的一个开源项目,通过使用Log4j或log4j2可以控制日志信息输送的目的地,也可以控制每一条日志的输出格式。通过定义每一条日志信息的级别,能够更加细致地控制日志的生成过程。而输的目的地、输出格式、级别等都通过一个配置文件来灵活地进行配置,而不需要修改应用的代码。Log4j通过log4j.properties的文件作为主配置文件,log4j2通过log4j2.xml或log4j2.yml或者log4j2.json为主配置文件。The log framework preset in the server includes log frameworks such as log4j or log4j2. Log4j and log4j2 are both open source projects of Apache. By using Log4j or log4j2, you can control the destination of log information delivery, as well as the output format of each log. By defining the level of each log message, you can control the log generation process in more detail. The input destination, output format, level, etc. are all flexibly configured through a configuration file without modifying the application code. Log4j uses the log4j.properties file as the main configuration file, and log4j2 uses log4j2.xml or log4j2.yml or log4j2.json as the main configuration file.
本步骤在原有的日志框架的配置信息基础上,在配置文件中,增设请求信息及唯一标识在内的配置信息,至此每次客户端请求的所有输出的日志信息都会带上配置文件中配置的上述若干属性,即日志框架输出的日志信息包含有请 求信息及唯一标识。In this step, on the basis of the configuration information of the original log framework, configuration information including the request information and unique identification is added to the configuration file. So far, all output log information requested by the client will be carried with the configuration information in the configuration file. The above attributes, that is, the log information output by the log framework includes request information and a unique identifier.
步骤S3,存储日志信息:拦截日志框架输出的日志信息,将日志信息拆分成请求时间、请求信息、唯一标识及日志内容,调用应用程序编程接口API将拆分后的日志信息进行存储。Step S3: Store log information: intercept the log information output by the log framework, split the log information into request time, request information, unique identifier, and log content, and call the application programming interface API to store the split log information.
本步骤在服务器后端预设的日志框架输出日志信息时,拦截输出的日志信息,并进行截取拆分处理,把日志信息拆分成请求时间、请求信息、唯一标识及日志内容,并调用对应的API进行存储。如果用数据库存储日志,则通过SQL脚本进行日志插入数据库中。如果用搜索引擎Elasticsearch存储,则调用Elasticsearch的存储方法进行存储,优选采用Elasticsearch存储引擎存储,这样后续搜索日志信息速度更快。In this step, when the preset log framework of the server backend outputs log information, the output log information is intercepted and intercepted and split. The log information is divided into the request time, request information, unique identifier, and log content, and the corresponding call is called. API for storage. If the database is used to store logs, the logs are inserted into the database through SQL scripts. If the search engine Elasticsearch is used for storage, the storage method of Elasticsearch is called for storage, and Elasticsearch storage engine is preferably used for storage, so that subsequent search log information is faster.
步骤S4,搜索及展示日志信息:在服务器前端页面预设搜索界面,搜索界面上设置搜索条件字段,搜索条件字段中含有至少一项请求信息对应的字段信息,当接收到包含有字段信息的搜索请求后,搜索存储的日志信息,并将字段信息对应的日志信息在搜索界面以列表的形式进行展示。Step S4: Search and display log information: a search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the requested information. After the request, search the stored log information, and display the log information corresponding to the field information in the form of a list on the search interface.
在正确的地方输出合适的日志消息,只是合理使用日志的第一步,日志记录的真正作用在于当有问题发生时,能够帮助开发人员很快的定位问题所在。本步骤通过预设搜索界面,为开发人员提供了日志信息的搜索功能。搜索界面通过至少一项请求信息对应的字段信息进行搜索,以供开发人员更快的查询到该次请求的所有日志信息,并快速定位到该次请求是否存在问题及问题所在。具体的搜索条件字段包括URL地址、请求时间区间、请求参数、请求主机IP、项目名称、唯一标识。其中URL地址为必填项,其他搜索条件字段为选填项。Outputting appropriate log messages in the right place is only the first step in using logs properly. The real role of log records is to help developers quickly locate problems when problems occur. This step provides developers with a search function for log information through a preset search interface. The search interface searches through field information corresponding to at least one request message, so that developers can quickly query all the log information of the request and quickly locate whether there is a problem with the request and where the problem lies. The specific search condition fields include URL address, request time interval, request parameters, request host IP, project name, and unique identifier. The URL address is required and the other search criteria fields are optional.
在通过搜索条件字段对存储的日志信息进行搜索时,根据步骤S3的存储方式选择搜索方式,如用数据库存储日志时,则本步骤采用SQL脚本查询的方式在数据库中进行搜索。如用搜索引擎Elasticsearch存储,则通过Elasticsearch的查询表达式Query DSL来进行查询。,因日志信息一般数据量比较大,存储在数据库里面查询速度较慢,不利于搜索。因此优选日志信息存储在Elasticsearch搜索引擎中,可以快速的搜索日志数据。When searching the stored log information through the search condition field, the search method is selected according to the storage method of step S3. If the database is used to store logs, this step uses a SQL script query to search the database. If the search engine Elasticsearch is used for storage, the query is performed by using Elasticsearch's query expression QueryDSL. Due to the large amount of data in the log information, the query speed stored in the database is slow, which is not conducive to searching. Therefore, the preferred log information is stored in the Elasticsearch search engine, which can quickly search the log data.
日志信息在搜索界面以列表的形式进行展示时,具体内容如下表1所示:When the log information is displayed in the form of a list on the search interface, the specific content is shown in Table 1 below:
Figure PCTCN2018106687-appb-000001
Figure PCTCN2018106687-appb-000001
表1Table 1
本实施例通过拦截客户端请求,获取请求信息及唯一标识存储于MDC工具中,配置统一日志输出格式,增加输出MDC工具中收集的请求信息,将输出的日志信息进行格式拆分,存入日志存储中,前端提供搜索界面,通过搜索条件字段搜索日志信息,展示给开发人员查看,整个日志存储和搜索过程,能提高开发人员处理线上错误的效率,减轻开发人员频繁搜索日志,频繁连接线上设备的操作。In this embodiment, the client request is intercepted, the request information and the unique identifier are stored in the MDC tool, the unified log output format is configured, the request information collected in the MDC tool is output, and the output log information is formatted and stored in the log. In storage, the front end provides a search interface, which searches for log information through the search condition field and displays it to developers. The entire log storage and search process can improve the efficiency of developers in handling online errors, reduce the developer's frequent search for logs, and frequent connection lines. On the device.
在一个实施例中,步骤S1,拦截客户端请求时如图2所示,采用如下步骤:In one embodiment, as shown in FIG. 2 when intercepting a client request in step S1, the following steps are adopted:
步骤S101,拦截并获取请求信息:在服务器后端拦截客户端请求时,通过调用预设的spring程序的filter方法拦截客户端请求,获取客户端请求中包括URL地址、请求参数,请求主机IP及客户端对应的项目名称的请求信息。Step S101: intercept and obtain the request information: when the server backend intercepts the client request, it intercepts the client request by calling the filter method of the preset spring program, and obtains the client request including the URL address, the request parameters, and requests the host IP and Request information for the project name corresponding to the client.
spring程序是一个开放源代码的设计层面框架,解决的是业务逻辑层和其他各层的松耦合问题。Spring程序是一个轻量级的Java开发框架,即Spring程序是一个分层的JavaSE/EE full-stack(一站式)轻量级开源框架。本步骤在服务器后端预先设置了spring程序,通过调用spring程序的filter方法,通过filter的链式处理的方式,将所有的URL请求进行拦截并获取请求信息。The spring program is an open-source design-level framework that solves the problem of loose coupling between the business logic layer and other layers. Spring program is a lightweight Java development framework, that is, Spring program is a layered JavaSE / EE full-stack (one-stop) lightweight open source framework. In this step, a spring program is set in the server backend in advance, and by calling the filter method of the spring program, all URL requests are intercepted and the request information is obtained through the chain processing of the filter.
步骤S102,生成唯一标识:调用UUID生成器生成唯一标识。Step S102, generating a unique identifier: calling a UUID generator to generate a unique identifier.
为了区别各客户端请求,本步骤通过UUID生成器对每次请求均生成唯一标识以区别各客户端请求。In order to distinguish each client request, this step uses a UUID generator to generate a unique identifier for each request to distinguish each client request.
步骤S103,存储请求信息和唯一标识:将请求信息和唯一标识采用MDC.put方法存储至MDC工具中。In step S103, the request information and the unique identifier are stored: the request information and the unique identifier are stored in the MDC tool by using the MDC.put method.
在服务器后端预先设置MDC工具,将生成的唯一标识和获取的请求信息存入MDC工具中,具体的存入方式如下:The MDC tool is set in advance on the server back end, and the unique identifier generated and the obtained request information are stored in the MDC tool. The specific storage method is as follows:
MDC.put(”REQUEST_URL”,”URL地址”);MDC.put ("REQUEST_URL", "URL address");
MDC.put(”REQUEST_UUID”,”唯一标识”);MDC.put ("REQUEST_UUID", "Unique ID");
MDC.put(”REQUEST_IP”,”请求主机IP”);MDC.put ("REQUEST_IP", "Request host IP");
MDC.put(”REQUEST_PARAMS”,”请求参数”);MDC.put ("REQUEST_PARAMS", "request parameter");
MDC.put(”APPKEY”,”项目名称”)。MDC.put ("APPKEY", "Project name").
本实施例通过调用spring程序的filter方法将所有的URL请求进行拦截并获取请求信息,通过UUID生成器对每次请求均生成唯一标识以区别各客户端请求,采用MDC.put方法将请求信息和唯一标识进行存储,将客户端每次请求的所有日志进行获取和存储的目的。In this embodiment, all URL requests are intercepted and the request information is obtained by calling the filter method of the spring program, and a unique identifier is generated for each request by the UUID generator to distinguish each client request, and the request information and The unique identifier is stored for the purpose of obtaining and storing all logs of each client request.
在一个实施例中,日志框架输出日志信息时,日志框架根据配置信息获取MDC工具中的请求信息和唯一标识后,日志框架输出包括请求时间、请求信息及唯一标识的日志信息。In one embodiment, when the log framework outputs the log information, after the log framework obtains the request information and the unique identifier in the MDC tool according to the configuration information, the log framework outputs the log information including the request time, the request information, and the unique identifier.
在对日志框架的配置文件进行配置时,增加如下配置信息:When configuring the configuration file of the logging framework, add the following configuration information:
%d{yyyy-MM-dd HH:mm:ss.SSS}-[%thread]%-5level%logger{36}-%msg%n% D {yyyy-MM-dd HH: mm: ss.SSS}-[% thread]%-5level% logger {36}-% msg% n
-%X{REQUEST_UUID}-%X{REQUEST_URL}-%X{REQUEST_IP}-%X{REQUEST_PARA MS}-%X{APPKEY}-% X {REQUEST_UUID}-% X {REQUEST_URL}-% X {REQUEST_IP}-% X {REQUEST_PARA MS}-% X {APPKEY}
本实施例上述增加的配置信息中的REQUEST_UUID、REQUEST_URL、REQUEST_IP、REQUEST_PARAMS及APPKEY即为存储在MDC工具中的请求信息和唯一标识,至此每次请求的所有输出的日志信息均会带上上述几种属性。The REQUEST_UUID, REQUEST_URL, REQUEST_IP, REQUEST_PARAMS, and APPKEY in the configuration information added in this embodiment are the request information and the unique identifier stored in the MDC tool. All the output log information for each request will bring the above-mentioned types. Attributes.
在一个实施例中,拦截日志框架输出的日志信息,将日志信息拆分成请求时间、请求信息、唯一标识及日志内容包括:通过调用预设的spring程序的aop方法切面拦截日志框架输出的日志信息;通过调用预设的spring程序的split方法将日志信息拆分成请求时间、请求信息、唯一标识及日志内容。In one embodiment, intercepting the log information output by the log framework, and splitting the log information into request time, request information, unique identifier and log content includes: intercepting the logs output by the log framework by calling the preset aop method of the spring program Information; the log information is split into request time, request information, unique identifier, and log content by calling the split method of the preset spring program.
aop方法是spring程序中的一个重要内容,它通过对既有程序定义一个切入点,然后在其前后切入不同的执行内容,基于aop方法不会破坏原来程序逻辑,因此它可以很好的对业务逻辑的各个部分进行隔离,从而使得业务逻辑各部分之间的耦合度降低,提高程序的可重用性,同时提高了开发的效率。在本步骤中,通过调用aop方法切面拦截日志框架输出的日志信息。The aop method is an important content in a spring program. It defines an entry point for an existing program, and then cuts in different execution contents before and after it. Based on the aop method, it does not destroy the original program logic, so it can be very good for business. Each part of the logic is isolated, so that the coupling between the parts of the business logic is reduced, the program is reusable, and the development efficiency is improved. In this step, the log information output by the logging framework is intercepted by calling the aop method.
split方法能现的功能是将一个字符串分割为子字符串,然后将结果作为字符串数组返回。在日志框架的配置文件中,可以在配置规则时,通过“-”间隔来隔开日志信息中的各个属性。因此在对日志信息进行拦截后,可直接通过调用split方法对日志信息进行拆分,拆分成请求时间、请求信息、唯一标识及日志内容。What the split method can do is split a string into substrings and return the result as an array of strings. In the configuration file of the logging framework, you can use the "-" interval to separate each attribute in the log information when you configure the rule. Therefore, after the log information is intercepted, the log information can be split directly by calling the split method, and then split into the request time, request information, unique identifier, and log content.
本实施例通过调用spring程序中aop方法和split方法,实现日志信息的拦截和拆分,为后续展示日志信息提供拆分内容。In this embodiment, the log information is intercepted and split by calling the aop method and the split method in the spring program, and the split content is provided for subsequent display of the log information.
在一个实施例中,调用应用程序编程接口API将拆分后的日志信息进行存储时,采用如下步骤:在数据库中设置日志记录表,在日志记录表中设置拆分字段,拆分字段包括请求时间字段、请求信息字段、唯一标识字段及日志内容字段。调用API执行SQL脚本,将拆分后的日志信息依次插入拆分字段中,完成日志信息的存储。In one embodiment, when the application programming interface API is called to store the split log information, the following steps are taken: setting a logging table in the database, setting a split field in the logging table, and the split field including the request Time field, request information field, unique identification field, and log content field. Call the API to execute the SQL script, insert the split log information into the split fields in turn, and complete the storage of the log information.
SQL是结构化查询语言(Structured Query Language)的简称,是一种数据库查询和程序设计语言,用于存取数据以及查询、更新和管理关系数据库系统,同时也是数据库脚本文件的扩展名。SQL不要求用户指定对数据的存放方法,也不需要用户了解具体的数据存放方式,所以具有完全不同底层结构的不同数据库系统,可以使用相同的结构化查询语言作为数据输入与管理的接口,结构化查询语言语句可以嵌套,这使它具有极大的灵活性和强大的功能。本实施例通过执行SQL脚本,完成日志信息的存储。SQL is the abbreviation of Structured Query Language, which is a database query and programming language. It is used to access data and query, update, and manage relational database systems. It is also the extension of database script files. SQL does not require the user to specify the method of storing data, nor does the user need to know the specific data storage method. Therefore, different database systems with completely different underlying structures can use the same structured query language as the interface for data input and management. The query language statement can be nested, which makes it extremely flexible and powerful. In this embodiment, log information is stored by executing a SQL script.
本实施例应用于数据库存储日志时,建立的日志记录表如下表2所示:When this embodiment is applied to a database to store logs, a log record table established is shown in Table 2 below:
Figure PCTCN2018106687-appb-000002
Figure PCTCN2018106687-appb-000002
表2Table 2
在一个实施例中,调用应用程序编程接口API将拆分后的日志信息进行存储时,采用如下步骤:API采用搜索引擎Elasticsearch,调用Elasticsearch,将拆分后的日志信息存储在Elasticsearch中。In one embodiment, when the application programming interface API is called to store the split log information, the following steps are used: The API uses the search engine Elasticsearch, calls Elasticsearch, and stores the split log information in Elasticsearch.
ElasticSearch是一个基于Lucene的搜索服务器。Elasticsearch提供了一个分布式多用户能力的全文搜索引擎,基于RESTful web接口。Elasticsearch是用Java开发的,并作为Apache许可条款下的开放源码发布,是当前流行的企业级搜索引擎,能够达到实时搜索,稳定、可靠、快速,且安装使用方便。ElasticSearch is a Lucene-based search server. Elasticsearch provides a full-text search engine with distributed multi-user capabilities, based on a RESTful web interface. Elasticsearch is developed in Java and released as an open source under the Apache license. It is a popular enterprise search engine that can achieve real-time search, is stable, reliable, fast, and easy to install and use.
由于日志信息的数据量比较大,存储在数据库里面查询速度较慢,不利于后续的搜索,本申请优选使用搜索引擎Elasticsearch存储日志信息,实现快速存储和后续搜索功能。Due to the large amount of log information, the query speed stored in the database is slow, which is not conducive to subsequent searches. This application preferably uses the search engine Elasticsearch to store log information to achieve fast storage and subsequent search functions.
在一个实施例中,提出了一种日志搜索系统,如图3所示,包括如下单元:In one embodiment, a log search system is proposed, as shown in FIG. 3, and includes the following units:
拦截客户端请求单元,设置为在服务器后端拦截客户端请求,获取客户端请求中的请求信息,通过调用预设在服务器后端的唯一识别码UUID生成器生成客户端请求对应的唯一标识,将请求信息和唯一标识存入日志记录MDC工具中;The client request interception unit is configured to intercept the client request on the server backend, obtain the request information in the client request, and generate a unique identifier corresponding to the client request by calling the unique identifier UUID generator preset on the server backend. Request information and unique identification are stored in the log record MDC tool;
预设日志框架单元,设置为在服务器后端预设日志框架,日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将MDC工具中的请求信息和唯一标识赋值给日志框架中的请求信息及唯一标识;The preset log frame unit is set to preset a log frame on the server backend. The log frame contains configuration information including request time, request information, and unique identifier. The request information and unique identifier in the MDC tool are assigned to the log frame. Request information and unique identification;
存储日志信息单元,设置为拦截日志框架输出的日志信息,将日志信息拆分成请求时间、请求信息、唯一标识及日志内容,调用应用程序编程接口API将拆分后的日志信息进行存储;The storage log information unit is configured to intercept the log information output by the log framework, split the log information into request time, request information, unique identifier, and log content, and call the application programming interface API to store the split log information;
搜索单元,设置为在服务器前端页面预设搜索界面,搜索界面上设置搜索条件字段,搜索条件字段中含有至少一项请求信息对应的字段信息,当接收到包含有字段信息的搜索请求后,搜索存储的日志信息,并将字段信息对应的日志信息在搜索界面以列表的形式进行展示。The search unit is set to preset a search interface on a server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the request information. When a search request containing field information is received, the search is performed. Store the log information, and display the log information corresponding to the field information in the form of a list on the search interface.
在一个实施例中,如图4所示,拦截客户端请求单元包括:In one embodiment, as shown in FIG. 4, the intercepting client request unit includes:
拦截模块,设置为通过调用预设的spring程序的filter方法拦截客户端请求,获取客户端请求中包括URL地址、请求参数,请求主机IP及客户端对应的项目名称的请求信息;The interception module is configured to intercept the client request by calling the filter method of a preset spring program, and obtain the request information including the URL address, the request parameter, the request host IP and the corresponding project name of the client in the client request;
生成唯一标识模块,设置为调用UUID生成器生成唯一标识;Generate unique identification module, set to call UUID generator to generate unique identification;
存储模块,设置为将请求信息和唯一标识采用MDC.put方法存储至MDC工具中。The storage module is configured to store the request information and the unique identifier into the MDC tool by using the MDC.put method.
在一个实施例中,存储日志信息单元,还设置为日志框架根据配置信息获取MDC工具中的请求信息和唯一标识后,日志框架输出包括请求时间、请求信息及唯一标识的日志信息。In one embodiment, the log information unit is further configured to log information obtained from the MDC tool according to the configuration information and the unique identification, and the log frame outputs log information including the request time, the request information and the unique identification.
在一个实施例中,存储日志信息单元,包括:拦截日志信息模块,设置为通过调用预设的spring程序的aop方法切面拦截日志框架输出的日志信息;拆分日志信息模块,设置为通过调用预设的spring程序的split方法将日志信息拆分成请求时间、请求信息、唯一标识及日志内容。In one embodiment, the log information storage unit includes: an interception log information module configured to intercept the log information output by the log framework by calling the aop method of a preset spring program; a split log information module configured to call The split method of the spring program is used to split the log information into request time, request information, unique identifier, and log content.
在一个实施例中,存储日志信息单元,还包括:设置日志记录表模块,设置为在数据库中设置日志记录表,在日志记录表中设置拆分字段,拆分字段包括请求时间字段、请求信息字段、唯一标识字段及日志内容字段;插入日志信息模块,设置为调用API执行SQL脚本,将拆分后的日志信息依次插入拆分字段中,完成日志信息的存储。In one embodiment, the log information storage unit further includes: a log record table module configured to set a log record table in a database, and a split field in the log record table. The split field includes a request time field and request information. Field, unique identification field, and log content field; insert the log information module, set to call the API to execute the SQL script, and insert the split log information into the split fields in turn to complete the storage of the log information.
在一个实施例中,存储日志信息单元,还包括搜索引擎模块,设置为API采用搜索引擎Elasticsearch,调用Elasticsearch,将拆分后的日志信息存储在Elasticsearch中。In one embodiment, the log information storage unit further includes a search engine module, and the API is set to use the search engine Elasticsearch, call Elasticsearch, and store the split log information in Elasticsearch.
在一个实施例中,提出了一种计算机设备,包括存储器和处理器,存储器中存储有计算机可读指令,计算机可读指令被处理器执行时,使得处理器执行上述各实施例里日志搜索方法中的步骤。In one embodiment, a computer device is provided, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor is caused to execute the log search method in the foregoing embodiments. Steps.
在一个实施例中,提出了一种存储有计算机可读指令的存储介质,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行上述各实施例里日志搜索方法中的步骤。其中,存储介质可以为非易失性存储介质。In one embodiment, a storage medium storing computer-readable instructions is provided. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the log search method in the foregoing embodiments. Steps. The storage medium may be a non-volatile storage medium.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序可以存储于一计算机可读存储介质中,存储介质可以包括:只读存储器(ROM,Read Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁盘或光盘等。A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be implemented by a program instructing related hardware. The program may be stored in a computer-readable storage medium. The storage medium may include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc.
以上所述实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技 术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。The technical features of the embodiments described above can be arbitrarily combined. In order to simplify the description, all possible combinations of the technical features in the above embodiments have not been described. However, as long as there is no contradiction in the combination of these technical features, It should be considered as the scope described in this specification.
以上所述实施例仅表达了本申请一些示例性实施例,其描述较为具体和详细,但并不能因此而理解为对本申请专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only express some exemplary embodiments of the present application, and their descriptions are more specific and detailed, but cannot be understood as a limitation on the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of this application patent shall be subject to the appended claims.

Claims (20)

  1. 一种日志搜索方法,包括:A log search method includes:
    在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
    在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;A log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
    拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;Intercept the log information output by the log framework, split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
    在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。A search interface is preset on the server front-end page, and a search condition field is set on the search interface, and the search condition field contains at least one field information corresponding to the request information. After that, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
  2. 根据权利要求1所述的日志搜索方法,其中,所述在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器,生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中,包括:The log search method according to claim 1, wherein the server intercepts a client request at the server backend, obtains the request information in the client request, and generates by calling a unique identification code UUID preset on the server backend And generating a unique identifier corresponding to the client request, and storing the request information and the unique identifier in a log record MDC tool, including:
    通过调用预设的spring程序的filter方法拦截客户端请求,获取所述客户端请求中包括URL地址、请求参数,请求主机IP及所述客户端对应的项目名称的请求信息;Intercepting a client request by calling a filter method of a preset spring program, obtaining request information including a URL address, request parameters, a request host IP, and a project name corresponding to the client in the client request;
    调用所述UUID生成器生成所述唯一标识;Calling the UUID generator to generate the unique identifier;
    将所述请求信息和所述唯一标识采用MDC.put方法存储至MDC工具中。The request information and the unique identifier are stored in an MDC tool using an MDC.put method.
  3. 根据权利要求1所述的日志搜索方法,其中,所述日志框架输出日志信息,包括:The log search method according to claim 1, wherein the log framework outputs log information, comprising:
    所述日志框架根据所述配置信息获取所述MDC工具中的所述请求信息和唯一标识后,所述日志框架输出包括请求时间、所述请求信息及所述唯一标识的日志信息。After the log framework obtains the request information and the unique identifier in the MDC tool according to the configuration information, the log framework outputs log information including a request time, the request information, and the unique identifier.
  4. 根据权利要求1所述的日志搜索方法,其中,所述拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容,包括:The log search method according to claim 1, wherein the intercepting the log information output by the log framework, and splitting the log information into a request time, the request information, the unique identifier, and a log content comprises: :
    通过调用预设的spring程序的aop方法切面拦截所述日志框架输出的日志信息;Intercept the log information output by the logging framework by calling the aop method of a preset spring program;
    通过调用预设的所述spring程序的split方法将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容。The log information is split into a request time, request information, unique identifier, and log content by calling a preset split method of the spring program.
  5. 根据权利要求1所述的日志搜索方法,其中,所述调用应用程序编程接口API将拆分后的所述日志信息进行存储,包括:The log search method according to claim 1, wherein said calling an application programming interface API to store said split log information comprises:
    在数据库中设置日志记录表,在所述日志记录表中设置拆分字段,所述拆分字段包括请求时间字段、请求信息字段、唯一标识字段及日志内容字段;Setting a log record table in a database, and setting a split field in the log record table, the split field including a request time field, a request information field, a unique identification field, and a log content field;
    调用所述API执行SQL脚本,将拆分后的所述日志信息依次插入所述拆分字段中,完成日志信息的存储。Call the API to execute the SQL script, insert the split log information into the split fields in turn, and complete the storage of the log information.
  6. 根据权利要求1所述的日志搜索方法,其中,所述调用应用程序编程接口API将拆分后的所述日志信息进行存储,包括:The log search method according to claim 1, wherein said calling an application programming interface API to store said split log information comprises:
    所述API采用搜索引擎Elasticsearch,调用所述Elasticsearch,将拆分后的所述日志信息存储在所述Elasticsearch中。The API uses a search engine Elasticsearch, calls the Elasticsearch, and stores the split log information in the Elasticsearch.
  7. 一种日志搜索系统,包括:A log search system includes:
    拦截客户端请求单元,设置为在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request interception unit is configured to intercept the client request at the server backend, obtain the request information in the client request, and generate the client request by calling a unique identification code UUID generator preset on the server backend. The corresponding unique identifier, and storing the request information and the unique identifier in a log record MDC tool;
    预设日志框架单元,设置为在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;The preset log frame unit is configured to preset a log frame on the back end of the server, the log frame contains configuration information including a request time, request information, and unique identifier, and the request information in the MDC tool And a unique identifier assigned to the request information and the unique identifier in the log frame;
    存储日志信息单元,设置为拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容,调用应用程序编程 接口API将拆分后的所述日志信息进行存储;The storage log information unit is configured to intercept the log information output by the log framework, split the log information into a request time, request information, unique identifier and log content, and call an application programming interface API to split the log information. Log information for storage;
    搜索单元,设置为在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。The search unit is configured to preset a search interface on a server front-end page, and a search condition field is set on the search interface, and the search condition field contains at least one piece of field information corresponding to the request information. After the field information search request, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
  8. 根据权利要求7所述的日志搜索系统,其中,所述拦截客户端请求单元包括:The log search system according to claim 7, wherein the intercepting client request unit comprises:
    拦截模块,设置为通过调用预设的spring程序的filter方法拦截客户端请求,获取所述客户端请求中包括URL地址、请求参数,请求主机IP及所述客户端对应的项目名称的请求信息;The intercepting module is configured to intercept a client request by calling a filter method of a preset spring program, and obtain request information including a URL address, a request parameter, a request host IP and a project name corresponding to the client in the client request;
    生成唯一标识模块,设置为调用所述UUID生成器生成所述唯一标识;Generating a unique identification module, configured to call the UUID generator to generate the unique identification;
    存储模块,设置为将所述请求信息和所述唯一标识采用MDC.put方法存储至MDC工具中。The storage module is configured to store the request information and the unique identifier into an MDC tool using an MDC.put method.
  9. 根据权利要求7所述的日志搜索系统,其中,所述存储日志信息单元,还设置为所述日志框架根据所述配置信息获取所述MDC工具中的所述请求信息和唯一标识后,所述日志框架输出包括请求时间、所述请求信息及所述唯一标识的日志信息。The log search system according to claim 7, wherein the storage log information unit is further configured to obtain, after the log framework obtains the request information and the unique identifier in the MDC tool according to the configuration information, the The log frame output includes request time, the request information, and the uniquely identified log information.
  10. 根据权利要求7所述的日志搜索系统,其中,所述存储日志信息单元,包括:The log search system according to claim 7, wherein the stored log information unit comprises:
    拦截日志信息模块,设置为通过调用预设的spring程序的aop方法切面拦截所述日志框架输出的日志信息;The interception log information module is configured to intercept the log information output by the log framework by invoking the preset aop method of the spring program;
    拆分日志信息模块,设置为通过调用预设的所述spring程序的split方法将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容。The split log information module is configured to split the log information into a request time, request information, unique identifier, and log content by calling a preset split method of the spring program.
  11. 根据权利要求7所述的日志搜索系统,其中,所述存储日志信息单元,还包括:The log search system according to claim 7, wherein the stored log information unit further comprises:
    设置日志记录表模块,设置为在数据库中设置日志记录表,在所述日志记录表中设置拆分字段,所述拆分字段包括请求时间字段、请求信息字段、唯一 标识字段及日志内容字段;Setting a logging table module, configured to set a logging table in a database, and setting a split field in the log table, the split field includes a request time field, a request information field, a unique identification field, and a log content field;
    插入日志信息模块,设置为调用所述API执行SQL脚本,将拆分后的所述日志信息依次插入所述拆分字段中,完成日志信息的存储。The log information module is configured to call the API to execute a SQL script, and insert the split log information into the split fields in order to complete the storage of the log information.
  12. 根据权利要求7所述的日志搜索系统,其中,所述存储日志信息单元,还包括搜索引擎模块,设置为所述API采用搜索引擎Elasticsearch,调用所述Elasticsearch,将拆分后的所述日志信息存储在所述Elasticsearch中。The log search system according to claim 7, wherein the storage log information unit further comprises a search engine module configured to use the search engine Elasticsearch for the API, call the Elasticsearch, and split the log information Stored in the Elasticsearch.
  13. 一种计算机设备,包括存储器和处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被所述处理器执行时,使得所述处理器执行以下步骤:A computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor causes the processor to perform the following steps:
    在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
    在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;A log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
    拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;Intercept the log information output by the log framework, split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
    在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。A search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the request information. When a search request is received that includes the field information, After that, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
  14. 根据权利要求13所述的计算机设备,其中,所述在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器,生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中时,使得所述处理器执行以下步骤:The computer device according to claim 13, wherein the server intercepts a client request at the server backend, obtains the request information in the client request, and calls a unique identification code UUID generator preset at the server backend Generating a unique identifier corresponding to the client request, and storing the request information and the unique identifier in a logging MDC tool, so that the processor performs the following steps:
    通过调用预设的spring程序的filter方法拦截客户端请求,获取所述客 户端请求中包括URL地址、请求参数,请求主机IP及所述客户端对应的项目名称的请求信息;Intercepting a client request by calling a filter method of a preset spring program, obtaining request information including a URL address, request parameters in the client request, requesting a host IP and a project name corresponding to the client;
    调用所述UUID生成器生成所述唯一标识;Calling the UUID generator to generate the unique identifier;
    将所述请求信息和所述唯一标识采用MDC.put方法存储至MDC工具中。The request information and the unique identifier are stored in an MDC tool using an MDC.put method.
  15. 根据权利要求13所述的计算机设备,其中,所述日志框架输出日志信息时,使得所述处理器执行以下步骤:The computer device according to claim 13, wherein when the log framework outputs log information, the processor is caused to perform the following steps:
    所述日志框架根据所述配置信息获取所述MDC工具中的所述请求信息和唯一标识后,所述日志框架输出包括请求时间、所述请求信息及所述唯一标识的日志信息。After the log framework obtains the request information and the unique identifier in the MDC tool according to the configuration information, the log framework outputs log information including a request time, the request information, and the unique identifier.
  16. 根据权利要求13所述的计算机设备,其中,所述拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容时,使得所述处理器执行以下步骤:The computer device according to claim 13, wherein when intercepting the log information output by the log framework, splitting the log information into a request time, the request information, the unique identifier, and a log content, such that The processor performs the following steps:
    通过调用预设的spring程序的aop方法切面拦截所述日志框架输出的日志信息;Intercept the log information output by the logging framework by calling the aop method of a preset spring program;
    通过调用预设的所述spring程序的split方法将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容。The log information is split into a request time, request information, unique identifier, and log content by calling a preset split method of the spring program.
  17. 一种存储有计算机可读指令的存储介质,所述计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行以下步骤:A storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:
    在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中;The client request is intercepted at the server backend, the request information in the client request is obtained, and a unique identifier corresponding to the client request is generated by calling a unique identifier UUID generator preset on the server backend, and the The request information and the unique identifier are stored in a log record MDC tool;
    在所述服务器后端预设日志框架,所述日志框架含有包括请求时间、请求信息及唯一标识在内的配置信息,将所述MDC工具中的所述请求信息和唯一标识赋值给所述日志框架中的所述请求信息及唯一标识;A log frame is preset in the server backend, the log frame contains configuration information including request time, request information, and unique identifier, and the request information and unique identifier in the MDC tool are assigned to the log The request information and unique identification in the frame;
    拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容,调用应用程序编程接口API将拆分后的所述日志信息进行存储;Intercept the log information output by the log framework, split the log information into a request time, the request information, the unique identifier and the log content, and call an application programming interface API to perform the split log information storage;
    在服务器前端页面预设搜索界面,所述搜索界面上设置搜索条件字段,所 述搜索条件字段中含有至少一项所述请求信息对应的字段信息,当接收到包含有所述字段信息的搜索请求后,搜索存储的所述日志信息,并将所述字段信息对应的所述日志信息在所述搜索界面以列表的形式进行展示。A search interface is preset on the server front-end page, and a search condition field is set on the search interface. The search condition field contains at least one field information corresponding to the request information. When a search request is received that includes the field information, After that, the stored log information is searched, and the log information corresponding to the field information is displayed in a list form on the search interface.
  18. 根据权利要求17所述的存储介质,其中,所述在服务器后端拦截客户端请求,获取所述客户端请求中的请求信息,通过调用预设在所述服务器后端的唯一识别码UUID生成器,生成所述客户端请求对应的唯一标识,将所述请求信息和所述唯一标识存入日志记录MDC工具中时,使得一个或多个所述处理器执行以下步骤:The storage medium according to claim 17, wherein the server intercepts a client request at the server backend, obtains the request information in the client request, and calls a unique identification code UUID generator preset on the server backend Generating a unique identifier corresponding to the client request, and storing the request information and the unique identifier in a logging MDC tool, so that one or more of the processors perform the following steps:
    通过调用预设的spring程序的filter方法拦截客户端请求,获取所述客户端请求中包括URL地址、请求参数,请求主机IP及所述客户端对应的项目名称的请求信息;Intercepting a client request by calling a filter method of a preset spring program, obtaining request information including a URL address, request parameters, a request host IP, and a project name corresponding to the client in the client request;
    调用所述UUID生成器生成所述唯一标识;Calling the UUID generator to generate the unique identifier;
    将所述请求信息和所述唯一标识采用MDC.put方法存储至MDC工具中。The request information and the unique identifier are stored in an MDC tool using an MDC.put method.
  19. 根据权利要求17所述的存储介质,其中,所述日志框架输出日志信息时,使得一个或多个所述处理器执行以下步骤:The storage medium according to claim 17, wherein when the log framework outputs log information, one or more of the processors are caused to perform the following steps:
    所述日志框架根据所述配置信息获取所述MDC工具中的所述请求信息和唯一标识后,所述日志框架输出包括请求时间、所述请求信息及所述唯一标识的日志信息。After the log framework obtains the request information and the unique identifier in the MDC tool according to the configuration information, the log framework outputs log information including a request time, the request information, and the unique identifier.
  20. 根据权利要求17所述的存储介质,其中,所述拦截所述日志框架输出的日志信息,将所述日志信息拆分成请求时间、所述请求信息、所述唯一标识及日志内容时,使得一个或多个所述处理器执行以下步骤:The storage medium according to claim 17, wherein when the log information output by the log framework is intercepted, the log information is split into a request time, the request information, the unique identifier, and a log content such that One or more of the processors perform the following steps:
    通过调用预设的spring程序的aop方法切面拦截所述日志框架输出的日志信息;Intercept the log information output by the logging framework by calling the aop method of a preset spring program;
    通过调用预设的所述spring程序的split方法将所述日志信息拆分成请求时间、请求信息、唯一标识及日志内容。The log information is split into a request time, request information, unique identifier, and log content by calling a preset split method of the spring program.
PCT/CN2018/106687 2018-07-11 2018-09-20 Log search method and system, computer device, and storage medium WO2020010702A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810756846.6 2018-07-11
CN201810756846.6A CN109033280A (en) 2018-07-11 2018-07-11 Blog search method, system, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
WO2020010702A1 true WO2020010702A1 (en) 2020-01-16

Family

ID=64641751

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/106687 WO2020010702A1 (en) 2018-07-11 2018-09-20 Log search method and system, computer device, and storage medium

Country Status (2)

Country Link
CN (1) CN109033280A (en)
WO (1) WO2020010702A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113438227A (en) * 2021-06-23 2021-09-24 平安普惠企业管理有限公司 Method, device and equipment for limiting current of repeated requests and storage medium

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110134538B (en) * 2019-05-10 2020-08-04 重庆天蓬网络有限公司 Method, device, medium and electronic equipment for quickly positioning problem log
CN110209644A (en) * 2019-05-21 2019-09-06 上海易点时空网络有限公司 The method, apparatus and system of log management
CN110765089A (en) * 2019-10-25 2020-02-07 北京浪潮数据技术有限公司 Distributed equipment and log recording method, device and storage medium thereof
CN112783842B (en) * 2019-11-07 2024-03-01 北京沃东天骏信息技术有限公司 Log collection method and device
CN111092883B (en) * 2019-12-16 2022-08-19 重庆锐云科技有限公司 Method and device for automatically generating user access record, network equipment and storage medium
CN111309497B (en) * 2020-02-10 2023-09-26 北京有竹居网络技术有限公司 Information calling method and device, server, terminal and storage medium
CN111459756A (en) * 2020-03-30 2020-07-28 深圳市金蝶天燕云计算股份有限公司 Log processing method and related equipment
CN112148279B (en) * 2020-09-27 2024-06-07 阿波罗智联(北京)科技有限公司 Log information processing method, device, electronic equipment and storage medium
CN112256530A (en) * 2020-10-27 2021-01-22 广州助蜂网络科技有限公司 Log service method capable of tracking link and system thereof
CN113434464A (en) * 2021-06-24 2021-09-24 江苏创源电子有限公司 Distributed log processing system and method
CN116662218A (en) * 2023-08-01 2023-08-29 北京德塔精要信息技术有限公司 Method and device for collecting and processing logs in real time

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050234952A1 (en) * 2004-04-15 2005-10-20 Microsoft Corporation Content propagation for enhanced document retrieval
CN101808121A (en) * 2010-02-24 2010-08-18 深圳市五巨科技有限公司 Method and device for writing server log of mobile terminal into database
CN107818024A (en) * 2017-11-22 2018-03-20 北京汇通金财信息科技有限公司 A kind of request ID transmission methods and system based on spring blockers
CN107861859A (en) * 2017-11-22 2018-03-30 北京汇通金财信息科技有限公司 A kind of blog management method and system based on micro services framework

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050234952A1 (en) * 2004-04-15 2005-10-20 Microsoft Corporation Content propagation for enhanced document retrieval
CN101808121A (en) * 2010-02-24 2010-08-18 深圳市五巨科技有限公司 Method and device for writing server log of mobile terminal into database
CN107818024A (en) * 2017-11-22 2018-03-20 北京汇通金财信息科技有限公司 A kind of request ID transmission methods and system based on spring blockers
CN107861859A (en) * 2017-11-22 2018-03-30 北京汇通金财信息科技有限公司 A kind of blog management method and system based on micro services framework

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113438227A (en) * 2021-06-23 2021-09-24 平安普惠企业管理有限公司 Method, device and equipment for limiting current of repeated requests and storage medium
CN113438227B (en) * 2021-06-23 2024-02-27 平安普惠企业管理有限公司 Method, device, equipment and storage medium for limiting repeated request

Also Published As

Publication number Publication date
CN109033280A (en) 2018-12-18

Similar Documents

Publication Publication Date Title
WO2020010702A1 (en) Log search method and system, computer device, and storage medium
US10585683B2 (en) Defining application programming interfaces (APIs) using object schemas
US20200183932A1 (en) Optimizing write operations in object schema-based application programming interfaces (apis)
US11314698B2 (en) Dynamically performing data processing in a data pipeline system
CA3025493C (en) Optimizing read and write operations in object schema-based application programming interfaces (apis)
US8452925B2 (en) System, method and computer program product for automatically updating content in a cache
US10866828B2 (en) Extending object-schema-based application programming interfaces (APIs)
US10824544B1 (en) Generating test data as a service for use in testing software during software development
US11176044B2 (en) Systems and methods for implementing overlapping data caching for object application program interfaces
US20210232603A1 (en) Capturing data lake changes
CN111966692A (en) Data processing method, medium, device and computing equipment for data warehouse
CN111143383B (en) Data updating method and device, electronic equipment and storage medium
Sachdeva Practical ELK Stack
US8880586B2 (en) Metadata subscription registry
US11770413B2 (en) Domain-independent resource security and management
Stelly et al. Language-based integration of digital forensics & incident response
US20240152521A1 (en) Database system observability data querying and access
US20230229800A1 (en) Content variation to track documents

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18926346

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 21/04/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18926346

Country of ref document: EP

Kind code of ref document: A1