WO2020252724A1 - Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur - Google Patents

Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur Download PDF

Info

Publication number
WO2020252724A1
WO2020252724A1 PCT/CN2019/091998 CN2019091998W WO2020252724A1 WO 2020252724 A1 WO2020252724 A1 WO 2020252724A1 CN 2019091998 W CN2019091998 W CN 2019091998W WO 2020252724 A1 WO2020252724 A1 WO 2020252724A1
Authority
WO
WIPO (PCT)
Prior art keywords
client
shared memory
server
log
memory
Prior art date
Application number
PCT/CN2019/091998
Other languages
English (en)
Chinese (zh)
Inventor
廖凯
邱桂添
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980012096.4A priority Critical patent/CN111699479A/zh
Priority to PCT/CN2019/091998 priority patent/WO2020252724A1/fr
Publication of WO2020252724A1 publication Critical patent/WO2020252724A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815—Journaling file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/544—Buffers; Shared memory; Pipes

Definitions

  • the embodiments of the present invention relate to the field of communications, and in particular to a log processing method, device and computer-readable storage medium.
  • Network equipment, systems and service programs, etc. will generate log logs during operation, and each log records the description of related operations such as date, time, user, and action.
  • large software projects often generate a large number of logs. Because of the large number of logs, how to quickly communicate and store logs has become an urgent problem to be solved.
  • the storage of logs is generally implemented through a Syslog log system.
  • the Syslog log system supports local and remote log storage services by providing calling interfaces, configuration files, and daemon processes.
  • the remote log storage service can obtain logs by monitoring fixed ports.
  • the embodiments of the present invention provide a log processing method, device, and computer-readable storage medium to solve the technical problem that the existing log processing method is not suitable for high-concurrency scenarios and cannot manage logs of multiple processes.
  • the first aspect of the embodiments of the present invention is to provide a log processing method, including:
  • the client sends a registration request to the server through the preset calling interface
  • the server receives the registration request sent by the client
  • the server applies for shared memory for the client according to the registration request
  • the server After the shared memory application is successful, the server sends registration success information to the client;
  • the server obtains the log information from the shared memory and stores it in a preset storage path.
  • the second aspect of the embodiments of the present invention is to provide a log processing method applied to a client, and the method includes:
  • the log information of the client is written into the shared memory of the client and the server.
  • the third aspect of the embodiments of the present invention is to provide a log processing method applied to the server, and the method includes:
  • the fourth aspect of the embodiments of the present invention is to provide a log processing system, including
  • One or more processors are One or more processors;
  • One or more memories are One or more memories
  • the memory is used to store program codes
  • the processor is used to call the program code, and when the program code is executed, it is used to perform the following operations:
  • the log information is obtained from the shared memory through the server and stored in a preset storage path.
  • the fifth aspect of the embodiments of the present invention is to provide a client terminal device, and the terminal device includes:
  • the memory is used to store program codes
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • the log information is written into the shared memory of the client and the server.
  • a sixth aspect of the embodiments of the present invention is to provide a service terminal device, where the service terminal device includes: a memory and a processor;
  • the memory is used to store program codes
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • a seventh aspect of the embodiments of the present invention is to provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the first aspect or the second aspect or the third aspect .
  • the log processing method, device, and computer-readable storage medium provided in this embodiment send a registration request to the server through a preset calling interface, so that the server applies for shared memory for the client according to the registration request, and applies for shared memory After success, the registration success information is fed back to the client, where the client and the server share the shared memory. After receiving the registration success information fed back by the server, the log information generated by the client is written into the shared memory so that the server can obtain the log information written by the client from the shared memory and store it in the preset In the storage path. Since the server can apply for shared memory according to at least one thread of at least one client, it can be applied to high-concurrency scenarios, and can effectively manage the logs of multiple processes.
  • FIG. 1 is a schematic flowchart of a log processing method provided by Embodiment 1 of the present invention
  • Embodiment 2 is a schematic flowchart of a log processing method provided by Embodiment 2 of the present invention.
  • Embodiment 3 is a schematic flowchart of a log processing method provided by Embodiment 3 of the present invention.
  • Embodiment 4 is a schematic flowchart of a log processing method provided by Embodiment 4 of the present invention.
  • FIG. 5 is a schematic flowchart of a log processing method according to Embodiment 5 of the present invention.
  • FIG. 6 is a schematic flowchart of a log processing method provided by Embodiment 6 of the present invention.
  • FIG. 7 is a schematic flowchart of a log processing method provided by Embodiment 7 of the present invention.
  • FIG. 8 is a schematic flowchart of a log processing method provided by Embodiment 8 of the present invention.
  • Embodiment 9 is a schematic flowchart of a log processing method provided by Embodiment 9 of the present invention.
  • FIG. 10 is a schematic structural diagram of a client terminal device according to Embodiment 10 of the present invention.
  • FIG. 11 is a schematic structural diagram of a service terminal device provided by Embodiment 11 of the present invention.
  • a component when a component is said to be “fixed to” another component, it can be directly on the other component or a central component may also exist. When a component is considered to be “connected” to another component, it can be directly connected to another component or there may be a centered component at the same time.
  • the present invention provides a log processing method, device, and computer-readable storage medium.
  • the present invention provides The log processing method, device, and computer-readable storage medium can be applied to any log processing scenario.
  • FIG. 1 is a schematic flowchart of a log processing method provided in Embodiment 1 of the present invention. As shown in FIG. 1, the log processing method includes:
  • Step 101 The client sends a registration request to the server through a preset calling interface
  • Step 102 The server receives the registration request sent by the client.
  • Step 103 The server applies for shared memory for the client according to the registration request;
  • Step 104 After the shared memory application is successful, the server sends registration success information to the client;
  • Step 105 When receiving the registration success information fed back by the server, the client writes the log information of the client into the shared memory;
  • Step 106 The server obtains the log information from the shared memory and stores it in a preset storage path.
  • the execution subject of this embodiment is a log processing system, where the log processing system includes a client and a server.
  • the client can send a registration request to the server through a preset calling interface.
  • the server can determine whether to apply for shared memory for the client according to the registration request. For example, if the server’s current storage space is insufficient and its processing capacity is weak, it is not necessary to apply for shared memory for the client according to the registration request; if the server’s current storage space is large and its processing capacity is strong, it can be The registration request applies for shared memory for the client.
  • the client and the server share the shared memory.
  • the client can write the generated log information into the shared memory, and the server can take out the log information from the shared memory. Further, after the server determines that it can apply for shared memory for the client according to the registration request, it can feed back registration success information to the client, so that the client can understand the status of the current registration request in time.
  • the client receives the registration success message sent by the server, it can write the log information that needs to be stored into the shared memory.
  • the registration success information may include the storage address of the shared memory, and the registration success information may be sent by the server to the client through a preset or newly added calling interface. Therefore, the server can obtain the log information stored by the client from the shared memory and store it in a preset storage path.
  • the shared memory is a locked queue instance.
  • the registration request is sent by one of the threads of the client.
  • the registration request may specifically be sent by one of the threads in the client. Since there are multiple threads in the client, each thread can send a registration request to the server. Accordingly, the server can apply for shared memory for the client where the thread is located according to the registration request sent by each thread. By separately applying for shared memory for multiple threads of the client, it is possible to realize log storage under high concurrency.
  • the client sends a registration request to the server through a preset calling interface; the server receives the registration request sent by the client; the server applies for shared memory for the client according to the registration request; when the shared memory applies After success, the server sends the registration success information to the client; when the client receives the registration success information fed back by the server, it writes the log information of the client to the shared memory; the server obtains the log information from the shared memory and stores it To the preset storage path. Since the server can apply for shared memory for the client corresponding to the thread according to at least one thread of at least one client, it can be applied to high-concurrency scenarios, and can effectively manage the logs of multiple processes.
  • the shared memory is a lock-free queue instance.
  • the shared memory may specifically be a lock-free queue instance.
  • the lock-free queue instance may be a lock-free queue instance implemented by a compare and replace (Compare and Set, CAS for short) mechanism, or any other one may be used.
  • An example of a lock-free queue implemented by this mechanism is not limited in the present invention. Using a lock-free queue instance can save lock overhead on the basis of improving log processing efficiency.
  • the log processing method provided in this embodiment uses a lock-free queue instance as a shared memory, thereby improving the log processing effect, and further enabling log processing in a multi-process high-concurrency scenario.
  • the writing the log information of the client into the shared memory includes:
  • the log of each thread in the client is written into the shared memory according to a predetermined thread storage sequence.
  • a thread storage order in order to improve the efficiency of log processing, can be preset.
  • the thread storage order can be determined by one or more of the priority, urgency, data size, and sending time of each thread.
  • the user can also make real-time adjustments according to current needs, and the present invention is not limited here.
  • the log processing method provided in this embodiment can effectively improve the efficiency of log processing by writing the log of each thread in the client into the shared memory in a predetermined thread storage sequence.
  • the writing the log information of the client into the shared memory includes:
  • the priority of the log information can be determined first, and the log information with a priority higher than the preset first printing level can be written into the log information.
  • the first printing level may be set by the user according to actual needs, or may be a printing level preset by the system, which is not limited in the present invention.
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • log information with a priority higher than the preset first print level is stored in the shared memory, so that the quality of log information can be optimized and the storage space can be reduced. Log processing efficiency.
  • the writing the log information of the client to the shared memory includes:
  • the client writes the log information of the client into the shared memory in an asynchronous storage manner.
  • the existing log processing method since the existing log processing method generally waits for the server to feedback that the log information is stored successfully after the client writes the log information on the server, and performs subsequent log write operations, it is often more Time-consuming, resulting in low log processing efficiency.
  • the log information of the client can be written to the shared memory in the manner of asynchronous storage. Specifically, after the client writes the log information in the shared memory, the subsequent log writing operation can be performed without waiting for the storage success information fed back by the server.
  • the log processing method provided in this embodiment can effectively increase the log writing speed by writing the log information of the client into the shared memory in an asynchronous storage manner, thereby improving log processing efficiency.
  • the storage of log information through asynchronous storage can effectively solve the problem of IO stuck in the prior art using synchronous Zlog storage, which is suitable for real-time requirements. Higher system.
  • the method before sending a registration request to the server through the calling interface preset by the client, the method further includes:
  • the client generates the registration request according to a preset log writing configuration.
  • the log writing configuration may be determined first.
  • the log writing configuration may specifically be input by the user according to current requirements, or may be a default writing configuration, which is not limited in the present invention.
  • Generate a registration request according to the log writing configuration and send the registration request to the server, so that the server can apply for shared memory for the client according to the log writing configuration.
  • the log writing configuration includes at least one of memory size to be occupied, storage mode, storage location, storage subfolder division, and storage subfolder size.
  • different priorities can correspond to different log writing configurations.
  • the log writing configuration with a lower priority can only include the memory size to be occupied, while the log writing of the client with a higher priority
  • the configuration can also include other personalized demand parameters.
  • the log processing method provided in this embodiment generates the registration request according to a preset log writing configuration, so that on the basis of improving the log processing efficiency, the client's personalized requirements in the log writing process can be met.
  • Figure 2 is a schematic flow chart of the log processing method provided by the second embodiment of the present invention.
  • the registration success information fed back by the server is received, the After the log information of the client is written into the shared memory, it further includes:
  • Step 201 The client obtains the number of connections of the shared memory
  • Step 202 The client determines whether the server is disconnected from the shared memory according to the number of connections.
  • the client and the server need to be connected to the shared memory at the same time. Therefore, in order to ensure that the server is not dropped during log processing, the number of shared memory connections can be obtained. It should be noted that when no server is disconnected, the total number of shared memory connections is the total number of servers and clients currently connected to the shared memory. Therefore, the current status of the server can be determined according to the number of connections. After determining the number of connections, you can determine whether the server is disconnected from the shared memory according to the number of connections.
  • the log processing method by acquiring the number of connections of the shared memory, it is determined whether the server is disconnected from the shared memory according to the number of connections. Thereby, the effectiveness of log writing can be ensured, and the efficiency of log processing can be further improved.
  • the determining whether the server is disconnected from the shared memory according to the number of connections includes:
  • the client remains connected to the shared memory, and the number of connections is less than the preset total number of connections, it is determined that the server is disconnected from the shared memory; wherein, the total number of connections is The number of connections to the shared memory in the normal log storage state;
  • the client if the client is currently connected to the shared memory and the number of connections is less than the preset total number of connections, it can be determined that the server is disconnected from the shared memory, where the total number of connections is in the normal log storage state.
  • the number of shared memory connections if the client is currently connected to the shared memory and the number of connections is not less than the preset total number of connections, it can be determined that the server is not currently disconnected. Taking a practical application as an example, if there is currently a client and a server connected to the shared memory, then the number of shared memory connections is 2. If the client detects that the current number of connections in the shared memory is 2, it can be determined that the current server is running normally and not disconnected. Correspondingly, if the client detects that the current number of connections in the shared memory is 1, it can be determined that the server has dropped In order to ensure the normal processing of log information, the server needs to be reminded.
  • the log processing method provided in this embodiment determines that the server is disconnected from the shared memory if the client remains connected to the shared memory and the number of connections is less than the preset total number of connections Connection; the total number of connections is the number of connections to the shared memory in the normal log storage state, if the client is connected to the shared memory, and the number of connections is not less than the preset total number of connections, it is determined
  • the server is not disconnected from the shared memory.
  • FIG. 3 is a schematic flowchart of a log processing method provided in Embodiment 3 of the present invention. Based on any of the above embodiments, as shown in FIG. 3, the method further includes:
  • Step 301 When it is determined that the server is disconnected, send a reconnection request to the server according to a preset frequency
  • Step 302 When the server receives the reconnection request sent by the client, the server reconnects to the shared memory according to the reconnection request.
  • a reconnection request can be sent to the server at a preset frequency, so that the server can re-establish a communication connection with the shared memory according to the reconnection request.
  • the preset frequency can be a preset frequency, or it can be set according to actual needs.
  • the client can send a reconnection request at a higher frequency; if the client The occupied shared memory is relatively large, and the reconnection request can also be sent at a higher frequency, which is not limited in the present invention.
  • the server by sending a reconnection request to the server at a preset frequency when it is determined that the server is disconnected, the server can connect to the shared memory according to the reconnection request , Which can ensure the effectiveness of log writing, and further improve the efficiency of log processing.
  • the registration success information includes the identification code allocated by the server to the client;
  • the reconnection request includes the identification code;
  • the reconnection request to reconnect with the shared memory includes:
  • the server connects to the shared memory according to the identification code in the reconnection request.
  • the server can send a registration success message to the client, where the registration success message includes the identification code assigned by the server to the client.
  • the client detects that the server is disconnected from the shared memory, it can send a reconnection request including the identification code to the server. Therefore, the server can reconnect to the shared memory according to the identification code.
  • the identification code in the received registration success message is placed in the reconnection request and sent to the server, so that the server can reconnect to the shared memory according to the identification code, and then be able to On the basis of ensuring the validity of log writing, the efficiency of log processing is further improved.
  • the server obtains the log information from the shared memory and stores it in a preset storage path, including:
  • the server obtains log information with a priority higher than a preset second printing level from the shared memory, and stores it in a preset storage path.
  • the server since the number of log information written by the client is large, different log information has different priorities. Therefore, in order to optimize the quality of log information and reduce storage space, when the server obtains the log information from the shared memory, it first needs to determine the priority of the log information and set the priority higher than the preset second printing level of the log information , Save to the preset storage path.
  • the second printing level can be set by the user according to actual needs, or can be a printing level preset by the system, which is not limited in the present invention.
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • the log processing method provided in this embodiment obtains log information with a priority higher than the preset second printing level from the shared memory and stores it in a preset storage path, thereby optimizing the quality of log information and reducing Improve log processing efficiency based on small storage space.
  • the server obtains the log information from the shared memory and stores it in a preset storage path, including:
  • the server obtains the log information from the shared memory in a single-threaded manner and stores it in a preset storage path.
  • the server in order to improve log processing efficiency, after the client writes the log information into the shared memory, the server can obtain the log information from the shared memory in a single process and store it in a preset storage path.
  • the log processing method provided in this embodiment can effectively increase the log writing speed by acquiring the log information from the shared memory in a single-threaded manner and storing it in a preset storage path, thereby improving log processing. effectiveness.
  • the registration request includes the amount of memory to be occupied
  • the application for shared memory for the client according to the registration request includes:
  • the server applies for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied.
  • the registration request may include the amount of memory to be occupied.
  • the client when it sends a registration request to the server, it can determine the amount of memory to be occupied, and generate a registration request according to the amount of memory to be occupied.
  • the server After receiving the registration request, the server can first determine the current remaining amount of memory, and apply for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied.
  • the applying for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied includes:
  • the client can apply for shared memory corresponding to the amount of memory to be occupied; otherwise, if the amount of memory to be occupied is not less than the remaining amount of memory, it means The current remaining memory of the server is insufficient to apply for shared memory for the client. At this time, it can refuse to apply for shared memory for the client.
  • the log processing method provided in this embodiment determines the current remaining amount of memory on the server, and applies for shared memory for the client based on the remaining amount of memory and the amount of memory to be occupied, so that it can quickly apply for the client Shared memory improves log processing efficiency.
  • the registration request further includes the process identifier of the client
  • the applying for shared memory for the client according to the registration request includes:
  • the registration request also includes the process identification.
  • the server can generate a blacklist based on historical experience.
  • the blacklist Include the process ID. Therefore, after the server receives the registration request that includes the process ID, it can detect whether the process ID is included in the blacklist. If so, it can refuse to apply for shared memory for the client. Otherwise, it can be the client according to the registration request. Apply for shared memory.
  • the log information generated by some clients may be more important or some clients have higher priority. Therefore, the server may also generate a whitelist based on historical experience, where the whitelist includes the process identifier.
  • the server After the server receives the registration request including the process ID, it can detect whether the process ID is included in the whitelist. If so, it can provide the client corresponding to the process ID with larger shared memory, or can provide the process Identifies the corresponding client to store log information with lower priority.
  • the server can provide different services to the clients according to different priorities. Therefore, after receiving the registration request including the process identifier, the server can first determine the priority corresponding to the process identifier, and provide the client with a service corresponding to the priority according to the priority.
  • the log processing method provided in this embodiment judges whether the process identifier is included in the preset process blacklist, if it does not exist, apply for shared memory for the process, and if it exists, refuse to apply for shared memory for the process , So that the server can selectively apply for shared memory for the client, and then can ensure the security of the server on the basis of ensuring the efficiency of log processing.
  • the registration request is generated according to a preset log writing configuration
  • the applying for shared memory for the client according to the registration request includes:
  • the server applies for shared memory for the client according to the log writing configuration.
  • the client may first determine the log writing configuration, and generate a registration request according to the log writing configuration.
  • the log writing configuration may specifically be input by the user according to current requirements, or may be a default writing configuration, which is not limited in the present invention.
  • the server can apply for shared memory for the client according to the log writing configuration.
  • the log writing configuration includes at least one of memory size to be occupied, storage mode, storage location, storage subfolder division, and storage subfolder size.
  • the stored subfolders are used to divide the stored subfolders according to different functions of threads, and the stored subfolders can also be divided according to different types of logs.
  • different priorities can correspond to different log writing configurations. For example, the log writing configuration with a lower priority can only include the memory size to be occupied, while the log writing of the client with a higher priority
  • the configuration can also include other personalized demand parameters.
  • the method further includes:
  • the server performs an initialization operation on the shared memory to obtain a lock-free queue instance.
  • the shared memory may be initialized to obtain a lock-free queue instance.
  • the lock-free queue instance can be a lock-free queue instance implemented by a Compare and Replace (Compare and Set, CAS for short) mechanism, or a lock-free queue instance implemented by any other mechanism, and the present invention is not limited here. .
  • Using a lock-free queue instance can save lock overhead on the basis of improving log processing efficiency.
  • the log processing method provided in this embodiment obtains a lock-free queue instance by initializing the shared memory, thereby improving the log processing effect, and further enabling log processing in a multi-process high concurrency scenario.
  • the method further includes:
  • the method further includes:
  • the server monitors the log writing status in the shared memory, and manages the client according to the monitoring result.
  • the client can write log information in the shared memory after receiving the registration success information.
  • the server can monitor the log writing status in the shared memory, and control the client according to the monitoring results. For example, you can determine whether the client is a malicious client based on the log writing status, and whether the client is currently malfunctioning.
  • the log processing method provided in this embodiment monitors the log writing status in the shared memory, and manages and controls the client according to the monitoring result, so that the log processing efficiency can be improved on the basis of improving the log processing efficiency. Effective management and control.
  • the server monitors the log writing status in the shared memory, and manages the client according to the monitoring result, including:
  • the server determines whether the speed at which the client writes the log exceeds a preset speed threshold
  • the current speed at which the client writes the log can be determined, and the status of the client can be determined according to the speed. Specifically, if it is detected that the speed at which the client writes the log exceeds a preset speed threshold, it can be determined that the client is currently malfunctioning, anyway, it means that the current client is operating normally.
  • the monitoring of the log writing status in the shared memory and the management and control of the client according to the monitoring result includes:
  • the number of logs currently written by the client can be determined, and the status of the client can be determined according to the number. Specifically, if it is detected that the number of logs written by the client exceeds a preset number threshold, it can be determined that the client is currently malfunctioning, otherwise, it means that the current client is operating normally.
  • the above two embodiments can be implemented separately or in combination.
  • it can be determined whether the number of logs currently written by the client exceeds the preset number threshold and whether the speed of the client currently writing logs exceeds the preset speed according to the number and speed of logs currently written by the client The threshold, if yes, it is determined that the client is currently malfunctioning, anyway, it means that the current client is operating normally.
  • the log processing method provided in this embodiment determines whether the client is currently malfunctioning by determining whether the number and/or speed of logs currently written by the client exceed the preset number and/or speed threshold, which can effectively achieve the End monitoring.
  • the method further includes:
  • the server manages and controls the log information written by the client according to the authority information.
  • the permission information corresponding to the client can be determined according to the process identifier of the client, and the log information written by the client can be controlled according to the permission information.
  • the All log information is stored; accordingly, if the priority of the client currently writing to the log is lower, when it is detected that the number and/or speed of the current written log of the client exceeds the preset number and/or speed threshold, Part of the log information written by the client can be stored, or the client can be refused to continue to provide shared memory.
  • the authority information corresponding to the client is determined according to the process identifier, and the log information written by the client is controlled according to the authority information, so as to be effective for the client. Control.
  • the method includes:
  • the server obtains the log information from the shared memory, and publishes the log information.
  • the client writes the log information into the shared memory
  • the server can obtain the log information from the shared memory and perform publishing processing.
  • FIG. 4 is a schematic flow chart of a log processing method provided by Embodiment 4 of the present invention, which is applied to a client terminal. As shown in FIG. 4, the method includes:
  • Step 401 Send a registration request to the server through a preset calling interface.
  • the execution subject of this embodiment is a client, where the client may be an application, a process, or a terminal device.
  • the client can send a registration request to the server through a preset calling interface.
  • the server can determine whether to apply for shared memory for the client according to the registration request. For example, if the server’s current storage space is insufficient and its processing capacity is weak, it is not necessary to apply for shared memory for the client according to the registration request; if the server’s current storage space is large and its processing capacity is strong, it can be The registration request applies for shared memory for the client.
  • the client and the server share the shared memory.
  • the client can write the generated log information into the shared memory, and the server can take out the log information from the shared memory. Further, after the server determines that it can apply for shared memory for the client according to the registration request, it can feed back registration success information to the client, so that the client can understand the status of the current registration request in time.
  • the registration request is sent by one of the threads of the client.
  • the registration request may specifically be sent by one of the threads in the client. Since there are multiple threads in the client, each thread can send a registration request to the server. Accordingly, the server can apply for shared memory for the client where the thread is located according to the registration request sent by each thread. By separately applying for shared memory for multiple threads of the client, it is possible to realize log storage under high concurrency.
  • Step 402 When the registration success information fed back by the server is received, the log information of the client is written into the shared memory.
  • the client when the client receives the registration success information sent by the server, it can write the log information that needs to be stored into the shared memory.
  • the registration success information may include the storage address of the shared memory, and the registration success information may be sent by the server to the client through a preset or newly added calling interface. Therefore, the server can obtain the log information stored by the client from the shared memory and store it in a preset storage path.
  • the log processing method provided in this embodiment sends a registration request to the server through a preset calling interface, so that the server applies for shared memory for the client according to the registration request, and reports the successful registration to the client after the shared memory application is successful Information, where the client and server share the shared memory.
  • the log information generated by the client is written into the shared memory so that the server can obtain the log information written by the client from the shared memory and store it in the preset In the storage path. Since the server can apply for shared memory according to at least one thread of at least one client, it can be applied to high-concurrency scenarios, and can effectively manage the logs of multiple processes.
  • the shared memory is a lock-free queue instance.
  • the shared memory may specifically be a lock-free queue instance.
  • the lock-free queue instance may be a lock-free queue instance implemented by a compare and replace (Compare and Set, CAS for short) mechanism, or any other one may be used.
  • An example of a lock-free queue implemented by this mechanism is not limited in the present invention. Using a lock-free queue instance can save lock overhead on the basis of improving log processing efficiency.
  • the log processing method provided in this embodiment uses a lock-free queue instance as a shared memory, thereby improving the log processing effect, and further enabling log processing in a multi-process high-concurrency scenario.
  • the shared memory is a locked queue instance.
  • the writing the log information of the client into the shared memory includes:
  • the log of each thread in the client is written into the shared memory according to a predetermined thread storage sequence.
  • a thread storage order in order to improve the efficiency of log processing, can be preset.
  • the thread storage order can be determined by one or more of the priority, urgency, data size, and sending time of each thread.
  • the user can also make real-time adjustments according to current needs, and the present invention is not limited here.
  • the log processing method provided in this embodiment can effectively improve the efficiency of log processing by writing the log of each thread in the client into the shared memory in a predetermined thread storage sequence.
  • the writing the log information of the client into the shared memory includes:
  • the priority of the log information can be determined first, and the log information with a priority higher than the preset first printing level can be written into the log information.
  • the first printing level may be set by the user according to actual needs, or may be a printing level preset by the system, which is not limited in the present invention.
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • log information with a priority higher than the preset first print level is stored in the shared memory, so that the quality of log information can be optimized and the storage space can be reduced. Log processing efficiency.
  • the writing the log information of the client to the shared memory includes:
  • the log information of the client is written into the shared memory in an asynchronous storage manner.
  • the existing log processing method since the existing log processing method generally waits for the server to feedback that the log information is stored successfully after the client writes the log information on the server, and performs subsequent log write operations, it is often more Time-consuming, resulting in low log processing efficiency.
  • the log information of the client can be written to the shared memory in the manner of asynchronous storage. Specifically, after the client writes the log information in the shared memory, the subsequent log writing operation can be performed without waiting for the storage success information fed back by the server.
  • the log processing method provided in this embodiment can effectively increase the log writing speed by writing the log information of the client into the shared memory in an asynchronous storage manner, thereby improving log processing efficiency.
  • the storage of log information through asynchronous storage can effectively solve the problem of IO stuck in the prior art using synchronous Zlog storage, which is suitable for real-time requirements. Higher system.
  • the registration information also includes the process identifier of the process that sent the registration request, so that the server can determine whether to provide the process for the process according to the process identifier.
  • Memory information
  • the registration information also includes the process identifier of the process that sent the registration request.
  • the server can generate a blacklist based on historical experience, where the blacklist includes the process identifier. Therefore, after the server receives the registration request that includes the process ID, it can detect whether the process ID is included in the blacklist. If so, it can refuse to apply for shared memory for the client. Otherwise, it can be the client according to the registration request. Apply for shared memory.
  • the log information generated by some clients may be more important or some clients have a higher priority. Therefore, the server may also generate a whitelist based on historical experience, where the whitelist includes the process identifier.
  • the server After the server receives the registration request including the process ID, it can detect whether the process ID is included in the whitelist. If so, it can provide the client corresponding to the process ID with larger shared memory, or can provide the process Identifies the corresponding client to store log information with lower priority.
  • the server can provide different services to the clients according to different priorities. Therefore, after receiving the registration request including the process identifier, the server can first determine the priority corresponding to the process identifier, and provide the client with a service corresponding to the priority according to the priority.
  • the registration information also includes the process identification of the process that sent the registration request, so that the server can selectively apply for shared memory for the client, thereby ensuring log processing efficiency. , To ensure the security of the server.
  • the method before sending a registration request to the server through the calling interface preset by the client, the method further includes:
  • the registration request is generated according to the preset log writing configuration.
  • the log writing configuration may be determined first.
  • the log writing configuration may specifically be input by the user according to current requirements, or may be a default writing configuration, which is not limited in the present invention.
  • Generate a registration request according to the log writing configuration and send the registration request to the server, so that the server can apply for shared memory for the client according to the log writing configuration.
  • the log writing configuration includes at least one of memory size to be occupied, storage mode, storage location, storage subfolder division, and storage subfolder size.
  • different priorities can correspond to different log writing configurations.
  • the log writing configuration with a lower priority can only include the memory size to be occupied, while the log writing of the client with a higher priority
  • the configuration can also include other personalized demand parameters.
  • the log processing method provided in this embodiment generates the registration request according to a preset log writing configuration, so that on the basis of improving the log processing efficiency, the client's personalized requirements in the log writing process can be met.
  • FIG. 5 is a schematic flowchart of a log processing method provided by Embodiment 5 of the present invention.
  • the registration success information fed back by the server is received, the After the log information of the client is written into the shared memory, it further includes:
  • Step 501 Obtain the number of connections of the shared memory.
  • Step 502 Determine whether the server is disconnected from the shared memory according to the number of connections.
  • the client and the server need to be connected to the shared memory at the same time. Therefore, in order to ensure that the server is not dropped during log processing, the number of shared memory connections can be obtained. It should be noted that when no server is disconnected, the total number of shared memory connections is the total number of servers and clients currently connected to the shared memory. Therefore, the current status of the server can be determined according to the number of connections. After determining the number of connections, you can determine whether the server is disconnected from the shared memory according to the number of connections.
  • the log processing method by acquiring the number of connections of the shared memory, it is determined whether the server is disconnected from the shared memory according to the number of connections. Thereby, the effectiveness of log writing can be ensured, and the efficiency of log processing can be further improved.
  • the determining whether the server is disconnected from the shared memory according to the number of connections includes:
  • the client remains connected to the shared memory, and the number of connections is less than the preset total number of connections, it is determined that the server is disconnected from the shared memory; the total number of connections is a normal log The number of connections of the shared memory in the storage state;
  • the client if the client is currently connected to the shared memory and the number of connections is less than the preset total number of connections, it can be determined that the server is disconnected from the shared memory, where the total number of connections is in the normal log storage state.
  • the number of shared memory connections if the client is currently connected to the shared memory and the number of connections is not less than the preset total number of connections, it can be determined that the server is not currently disconnected. Taking a practical application as an example, if there is currently a client and a server connected to the shared memory, then the number of shared memory connections is 2. If the client detects that the current number of connections in the shared memory is 2, it can be determined that the current server is running normally and not disconnected. Correspondingly, if the client detects that the current number of connections in the shared memory is 1, it can be determined that the server has dropped In order to ensure the normal processing of log information, the server needs to be reminded.
  • the log processing method provided in this embodiment determines that the server is disconnected from the shared memory if the client remains connected to the shared memory and the number of connections is less than the preset total number of connections Connection; the total number of connections is the number of connections to the shared memory in the normal log storage state, if the client is connected to the shared memory, and the number of connections is not less than the preset total number of connections, it is determined
  • the server is not disconnected from the shared memory.
  • the method further includes:
  • a reconnection request is sent to the server at a preset frequency.
  • a reconnection request can be sent to the server at a preset frequency, so that the server can re-establish a communication connection with the shared memory according to the reconnection request.
  • the preset frequency can be a preset frequency, or it can be set according to actual needs.
  • the client can send a reconnection request at a higher frequency; if the client The occupied shared memory is relatively large, and the reconnection request can also be sent at a higher frequency, which is not limited in the present invention.
  • a reconnection request is sent to the server at a preset frequency, so that the server can connect according to the reconnection request.
  • the shared memory can ensure the effectiveness of log writing, and further improve the efficiency of log processing.
  • the registration success information includes an identification code allocated by the server to the client; the reconnection request includes the identification code; the identification code is used Therefore, the server can connect to the shared memory according to the identification code in the reconnection request.
  • the server may send registration success information to the client, where the registration success information includes the identification code assigned by the server to the client.
  • the client detects that the server is disconnected from the shared memory, it can send a reconnection request including the identification code to the server. Therefore, the server can reconnect to the shared memory according to the identification code.
  • the identification code in the received registration success message is placed in the reconnection request and sent to the server, so that the server can reconnect to the shared memory according to the identification code, thereby enabling On the basis of ensuring the validity of log writing, the efficiency of log processing is further improved.
  • FIG. 6 is a schematic flowchart of a log processing method provided by Embodiment 6 of the present invention, which is applied to the server side. As shown in FIG. 6, the method includes:
  • Step 601 Receive a registration request sent by the client.
  • Step 602 Apply for shared memory for the client according to the registration request
  • Step 603 After the shared memory application is successful, send registration success information to the client;
  • Step 604 Obtain the log information from the shared memory and store it in a preset storage path.
  • the execution body of this embodiment is the server, where the server can be an application, a process, or a terminal device.
  • the client can use a preset calling interface to The server sends a registration request.
  • the server can receive the registration request sent by the client, and apply for shared memory for the client according to the registration request. For example, if the server’s current storage space is insufficient and its processing capacity is weak, it is not necessary to apply for shared memory for the client according to the registration request; if the server’s current storage space is large and its processing capacity is strong, it can be The registration request applies for shared memory for the client. After successfully applying for shared memory for the client, the registration success message can be sent to the client.
  • the client When receiving the registration success information sent by the server, the client can write log information that needs to be stored into the shared memory, where the registration success information can include the storage address of the shared memory. Since the client and the server are connected to the shared memory at the same time, after the client writes the log information in the shared memory, the server can obtain the log information from the shared memory and store it in a preset storage path.
  • the registration request is sent by one of the threads of the client.
  • the registration request may specifically be sent by one of the threads in the client. Since there are multiple threads in the client, each thread can send a registration request to the server. Accordingly, the server can apply for shared memory for the client corresponding to the thread according to the registration request sent by each thread. By separately applying for shared memory for multiple threads of the client, it is possible to realize log storage under high concurrency.
  • the client finishes writing all the log information, the client disconnects from the shared memory, and the server can reclaim the shared memory currently used by the client so that no other clients can apply for it later.
  • the log processing method provided in this embodiment receives a registration request sent by a client, applies for shared memory for the client according to the registration request, and sends a registration success message to the client after the shared memory application is successful , So that the client writes log information in the shared memory according to the registration success information, acquires the log information from the shared memory, and stores it in a preset storage path. Since the server can apply for shared memory according to at least one thread of at least one client, the log information generated by all processes in the client can be stored in the shared memory, which can be applied to the high concurrency of multiple threads in an application. In turn, the logs of multiple processes can be effectively managed.
  • the acquiring the log information from the shared memory and storing it in a preset storage path includes:
  • the server since the number of log information written by the client is large, different log information has different priorities. Therefore, in order to optimize the quality of log information and reduce storage space, when the server obtains the log information from the shared memory, it first needs to determine the priority of the log information and set the priority higher than the preset second printing level of the log information , Save to the preset storage path.
  • the second printing level can be set by the user according to actual needs, or can be a printing level preset by the system, which is not limited in the present invention.
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • the log processing method provided in this embodiment obtains log information with a priority higher than the preset second printing level from the shared memory and stores it in a preset storage path, thereby optimizing the quality of log information and reducing Improve log processing efficiency based on small storage space.
  • the server acquiring the log information from the shared memory and storing it in a preset storage path includes:
  • the server obtains the log information from the shared memory in a single-threaded manner and stores it in a preset storage path.
  • the server in order to improve log processing efficiency, after the client writes the log information into the shared memory, the server can obtain the log information from the shared memory in a single process and store it in a preset storage path.
  • the log processing method provided in this embodiment can effectively increase the log writing speed by acquiring the log information from the shared memory in a single-threaded manner and storing it in a preset storage path, thereby improving log processing. effectiveness.
  • FIG. 7 is a schematic flow chart of a log processing method provided by Embodiment 7 of the present invention. Based on any of the foregoing embodiments, as shown in FIG. 7, the registration request includes the amount of memory to be occupied, and the The request to apply for shared memory for the client includes:
  • Step 701 Determine the current remaining amount of memory on the server
  • Step 702 Apply for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied.
  • the registration request may include the amount of memory to be occupied.
  • the client when it sends a registration request to the server, it can determine the amount of memory to be occupied, and generate a registration request according to the amount of memory to be occupied.
  • the server After receiving the registration request, the server can first determine the current remaining amount of memory, and apply for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied.
  • the applying for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied includes:
  • the client can apply for shared memory corresponding to the amount of memory to be occupied; otherwise, if the amount of memory to be occupied is not less than the remaining amount of memory, the The current remaining memory of the server is insufficient to apply for shared memory for the client. At this time, it can refuse to apply for shared memory for the client.
  • the log processing method provided in this embodiment determines the current remaining amount of memory on the server, and applies for shared memory for the client based on the remaining amount of memory and the amount of memory to be occupied, so that it can quickly apply for the client Shared memory improves log processing efficiency.
  • FIG. 8 is a schematic flowchart of a log processing method provided by Embodiment 8 of the present invention.
  • the registration request also includes the process identifier; accordingly, the Applying for shared memory for the client according to the registration request includes:
  • Step 801 Determine whether the process identifier is included in the preset process blacklist
  • Step 802 If it does not exist, apply for shared memory for the process;
  • Step 803 If it exists, refuse to apply for shared memory for the process.
  • the registration request also includes the process identification.
  • the server can generate a blacklist based on historical experience.
  • the blacklist Include the process ID. Therefore, after the server receives the registration request that includes the process ID, it can detect whether the process ID is included in the blacklist. If so, it can refuse to apply for shared memory for the client. Otherwise, it can be the client according to the registration request. Apply for shared memory.
  • the log information generated by some clients may be more important or some clients have higher priority. Therefore, the server may also generate a whitelist based on historical experience, where the whitelist includes the process identifier.
  • the server After the server receives the registration request including the process ID, it can detect whether the process ID is included in the whitelist. If so, it can provide the client corresponding to the process ID with larger shared memory, or can provide the process Identifies the corresponding client to store log information with lower priority.
  • the server can provide different services to the clients according to different priorities. Therefore, after receiving the registration request including the process identifier, the server can first determine the priority corresponding to the process identifier, and provide the client with a service corresponding to the priority according to the priority.
  • the log processing method provided in this embodiment judges whether the process identifier is included in the preset process blacklist, if it does not exist, apply for shared memory for the process, and if it exists, refuse to apply for shared memory for the process , So that the server can selectively apply for shared memory for the client, and then can ensure the security of the server on the basis of ensuring the efficiency of log processing.
  • the registration request is generated according to a preset log writing configuration
  • the applying for shared memory for the client according to the registration request includes:
  • the client may first determine the log writing configuration, and generate a registration request according to the log writing configuration.
  • the log writing configuration may specifically be input by the user according to current requirements, or may be a default writing configuration, which is not limited in the present invention.
  • the server can apply for shared memory for the client according to the log writing configuration.
  • the log writing configuration includes at least one of memory size to be occupied, storage mode, storage location, storage subfolder division, and storage subfolder size.
  • the stored subfolders are used to divide the stored subfolders according to different functions of threads, and the stored subfolders may also be divided according to different types of logs.
  • different priorities can correspond to different log writing configurations. For example, the log writing configuration with a lower priority can only include the memory size to be occupied, while the log writing of the client with a higher priority
  • the configuration can also include other personalized demand parameters.
  • the method further includes:
  • the shared memory may be initialized to obtain a lock-free queue instance.
  • the lock-free queue instance can be a lock-free queue instance implemented by a Compare and Replace (Compare and Set, CAS for short) mechanism, or a lock-free queue instance implemented by any other mechanism, and the present invention is not limited here. .
  • Using a lock-free queue instance can save lock overhead on the basis of improving log processing efficiency.
  • the log processing method provided in this embodiment obtains a lock-free queue instance by initializing the shared memory, thereby improving the log processing effect, and further enabling log processing in a multi-process high concurrency scenario.
  • the method further includes:
  • the method further includes:
  • the client and the server need to be connected to the shared memory at the same time. Therefore, when the client detects that the server is disconnected from the shared memory, it can send a reconnection request to the server. Correspondingly, after receiving the reconnection request, the server can re-establish a communication connection with the shared memory according to the reconnection request.
  • the registration success information includes the identification code allocated by the server to the client;
  • the reconnection request includes the identification code;
  • the reconnection request to reconnect with the shared memory includes:
  • the server may send registration success information to the client, where the registration success information includes the identification code assigned by the server to the client.
  • the client detects that the server is disconnected from the shared memory, it can send a reconnection request including the identification code to the server. Therefore, the server can reconnect to the shared memory according to the identification code.
  • the log processing method provided in this embodiment connects to the shared memory according to the identification code in the reconnection request, so that the server can reconnect to the shared memory according to the identification code, and thus can ensure that the log is written On the basis of the effectiveness, the efficiency of log processing is further improved.
  • the method further includes:
  • the log writing status in the shared memory is monitored, and the client is managed and controlled according to the monitoring result.
  • the client can write log information in the shared memory after receiving the registration success information.
  • the server can monitor the log writing status in the shared memory, and control the client according to the monitoring results. For example, you can determine whether the client is a malicious client based on the log writing status, and whether the client is currently malfunctioning.
  • the log processing method provided in this embodiment monitors the log writing status in the shared memory, and manages and controls the client according to the monitoring result, so that the log processing efficiency can be improved on the basis of improving the log processing efficiency. Effective management and control.
  • the monitoring of the log writing status in the shared memory and the management and control of the client according to the monitoring result includes:
  • the current speed at which the client writes the log can be determined, and the status of the client can be determined according to the speed. Specifically, if it is detected that the speed at which the client writes the log exceeds a preset speed threshold, it can be determined that the client is currently malfunctioning, anyway, it means that the current client is operating normally.
  • the monitoring of the log writing status in the shared memory and the management and control of the client according to the monitoring result includes:
  • the number of logs currently written by the client can be determined, and the status of the client can be determined according to the number. Specifically, if it is detected that the number of logs written by the client exceeds a preset number threshold, it can be determined that the client is currently malfunctioning, otherwise, it means that the current client is operating normally.
  • the above two embodiments can be implemented separately or in combination.
  • it can be determined whether the number of logs currently written by the client exceeds the preset number threshold and whether the speed of the client currently writing logs exceeds the preset speed according to the number and speed of logs currently written by the client The threshold, if yes, it is determined that the client is currently malfunctioning, anyway, it means that the current client is operating normally.
  • the log processing method provided in this embodiment determines whether the client is currently malfunctioning by determining whether the number and/or speed of logs currently written by the client exceed the preset number and/or speed threshold, which can effectively achieve the End monitoring.
  • Fig. 9 is a schematic flowchart of a log processing method according to the ninth embodiment of the present invention. Based on any of the foregoing embodiments, as shown in Fig. 9, after determining that the client is faulty, the method further includes:
  • Step 901 Determine permission information corresponding to the client according to the process identifier.
  • Step 902 Manage and control the log information written by the client according to the authority information.
  • the permission information corresponding to the client can be determined according to the process identifier of the client, and the log information written by the client can be controlled according to the permission information.
  • the All log information is stored; accordingly, if the priority of the client currently writing to the log is lower, when it is detected that the number and/or speed of the current written log of the client exceeds the preset number and/or speed threshold, Part of the log information written by the client can be stored, or the client can be refused to continue to provide shared memory.
  • the authority information corresponding to the client is determined according to the process identifier, and the log information written by the client is controlled according to the authority information, so as to be effective for the client. Control.
  • the method further includes: acquiring the log information from the shared memory, and performing publishing processing on the log information.
  • the client writes the log information into the shared memory
  • the server can obtain the log information from the shared memory and perform publishing processing.
  • FIG. 10 is a schematic structural diagram of a client terminal device according to Embodiment 10 of the present invention. As shown in FIG. 10, the terminal device includes:
  • the memory 111 is used to store program codes
  • the processor 112 calls the program code, and when the program code is executed, is configured to perform the following operations:
  • the shared memory is a lock-free queue instance.
  • the shared memory is an example of a locked queue.
  • the registration request is sent by one of the threads of the client.
  • the processor when the processor writes the log information of the client into the shared memory, it is configured to:
  • the log of each thread in the client is written into the shared memory according to a predetermined thread storage sequence.
  • the processor when the processor writes the log information of the client into the shared memory, it is configured to:
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • the processor when the processor writes the log information of the client to the shared memory, it is configured to:
  • the log information of the client is written into the shared memory in an asynchronous storage manner.
  • the registration information also includes the process identifier of the process that sent the registration request, so that the server can determine whether to provide the process for the process according to the process identifier.
  • Memory information
  • the processor before sending a registration request to the server through the calling interface preset by the client, the processor is further configured to:
  • the registration request is generated according to the preset log writing configuration.
  • the log writing configuration includes at least one of the size of the memory to be occupied, the storage mode, the storage location, the division of storage subfolders, and the size of the storage subfolders.
  • the processor writes the log information of the client into the shared memory when receiving the registration success information fed back by the server, it also uses in:
  • the processor when determining whether the server is disconnected from the shared memory according to the number of connections, the processor is configured to:
  • the client remains connected to the shared memory, and the number of connections is less than the preset total number of connections, it is determined that the server is disconnected from the shared memory; the total number of connections is a normal log The number of connections of the shared memory in the storage state;
  • the processor is further configured to:
  • a reconnection request is sent to the server at a preset frequency.
  • the registration success information includes an identification code allocated by the server to the client; the reconnection request includes the identification code; the identification code is used Therefore, the server can connect to the shared memory according to the identification code in the reconnection request.
  • FIG. 11 is a schematic structural diagram of a service terminal device according to Embodiment 11 of the present invention. As shown in FIG. 11, the service terminal device includes: a memory 121 and a processor 122;
  • the memory 121 is used to store program codes
  • the processor 122 calls the program code, and when the program code is executed, is used to perform the following operations:
  • the registration success information is sent to the client, so that the client writes log information in the shared memory according to the registration success information, and the registration success information includes all The shared memory;
  • the log information is obtained from the shared memory and stored in a preset storage path.
  • the processor when the processor obtains the log information from the shared memory and stores it in a preset storage path, it is configured to:
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • the processor when the server obtains the log information from the shared memory and stores it in a preset storage path, the processor is configured to:
  • the server obtains the log information from the shared memory in a single-threaded manner and stores it in a preset storage path.
  • the registration request includes the amount of memory to be occupied, and when applying for shared memory for the client according to the registration request, the processor is configured to:
  • the processor when the processor applies for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied, the processor is configured to:
  • the registration request further includes the process identifier
  • the processor when applying for shared memory for the client according to the registration request, the processor is configured to:
  • the registration request is generated according to a preset log writing configuration
  • the processor When applying for shared memory for the client according to the registration request, the processor is configured to:
  • the log writing configuration includes at least one of the size of the memory to be occupied, the storage mode, the storage location, the division of storage subfolders, and the size of the storage subfolders.
  • the processor is further configured to:
  • an initialization operation is performed on the shared memory to obtain a locked queue instance.
  • the processor is further configured to:
  • the registration success information includes an identification code assigned by the server to the client;
  • the reconnection request includes the identification code;
  • the processor is When reconnecting to the shared memory according to the reconnection request, it is used to:
  • the processor after sending the registration success information to the client, the processor is further configured to:
  • the log writing status in the shared memory is monitored, and the client is managed and controlled according to the monitoring result.
  • the processor is configured to monitor the log writing status in the shared memory and to manage and control the client according to the monitoring result:
  • the processor when the processor monitors the log writing status in the shared memory, and controls the client according to the monitoring result, it is configured to:
  • the processor is further configured to:
  • the log information written by the client is managed and controlled according to the authority information.
  • the processor is further configured to: obtain the log information from the shared memory, and perform publishing processing on the log information.
  • the present invention also provides a log processing system, which includes
  • One or more processors are One or more processors;
  • One or more memories are One or more memories
  • the memory is used to store program codes
  • the processor is used to call the program code, and when the program code is executed, it is used to perform the following operations:
  • the server receives the registration request sent by the client
  • the server applies for shared memory for the client according to the registration request
  • the server After the shared memory application is successful, the server sends registration success information to the client;
  • the server obtains the log information from the shared memory and stores it in a preset storage path.
  • the above method can be completed by one processor or by multiple servers.
  • the client program and the server program are both set in the same device.
  • the client program corresponds to one processor for execution
  • the server program corresponds to one processor for execution.
  • the two can be set in the same device or in two independent devices.
  • the present invention is There is no restriction.
  • the shared memory is a lock-free queue instance.
  • the shared memory is a locked queue instance.
  • the registration request is sent by one of the threads of the client.
  • the processor when the processor writes the log information of the client into the shared memory through the client, it is configured to:
  • the log of each thread in the client is written into the shared memory through the client according to a predetermined thread storage sequence.
  • the processor when the processor writes the log information of the client into the shared memory through the client, it is configured to:
  • the log information whose priority is higher than the preset first printing level is stored in the shared memory through the client.
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • the processor when the processor writes the log information of the client into the shared memory through the client, it is configured to:
  • the log information of the client is written into the shared memory through the client in an asynchronous storage manner.
  • the processor before sending a registration request to the server through the calling interface preset by the client, the processor is further configured to:
  • the registration request is generated by the client according to a preset log writing configuration.
  • the log writing configuration includes at least one of the size of the memory to be occupied, the storage mode, the storage location, the division of storage subfolders, and the size of the storage subfolders.
  • the processor when the processor receives the registration success information fed back by the server, writes the log information of the client into the shared memory through the client After the middle, it is also used for:
  • the client determines whether the server is disconnected from the shared memory according to the number of connections.
  • the processor determines whether the server is disconnected from the shared memory according to the number of connections through the client, it is configured to:
  • the client determines that the server is disconnected from the shared memory; wherein, The total number of connections is the number of connections of the shared memory in the normal log storage state;
  • the client maintains a connection with the shared memory, and the number of connections is not less than a preset total number of connections, it is determined by the client that the server is not disconnected from the shared memory.
  • the processor is further configured to:
  • the server When the server receives the reconnection request sent by the client, the server reconnects to the shared memory according to the reconnection request.
  • the registration success information includes the identification code allocated by the server to the client;
  • the reconnection request includes the identification code;
  • the processor is passing When the server reconnects to the shared memory according to the reconnection request, it is used to:
  • the processor when the processor obtains the log information from the shared memory through the server and stores it in a preset storage path, it is configured to:
  • the printing level of the log information includes OFF level, FATAL level, ERROR level, WARN level, INFO level, DEBUG level, and TRACE level in descending order of priority. And ALL level.
  • the processor when the processor obtains the log information from the shared memory through the server and stores it in a preset storage path, it is configured to:
  • the log information is obtained from the shared memory in a single-threaded manner through the server and stored in a preset storage path.
  • the registration request includes the amount of memory to be occupied
  • the processor uses the server to apply for shared memory for the client according to the registration request.
  • the server applies for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied.
  • the processor when the processor applies for shared memory for the client according to the remaining amount of memory and the amount of memory to be occupied through the server, the processor is configured to:
  • the registration request further includes the process identifier of the client
  • the processor when applying for shared memory for the client through the server according to the registration request, the processor is configured to:
  • the registration request is generated according to a preset log writing configuration
  • the processor applies for shared memory for the client according to the registration request through the server, it is configured to:
  • the server applies for shared memory for the client according to the log writing configuration.
  • the log writing configuration includes at least one of the size of the memory to be occupied, the storage mode, the storage location, the division of storage subfolders, and the size of the storage subfolders.
  • the processor after the processor receives the registration request sent by the client through the server, the processor further includes:
  • the server initializes the shared memory to obtain a locked queue instance.
  • the processor after the processor receives the registration request sent by the client through the server, the processor further includes:
  • the server initializes the shared memory to obtain a locked queue instance.
  • the processor after the processor sends registration success information to the client through the server, the processor further includes:
  • the log writing status in the shared memory is monitored by the server, and the client is managed and controlled according to the monitoring result.
  • the processor monitors the log writing status in the shared memory through the server, and controls the client according to the monitoring result. in:
  • the server determines whether the speed at which the client writes the log exceeds a preset speed threshold
  • the processor monitors the log writing status in the shared memory through the server, and controls the client according to the monitoring result. in:
  • the server determines whether the number of logs written by the client exceeds a preset number threshold
  • the processor is further configured to:
  • the server manages and controls the log information written by the client according to the authority information.
  • the processor is configured to:
  • the server obtains the log information from the shared memory, and publishes the log information.
  • this embodiment also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the log processing method described in the foregoing embodiment.
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.
  • the above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium.
  • the above-mentioned software functional unit is stored in a storage medium and includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute the method described in the various embodiments of the present invention. Part of the steps.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

La présente invention concerne, selon ses modes de réalisation, un procédé et un dispositif de traitement de journal, ainsi qu'un support d'informations lisible par ordinateur. Le procédé consiste : à envoyer, par un client, une requête d'enregistrement à un serveur au moyen d'une interface prédéfinie d'appel ; à recevoir, par le serveur, la requête d'enregistrement envoyée par le client ; à demander, par le serveur, une mémoire partagée pour le client, selon la requête d'enregistrement ; après succès de la demande de mémoire partagée, à envoyer, par le serveur, des informations de succès d'enregistrement au client ; lors de la réception des informations de succès d'enregistrement renvoyées par le serveur, à écrire, par le client, les informations de journal du client dans la mémoire partagée ; et à acquérir, par le serveur, les informations de journal à partir de la mémoire partagée et à stocker les informations de journal dans un chemin prédéfini de stockage. Comme le serveur peut demander, selon au moins un fil d'au moins un client, la mémoire partagée pour le client correspondant au fil, le procédé de traitement de journal peut s'appliquer à un scénario de simultanéité élevé et peut donc gérer efficacement les journaux d'une pluralité de processus.
PCT/CN2019/091998 2019-06-20 2019-06-20 Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur WO2020252724A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980012096.4A CN111699479A (zh) 2019-06-20 2019-06-20 日志处理方法、设备及计算机可读存储介质
PCT/CN2019/091998 WO2020252724A1 (fr) 2019-06-20 2019-06-20 Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/091998 WO2020252724A1 (fr) 2019-06-20 2019-06-20 Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur

Publications (1)

Publication Number Publication Date
WO2020252724A1 true WO2020252724A1 (fr) 2020-12-24

Family

ID=72476455

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/091998 WO2020252724A1 (fr) 2019-06-20 2019-06-20 Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN111699479A (fr)
WO (1) WO2020252724A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112527614A (zh) * 2020-12-08 2021-03-19 广东小天才科技有限公司 一种智能设备日志量预警系统及方法
CN113051243A (zh) * 2021-03-31 2021-06-29 上海阵量智能科技有限公司 日志处理方法、装置、系统、芯片、设备及存储介质
CN114172946B (zh) * 2021-12-06 2024-06-18 京东科技信息技术有限公司 请求结果发送方法、装置、电子设备和计算机可读介质

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6219672B1 (en) * 1997-12-11 2001-04-17 Kabushiki Kaisha Toshiba Distributed shared memory system and method of controlling distributed shared memory
US20030172093A1 (en) * 2001-03-26 2003-09-11 Mitsugu Nagoya Server duplexing method and duplexed server system
CN102420814A (zh) * 2011-11-21 2012-04-18 航天科工深圳（集团）有限公司 一种数据访问方法、装置及服务器
CN107066405A (zh) * 2017-03-31 2017-08-18 联想(北京)有限公司 一种内存装置、互连子系统以及内存的共享方法
CN107241335A (zh) * 2017-06-16 2017-10-10 上海闻泰信息技术有限公司 信息共享的方法及装置
CN107579864A (zh) * 2017-10-16 2018-01-12 深圳大宇无限科技有限公司 请求监控方法、装置及服务器
CN108566552A (zh) * 2018-04-20 2018-09-21 四川长虹网络科技有限责任公司 适用于数字机顶盒的多媒体播放方法及系统
CN109471743A (zh) * 2018-11-12 2019-03-15 深圳前海微众银行股份有限公司 日志收集方法、装置及计算机可读存储介质
CN109508246A (zh) * 2018-06-25 2019-03-22 广州多益网络股份有限公司 日志记录方法、系统和计算机可读存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103514053B (zh) * 2013-09-22 2017-01-25 中国科学院信息工程研究所 一种基于共享内存的进程间通讯方法
CN106648931A (zh) * 2016-12-12 2017-05-10 贵阳朗玛信息技术股份有限公司 一种日志信息记录的方法及装置
CN107491355A (zh) * 2017-08-17 2017-12-19 山东浪潮商用系统有限公司 一种基于共享内存的进程间功能调用方法及装置

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6219672B1 (en) * 1997-12-11 2001-04-17 Kabushiki Kaisha Toshiba Distributed shared memory system and method of controlling distributed shared memory
US20030172093A1 (en) * 2001-03-26 2003-09-11 Mitsugu Nagoya Server duplexing method and duplexed server system
CN102420814A (zh) * 2011-11-21 2012-04-18 航天科工深圳（集团）有限公司 一种数据访问方法、装置及服务器
CN107066405A (zh) * 2017-03-31 2017-08-18 联想(北京)有限公司 一种内存装置、互连子系统以及内存的共享方法
CN107241335A (zh) * 2017-06-16 2017-10-10 上海闻泰信息技术有限公司 信息共享的方法及装置
CN107579864A (zh) * 2017-10-16 2018-01-12 深圳大宇无限科技有限公司 请求监控方法、装置及服务器
CN108566552A (zh) * 2018-04-20 2018-09-21 四川长虹网络科技有限责任公司 适用于数字机顶盒的多媒体播放方法及系统
CN109508246A (zh) * 2018-06-25 2019-03-22 广州多益网络股份有限公司 日志记录方法、系统和计算机可读存储介质
CN109471743A (zh) * 2018-11-12 2019-03-15 深圳前海微众银行股份有限公司 日志收集方法、装置及计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
RUAN, XIACHENG: "The design and implementation of general logging system under distributed computing environment", CHINA MASTER'S THESES FULL-TEXT DATABASE, INFORMATION SCIENCE AND TECHNOLOGY, no. 03, 15 March 2016 (2016-03-15), XP055766303 *

Also Published As

Publication number Publication date
CN111699479A (zh) 2020-09-22

Similar Documents

Publication Publication Date Title
JP6600373B2 (ja) トラフィックディレクタ環境におけるトラフィックのアクティブ−パッシブルーティングおよび制御のためのシステムおよび方法
US9141491B2 (en) Highly available server system based on cloud computing
US11240745B2 (en) Network slice instance processing method and apparatus, and system
US9753786B2 (en) Client server communication system
WO2020252724A1 (fr) Procédé et dispositif de traitement de journal et support d'informations lisible par ordinateur
US10884880B2 (en) Method for transmitting request message and apparatus
US20110173319A1 (en) Apparatus and method for operating server using virtualization technique
CN111147274B (zh) 为集群解决方案创建高度可用的仲裁集的系统和方法
JP2008305070A (ja) 情報処理装置および情報処理装置システム
WO2024016624A1 (fr) Procédé et système d'accès multi-grappes
CA3138764A1 (fr) Methode de traitement de donnees, dispositif, equipement informatique et support de stockage
US8458702B1 (en) Method for implementing user space up-calls on java virtual machine before/after garbage collection
CN112559461A (zh) 文件传输方法及装置、存储介质及电子设备
CN113467873A (zh) 虚拟机的调度方法、装置、电子设备及存储介质
CN114553900B (zh) 一种分布式块存储管理系统、方法及电子设备
JP2013218449A (ja) クラウドコンピューティングシステム
US8407343B2 (en) System, method, and computer-readable medium for managing communications among service modules
CN112994918B (zh) 一种设备管理方法及装置
CN115361271B (zh) Ssh服务器切换与连接方法、云端服务器及存储介质
CN116301567A (zh) 一种数据处理系统、方法及设备
CN115378557A (zh) 热备实现方法、装置、系统、电子设备及存储介质
US10225135B2 (en) Provision of management information and requests among management servers within a computing network
CN116301568A (zh) 一种数据访问方法、装置以及设备
JP2011203848A (ja) データベースを分散共有する資源管理装置、プログラム及び方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19933744

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19933744

Country of ref document: EP

Kind code of ref document: A1