CN108427619B - Log management method and device, computing equipment and storage medium - Google Patents
Log management method and device, computing equipment and storage medium Download PDFInfo
- Publication number
- CN108427619B CN108427619B CN201810205067.7A CN201810205067A CN108427619B CN 108427619 B CN108427619 B CN 108427619B CN 201810205067 A CN201810205067 A CN 201810205067A CN 108427619 B CN108427619 B CN 108427619B
- Authority
- CN
- China
- Prior art keywords
- log
- server
- list
- user
- service processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a log management method and device, a computing device and a storage medium. The log management method comprises the following steps: acquiring one or more user identification lists; monitoring a business processing operation related to a business processing request, wherein the business processing request comprises one or more user identifications; generating a log record corresponding to a business processing operation, the log record comprising one or more user identifications related to the business processing request; storing the log record when at least one of the one or more user identities is determined to belong to the one or more user identity lists.
Description
Technical Field
The present application relates to the field of operation and maintenance technologies, and in particular, to a log management method and apparatus, a computing device, and a storage medium.
Background
With the development of the internet, a large amount of log records are recorded in the background of each website. Currently, the website background typically stores log records of all service requests. When the user access amount is large, the number of logs can be increased synchronously. However, existing solutions typically log the corresponding log on a per user request basis. Here, a large number of log storage operations occupy a large amount of resources, which degrades background performance.
Disclosure of Invention
The application provides a log management scheme, which can improve the efficiency of log analysis.
According to an aspect of the present application, there is provided a log management method, the method including: acquiring one or more user identification lists; service processing operations are monitored in connection with a service processing request, the service processing request including one or more subscriber identities. Generating a log record corresponding to a business processing operation, the log record comprising one or more user identifications related to the business processing request; storing the log record when at least one of the one or more user identities is determined to belong to the one or more user identity lists.
In some embodiments, the method further comprises: and when at least one user identifier in the one or more user identifiers is determined to belong to the one or more user identifier lists, sending the log record to a log management server.
In some embodiments, the obtaining one or more lists of user identities comprises: and acquiring the one or more user identification lists from the log management server.
In some embodiments, the method further comprises: and deleting the log record when determining that all the user identifications in the one or more user identifications do not belong to the one or more user identification lists.
In some embodiments, the one or more lists of subscriber identities include a first list of subscriber identities for recording globally unique identifiers and a second list of subscriber identities for recording international mobile equipment identities, the one or more subscriber identities include a globally unique identifier associated with the service handling request and an international mobile equipment identity associated with the service handling request; storing the log record when it is determined that at least one of the one or more subscriber identities belongs to the one or more subscriber identity lists of the first subscriber identity list, including: storing the log record when it is determined that the globally unique identifier associated with the service processing request belongs to the first subscriber identity list; storing the log record when it is determined that the international mobile equipment identity associated with the service processing request belongs to the second subscriber identity list.
According to another aspect of the present application, there is provided a log management method, the method including: acquiring one or more user identification lists; sending the one or more user identification lists to one or more service processing servers, so that each service processing server respectively returns a log record corresponding to the one or more user identification lists, wherein the user identification of the log record returned by each service processing server belongs to the one or more user identification lists; receiving the log record.
In some embodiments, the method further comprises: acquiring a log management server list, wherein the log management server list is used for describing server nodes contained in a log management server cluster; selecting a server node used for storing the log in the log management server list according to a screening rule, and taking the selected server node as a current storage server; storing the log record to the storage server.
In some embodiments, the obtaining the log management server list includes: the log management server list is obtained from a database.
In some embodiments, the selecting, according to the filtering rule, one server node in the log management server list for centrally storing the log, and using the selected server node as the current storage server, includes: and sequencing the server nodes according to the internet protocol addresses of the server nodes in the list in the log management server to obtain a sequencing result, and selecting one server node as the storage server according to the sequencing result.
In some embodiments, said storing said one or more log records to said storage server comprises: when the local server is determined to be the storage server, storing the log record at the local server; when the local server is determined not to be the storage server, sending the log record to the storage server so that the storage server can store the log record.
In some embodiments, the method further comprises: when the local server is started, sending a starting notification message indicating the starting of the local server to the database, so that the database adds the local server to the log management server list according to the starting notification message.
According to another aspect of the present application, there is provided a log management apparatus, the apparatus including: a list acquisition unit configured to acquire one or more user identifier lists; a service monitoring unit, configured to monitor a service processing operation related to a service processing request, where the service processing request includes one or more user identifiers; a log generating unit, configured to generate a log record corresponding to the service processing operation, where the log record includes one or more user identifiers related to the service processing request; and a log storage unit, configured to store the log record when it is determined that at least one of the one or more user identifiers belongs to the one or more user identifier lists.
In some embodiments, the apparatus further comprises: and the log forwarding unit is used for sending the log record to a log management server when at least one user identifier in the one or more user identifiers is determined to belong to the one or more user identifier lists.
In some embodiments, the list obtaining unit obtains one or more user identification lists according to the following: and acquiring the one or more user identification lists from the log management server.
In some embodiments, the log storage unit is further configured to: and deleting the log record when determining that all the user identifications in the one or more user identifications do not belong to the one or more user identification lists.
In some embodiments, the one or more lists of subscriber identities include a first list of subscriber identities for recording globally unique identifiers and a second list of subscriber identities for recording international mobile equipment identities, the one or more subscriber identities include a globally unique identifier associated with the service handling request and an international mobile equipment identity associated with the service handling request; the log storage unit stores the log record when determining that at least one of the one or more user identifications belongs to the one or more user identification lists according to the following mode: storing the log record when it is determined that the globally unique identifier associated with the service processing request belongs to the first subscriber identity list; storing the log record when it is determined that the international mobile equipment identity associated with the service processing request belongs to the second subscriber identity list.
According to another aspect of the present application, there is provided a log management apparatus, the apparatus including: a list acquisition unit configured to acquire one or more user identifier lists; a list sending unit, configured to send the one or more user identifier lists to one or more service processing servers, so that each service processing server returns a log record corresponding to the one or more user identifier lists, where the user identifier of the log record returned by each service processing server belongs to the one or more user identifier lists; a log receiving unit for receiving the log record.
In some embodiments, the list obtaining unit is further configured to: acquiring a log management server list, wherein the log management server list is used for describing server nodes contained in a log management server cluster; the device further comprises: the screening unit is used for selecting one server node used for storing the log in the log management server list according to a screening rule and taking the selected server node as a current storage server; and the log storage unit is used for storing the log record to the storage server.
In some embodiments, the list acquisition unit acquires the log management server list according to: the log management server list is obtained from a database.
In some embodiments, the screening unit selects one server node in the log management server list for collectively storing the logs according to the screening rule based on the following manner, and takes the selected server node as the current storage server: and sequencing the server nodes according to the internet protocol addresses of the server nodes in the list in the log management server to obtain a sequencing result, and selecting one server node as the storage server according to the sequencing result.
In some embodiments, the log storage unit stores the log record to the storage server according to: when the local server is determined to be the storage server, storing the log record at the local server; when the local server is determined not to be the storage server, sending the log record to the storage server so that the storage server can store the log record.
In some embodiments, the apparatus further comprises a notification unit to: when the local server is started, sending a starting notification message indicating the starting of the local server to the database, so that the database adds the local server to the log management server list according to the starting notification message.
According to another aspect of the present application, there is provided a computing device comprising: one or more processors, memory, and one or more programs. One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the log management methods of the present application.
According to another aspect of the present application, there is provided a storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the log management method of the present application.
In summary, according to the technical solution of the present application, a log record for analyzing a service processing request in detail can be generated. In addition, according to the technical scheme, the log records can be screened by judging whether the user identifiers in the log records are in the user identifier list or not, and the screening results are stored. Therefore, the technical scheme of the application can avoid resource consumption caused by storing all generated log records, and can store the log records meeting the screening condition. In other words, according to the technical scheme, by automatically storing the specific log records (namely, the log records meeting the screening condition), the log records required for log analysis can be accurately recorded, and server resources can be saved.
Drawings
In order to more clearly illustrate the technical solutions in the examples of the present application, the drawings needed to be used in the description of the examples are briefly introduced below, and it is obvious that the drawings in the following description are only some examples of the present application, and it is obvious for a person skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1A illustrates a schematic diagram of an application scenario in accordance with some embodiments of the present application;
FIG. 1B illustrates a flow diagram of one business process operation of the business process server 131, according to some embodiments of the present application;
FIG. 1C illustrates a schematic diagram of an application scenario in accordance with some embodiments of the present application;
FIG. 2A illustrates a flow diagram of a log management method 200 according to some embodiments of the present application;
fig. 2B shows a flowchart of step S204;
FIG. 3 illustrates a flow diagram of a log management method 300 according to some embodiments of the present application;
FIG. 4 illustrates a flow diagram of a log management method 400 according to some embodiments of the present application;
FIG. 5 illustrates a flow diagram of a log management method 500 according to some embodiments of the present application;
FIG. 6 illustrates a flow diagram of a method 600 of determining a storage server according to one embodiment of the present application;
FIG. 7 illustrates a schematic diagram of a log management apparatus 700 according to some embodiments of the present application;
FIG. 8 illustrates a schematic diagram of a log management apparatus 800 according to some embodiments of the present application;
FIG. 9 illustrates a schematic diagram of a log management apparatus 900 according to some embodiments of the present application;
FIG. 10 illustrates a system interaction diagram according to some embodiments of the present application; and
FIG. 11 illustrates a block diagram of the components of a computing device.
Detailed Description
The technical solutions in the examples of the present application will be clearly and completely described below with reference to the drawings in the examples of the present application, and it is obvious that the described examples are only a part of the examples of the present application, and not all examples. All other examples, which can be obtained by a person skilled in the art without making any inventive step based on the examples in this application, are within the scope of protection of this application.
FIG. 1A illustrates a schematic diagram of an application scenario in accordance with some embodiments of the present application. As shown in fig. 1A, the application scenario includes a terminal device 110, a log management system 120, and a service processing server cluster 130. The terminal device 110 may be, for example, a desktop computer, a laptop computer, a mobile phone, or a tablet computer, among various computing devices. The business processing server cluster 130 can include one or more business processing servers, such as the business processing servers 131, 132, and 133 identified in FIG. 1A. The service processing server cluster 130 may obtain service processing requests from various user devices (e.g., the terminal device 110, etc.) through the network 140, and perform corresponding service processing operations.
In some embodiments, the business process server cluster 130 may return a response result to the user equipment after completing the business process operation. It should be noted that the service processing server cluster 130 may be various types of service processing platforms, and the application is not limited thereto. In one embodiment, the cluster of business processing servers 130 may provide application store services to user devices. The service processing server cluster 130 may also be referred to as an application acquisition platform, such as an application daemon. In particular, the cluster of business processing servers 130 may provide user devices with a list of applications and associated introductory information regarding downloadable applications. The service processing server cluster 130 may also send an application installation package to the user equipment according to the download request of the user equipment. In another embodiment, the cluster of business process servers 130 can be a web game platform. The business processing cluster 130 may provide user devices with business processing services related to webgames.
In an embodiment, when acquiring the service processing request a from one user equipment, the service processing server cluster 130 may perform a service processing operation in one service processing server (e.g. 131), and return a service processing result. More specifically, the business process operations performed by the business process server may be divided into a plurality of operation phases. Each phase of operation may also be referred to herein as a processing logic. Depending on the business process of the business process operation, multiple operational phases may have respective upstream and downstream relationships (which may also be referred to as dependencies). For example, FIG. 1B shows a flow diagram of a business process operation of the business process server 131. As shown in fig. 1B, the business processing operation may include processing logic 1, processing logic 2, processing logic N, and processing logic N + 1, which are sequentially executed. The processing logic 1 is, for example, to receive a data processing request. The business processing operation corresponding to request a may also have one or more processing logics between processing logic 2 and processing logic N. N is an integer greater than 2. The processing logic N + 1 is for example to return a data packet to the user equipment. It should be understood that fig. 1B is only an exemplary illustration, and actually, the processing logics may also have other dependencies with each other, which are not described herein again.
In another embodiment, when acquiring the service processing request B from one user equipment, the service processing server cluster 130 may perform a service processing operation on a plurality of service processing servers. For example, in response to the service processing request B, the service processing server 131 may perform the service processing operation B1The business process server 132 may perform business process operation b2The business process server 133 can perform the business process operation b3. Here, the business process operation b1、b2And b3May comprise one or more processing logics, which the application does not limit.
In summary, each of the service processing servers in the service processing server cluster 130 may execute one or more processing logics when performing the service processing operation related to the service processing request. Here, the business processing server may monitor the execution of all or a portion of the processing logic. In this way, the business processing server can generate a respective one or more log records for each processing logic monitored. For example, the service processing server 131 generates a log record when executing processing logic 1, processing logic 2, processing logic N, and processing logic N + 1, respectively. Here, the service processing server 131 may adopt various log generation methods, which is not limited in this application. By monitoring multiple processing logic of a business processing operation and generating corresponding log records, the business processing cluster 131 can generate log records at multiple processing logic for one business processing request. It should be appreciated that multiple log records corresponding to a single business process request may be used for correlation analysis of aspects such as server performance.
It should be noted that the service processing request from the user equipment may include one or more user identities. The type range of the user Identifier may include, for example, a Globally Unique Identifier (GUID), an International Mobile Equipment Identity (IMEI), and an Internet Protocol address (IP), but is not limited thereto. The service processing server can also obtain one or more user identification lists. Each list of subscriber identities is a set of subscriber identities of one type, for example a GUID list, an IMEI list or an IP list. The log record generated by the service processing server may include one or more subscriber identities associated with the service processing request. In this way, the service processing server may determine whether the user identifier belongs to the user identifier list in the generated log record based on the user identifier. For example, when it is determined that a subscriber identity of a log record belongs to a subscriber identity list, the service processing server may determine that the log record satisfies the acquired one or more subscriber identity lists. On this basis, the service processing server may select the generated log record according to the obtained one or more subscriber identity lists (i.e. select the log record satisfying the obtained one or more subscriber identity lists). For example, the obtained list of one or more user identifications may be considered to be a monitoring whitelist. The service processing server may select the log records that satisfy the monitored white list.
For the selected log record, the transaction server may store it locally and may also send it to the log management system 120. In this way, the log management system 120 can centrally manage the log records selected by the respective service processing servers. It is particularly noted that, because the log records can be selected according to the user identifier list, the service processing server can avoid resource consumption caused by storing all log records, and can only pay attention to the selected log records. Here, the selected log records may be used for service analysis such as application performance analysis, access anomaly diagnosis, and the like. The log management system 120 can avoid the trouble of traversing the service processing servers related to a service processing request when analyzing the service processing request by collecting the log records selected by each service processing server. In other words, the log management system 120 can greatly improve the convenience and efficiency of analyzing a specific service processing request (i.e., a service processing request whose user identifier satisfies the user identifier list).
FIG. 1C illustrates a schematic diagram of an application scenario in accordance with some embodiments of the present application. As shown in fig. 1C, the log management system 120 may include a content management system 121, a database 122, and a log management server cluster 123.
The database 122 may be a database application running on one server node, or may be a distributed database application, which is not limited in this application. The database 122 may store one or more lists of user identifications. For example, the database 122 may store 2 user identification lists. One list of user identities is the GUID list and the other list of user identities is the IMEI list. The content management system 121 can communicate with a terminal device (e.g., 110) and operate the database 122 according to an operation request from the terminal device. For example, for a user-identified list, the content management system 121 may perform operations such as add listings, delete listings, modify listings, and query listings. In summary, the content management system 121 can manage the database 122.
The log management server cluster 123 may include a plurality of log management servers, such as 1231, 1232, and 1233, labeled in FIG. 1C. Each server in the log management server cluster 123 may obtain one or more user identification lists from the database 122. For example, the log management servers 1231, 1232, and 1233 may each periodically obtain the latest list of user identifications from the database 122. On this basis, the log management server cluster 123 may send the list of user identifications to the traffic processing server cluster 130. In this way, the database 122 sends the user identifier list to the service processing server cluster 130 through the log management server cluster 123, so that the problem that the database 122 is overloaded due to the fact that the user identifier list is directly sent from the database 122 to the service processing server cluster 130 can be avoided. In one embodiment, each log management server in the cluster of log management servers 123 may communicate with one or more traffic processing servers in the cluster of traffic processing servers 130, respectively, for load balancing. In one embodiment, the log management server 1231 may send the list of user identifications to the traffic processing server 131. Similarly, the log management servers 1232 and 1233 may transmit the user identification lists to the service processing servers 132 and 133, respectively. In one embodiment, the log management server 1231 may store the user id list in a memory and provide a pull interface for obtaining the user id list to the service processing server 131. In addition, the service processing server 131 may transmit the log record to the log management server 1231. The business processing servers 132 and 133 can send respective log records to the log management servers 1232 and 1233, respectively.
It should be noted that, in order to perform centralized management on the log records, each server in the service processing server cluster 120 may transmit the received log records to one storage server. Here, the cluster of traffic processing servers 120 may employ various selected policies to treat one server in the cluster as a storage server.
FIG. 2A illustrates a flow diagram of a log management method 200 according to some embodiments of the present application. The log management method 200 may be performed, for example, in the business processing server 131, but is not limited thereto.
As shown in fig. 2A, the log management method 200 may include a step S201 of obtaining one or more user identification lists. In one embodiment, step S201 may obtain the user identifier list from the log management server cluster 123, for example, obtain the user identifier list from the log management server 1231. In one embodiment, step S201 may obtain the user identification list from the management node of the service processing server cluster 130, but is not limited thereto. In one embodiment, step S201 may obtain a list of GUIDs. In another embodiment, step S201 may acquire an IMEI list. In yet another embodiment, step S201 may obtain two lists, the GUID list and the IMEI list.
The method 200 may perform step S202 when the service processing server 131 is associated with a service processing request. In step S202, a service processing operation related to the service processing request is monitored. Here, the service processing request includes one or more subscriber identities.
In step S203, a log record corresponding to the business process operation is generated. The log record includes one or more user identifications associated with the service processing request. In one embodiment, a business processing operation may include one or more processing logics. Step S202 may monitor all or a portion of the processing logic. Accordingly, step S203 may generate one or more log records for each monitored processing logic. In this way, the generated log record can be used for detailed analysis of the execution process of the business process request.
On this basis, the service processing server 131 may determine whether the user identifier of the generated log record belongs to the user identifier list. In step S204, when it is determined that at least one of the one or more user identifiers in the log record belongs to the one or more user identifier lists, the log record is stored. Here, the log record may be locally stored in step S204, but is not limited thereto. In one embodiment, the business process server 131 includes a list of GUIDs. The service processing server 131 may determine whether the GUID value in the log record belongs to the range of the GUID list. In another embodiment, the traffic processing server 131 includes a list of IMEIs. The service processing server 131 can determine whether the IMEI value in the log belongs to the range of the IMEI list. In yet another embodiment, the traffic processing server 131 may include a GUID list and an IMEI list. The service processing server 131 may store the log record through step S204 when at least one of the two conditions that the GUID value in the log record belongs to the GUID list and the IMEI value in the log record belongs to the IMEI list is determined to be satisfied. For example, step S204 may first determine whether the GUID value in the log record belongs to the GUID list. Upon determining that the GUID value does not belong to the GUID list, step S204 may determine whether the IMEI value in the log record belongs to the IMEI list. In one embodiment, step S204 may invoke a handler for storing log records. Step S204 may be implemented as a flowchart as shown in fig. 2B. As shown in fig. 2B, in sub-step S2041, the GUID value and IMEI value of one log record are obtained. Note that the log record generated in step S203 is usually stored in the memory area. Substep S2041 may read the IMEI value and GUID value of a log record from the memory region of the log record. In sub-step S2042, it is determined whether the GUID value belongs to a GUID list. In sub-step S2042 determining that it does not belong to the GUID list, step S204 may perform sub-step S2043 determining whether the IMEI value belongs to the IMEI list. When the sub-step S2043 determines that it does not belong to the IMEI list, step S204 may perform sub-step S2044 to delete the log record. In this way, sub-step S2044 can save resources of the service processing server 131. Upon determining in sub-step S2042 that the GUID value belongs to the GUID list, step S204 may perform sub-step S2045, storing the log record. Here, sub-step S2045 may store the log record locally. In addition, when the sub-step S2043 determines that the IMEI value belongs to the IMEI list, step S204 may perform sub-step S2045.
In summary, the method 200 can generate a log record for analyzing business process requests in detail. In addition, the method 200 may filter the log record by determining whether the user identifier in the log record is in the user identifier list, and store the filtering result. In this way, the method 200 may avoid resource consumption caused by storing all generated log records, and may store log records satisfying the screening condition. In other words, the method 200 may not only accurately record the log records required for log analysis, but also save server resources by automatically storing the specific log records (i.e., the log records satisfying the filtering condition). It is specifically noted that the list of user identifications can be easily modified. Therefore, the user identification list has good expansibility. Based on the list of user identifications, the method 200 may flexibly adjust the logs to be stored.
FIG. 3 illustrates a flow diagram of a log management method 300 according to some embodiments of the present application. The log management method 300 may be performed, for example, in the business processing server 131, but is not limited thereto.
The log management method 300 may include steps S301, S302, S303, and S304. Here, the implementation of steps S301 to S304 is consistent with steps S201 to S204, and is not described here again.
In addition, the log management method 300 further includes step S305, when it is determined that at least one of the one or more user identifiers of the log record belongs to the one or more user identifier lists, sending the log record to a log management server, such as the log management server 1231. In this way, the method 300 can centralize the log records to the log management cluster 123 through step S305, so that the trouble of traversing the service processing server related to a service processing request when performing log analysis on the service processing request can be avoided. In other words, the method 300 can greatly improve the convenience and efficiency of analyzing a specific service processing request (i.e. a service processing request whose user identifier satisfies the user identifier list) through step S305.
In step S306, when it is determined that none of the user identifiers in the log record belongs to the user identifier list, the log record is deleted, that is, the log record is abandoned. Thus, by way of step S306, the method 300 may save storage space.
FIG. 4 illustrates a flow diagram of a log management method 400 according to some embodiments of the present application. The log management method 400 may be performed in the log management server 1231, for example, but is not limited thereto.
In step S401, one or more user identity lists are obtained. For example, step S401 may acquire a user identification list from the database 122. On this basis, the method 400 may perform step S402. In step S402, the obtained one or more user identification lists are sent to one or more service processing servers. For example, the user identification list is sent to the service processing server 131. In this way, the service processing server 131 can return a log record corresponding to the user identification list. And the user identification of the log record returned by each service processing server belongs to the user identification list. Thus, the method 400 may perform step S403, receiving a log record. In summary, the method 400 may provide the service processing server with the user identifier list so that the service processing server may filter the log records according to the user identifier list. In this way, the method 400 may receive the log records for filtering, so that the log records for analyzing a specific service processing request (i.e. a service processing request whose user identifier belongs to the user identifier list) may be managed centrally. When a service processing request needs to be log analyzed, the method 400 can quickly provide log records related to the request, thereby improving the efficiency of log analysis.
FIG. 5 illustrates a flow diagram of a log management method 500 according to some embodiments of the present application. The log management method 500 may be performed in the log management server 1231, for example, but is not limited thereto. As shown in fig. 5, method 500 may include steps S501, S502, and S503. Here, steps S501, S502, and S503 are consistent with the embodiments of steps S401 to S403, respectively, and are not described again here.
In addition, the method 500 may further include step S504 of obtaining a log management server list. The log management server list is used for describing the server nodes contained in the log management server cluster. In this way, the method 500 may obtain the latest online node list when the server node in the log management server cluster 123 changes through step S504. In one embodiment, step S504 may obtain the log management server list from the database 122. Accordingly, the database 122 may dynamically manage the list of server nodes of the log management server cluster 123.
In step S505, one server node in the log management server list for storing the log is selected according to the filtering rule, and the selected server node is used as the current storage server. Here, the method 500 may employ various selected policies to determine the storage server. In one embodiment, step S505 may rank each server node according to an internet protocol address (i.e., IP address) of each server node in the log management server list to obtain a ranking result. On the basis, step S505 may select a server node as the storage server according to the sorting result. Here, each server node performing the method 500 may avoid the trouble of needing to learn about the storage server from other servers, but may automatically determine the storage server through the result of the ranking. In summary, the method 500 may determine the storage server according to the latest log management server list. The following description will be given taking a comparison of two IP addresses as an example. Assuming that the IP address C is C1.C2.C3.C4 and the IP address D is D1.D2.D3.D4, the IP address C is larger than the IP address D when any of the following conditions is satisfied.
Condition 1: c1> D1
Condition 2: c1 ═ D1 and C2> D2
Condition 3: c1 ═ D1 and C2 ═ D2 and C3> D3
Condition 4: c1 ═ C1 and C2 ═ D2 and C3 ═ D3 and C4> D4
In step S506, the log record is stored to the storage server. For example, when it is determined that the local server is the storage server, step S506 stores the log record locally. When the local server is determined to be not a storage server, the log record is sent to the storage server so that the storage server stores the log record. In this way, the method 500 may centralize the log records received by the server nodes in the log management server cluster 123 to the same server node by performing steps S505 and S506. In this way, when a log analysis is required for one service processing request, the log management server cluster 123 can quickly provide each log record related to one service processing request.
In one embodiment, the method 500 further includes step S507. In step S507, when the local server starts, a start notification message indicating the start of the local server is transmitted to the database 122. In this way, the database 122 can add the server to the log management server list according to the start notification message, i.e., dynamically update the log management server list. The database 122 may delete one log management server from the list upon determining that the log management server is down or shutdown. In one embodiment, the log management server 1231 may perform the method 600. FIG. 6 illustrates a flow diagram of a method 600 of determining a storage server according to one embodiment of the present application.
In step S601, when the log management server 1231 is started, a start notification message indicating the start of the local server is transmitted to the database 122. In this way, the database 122 may add the server to the log management server list according to the start notification message. The method 600 further includes a step S602 of monitoring whether the local server is to exit the service. When step S602 determines that the service is to be closed, the method 600 may perform step S603, sending a notification message to the database 122 indicating that the server is out of service, so that the data 122 deletes the log management server 1231 from the log management server list. When the service is determined to be normal in step S602, the method 600 may perform step S604, and obtain the log management server list from the database 122. The method 600 may perform step S605 according to the log management server list acquired in step S604. In step S605, the current storage server is determined according to the filtering rule.
FIG. 7 illustrates a schematic diagram of a log management apparatus 700 according to some embodiments of the present application. The apparatus 700 may reside, for example, in a business process server 131. As shown in fig. 7, the apparatus 700 may include: list acquisition unit 701, service monitoring unit 702, log generation unit 703, log storage unit 704, and log forwarding unit 705.
A list obtaining unit 701, configured to obtain one or more user identifier lists; a service monitoring unit 702, configured to monitor a service processing operation related to the service processing request. The service processing request includes one or more subscriber identities. In one embodiment, the list obtaining unit 701 may obtain the one or more user identification lists from the log management server.
A log generating unit 703, configured to generate a log record corresponding to the service processing operation. The log record includes one or more subscriber identities associated with the service processing request.
A log storing unit 704, configured to store the log record when it is determined that at least one of the one or more user identities belongs to the one or more user identity lists. In some embodiments, the one or more user identification lists include a first user identification list for recording globally unique identifiers and a second user identification list for recording international mobile equipment identities. The one or more subscriber identities include a globally unique identifier associated with the service processing request and an international mobile equipment identity associated with the service processing request. The log storage unit 704 stores the log record when it is determined that the globally unique identifier associated with the service processing request belongs to the first subscriber identity list. The log storage unit 704 stores a log record when it is determined that the international mobile equipment identity associated with the service processing request belongs to the second subscriber identity list. When it is determined that none of the one or more user identifiers belongs to the one or more user identifier lists, the log storage unit may further delete the log record.
The log forwarding unit 705 is configured to send a log record to a log management server when it is determined that at least one of the one or more user identifiers belongs to the one or more user identifier lists. More specific embodiments of the apparatus 700 are consistent with the log management method 300 shown in fig. 3, and are not described herein again.
In summary, the log management apparatus 700 can generate log records for analyzing the business process requests in detail. In addition, the log management apparatus 700 may filter the log record by determining whether the user identifier in the log record is in the user identifier list, and store the filtering result. In this way, log management apparatus 700 can store log records satisfying the filtering condition while avoiding resource consumption due to storage of all generated log records. In other words, the log management device 700 can accurately record the log records required for log analysis and save server resources by automatically storing specific log records (i.e., log records satisfying the filtering condition).
FIG. 8 illustrates a schematic diagram of a log management apparatus 800 according to some embodiments of the present application. The apparatus 800 may reside in a log management server 1231, for example. As shown in fig. 8, the apparatus 800 may include: a list acquisition unit 801, a list transmission unit 802, and a log reception unit 803.
A list obtaining unit 801, configured to obtain one or more user identifier lists. In one embodiment, the list acquisition unit 801 may acquire the log management server list from one database.
A list sending unit 802, configured to send the one or more user identifier lists to one or more service processing servers, so that each service processing server returns a log record corresponding to the one or more user identifier lists. And the user identification of the log record returned by each service processing server belongs to the one or more user identification lists. A log receiving unit 803, configured to receive a log record. More specific embodiments of the apparatus 800 are consistent with the log management method 400 shown in fig. 4, and are not described herein again.
In summary, the log management apparatus 800 may provide the service processing server with the user identifier list so that the service processing server may filter the log records according to the user identifier list. In this way, the log management apparatus 800 can receive the log records for filtering, so that the log records for analyzing a specific service processing request (i.e. a service processing request whose user identifier belongs to the user identifier list) can be managed centrally. When a service processing request needs to be subjected to log analysis, the log management device 800 can quickly provide log records related to the request, so that the log analysis efficiency is improved.
FIG. 9 illustrates a schematic diagram of a log management apparatus 900 according to some embodiments of the present application. The apparatus 900 may reside in a log management server 1231, for example. As shown in fig. 9, the apparatus 900 may include: a list acquisition unit 901, a list transmission unit 902, and a log reception unit 903. Here, the list acquisition unit 901, the list transmission unit 902, and the log reception unit 903 are consistent with the embodiments of the list acquisition unit 801, the list transmission unit 802, and the log reception unit 803, respectively, and are not described here again.
In some embodiments, the list obtaining unit 901 is further configured to obtain a log management server list. The log management server list is used for describing the server nodes contained in the log management server cluster. The apparatus 900 may also include a screening unit 904, a log storage unit 905, and a notification unit 906.
In an embodiment, the screening unit 904 may sort the server nodes according to the ip addresses of the server nodes in the list of the log management server to obtain a sorting result. In this way, the filtering unit 904 may select one server node as the storage server according to the sorting result.
The log storage unit 905 is used to store the log record to the storage server. In one embodiment, when the local server is determined to be the storage server, the log storage unit 905 stores the log record in the local server; and when the local server is determined not to be the storage server, sending the log record to the storage server so that the storage server stores the log record.
The notification unit 906 is configured to send a start notification message indicating start of the local server to the database when the local server starts, so that the database adds the local server to the log management server list according to the start notification message. More specific embodiments of the apparatus 900 are consistent with the log management method 500 shown in fig. 5, and are not described herein again.
FIG. 10 illustrates a system interaction diagram according to some embodiments of the present application. As shown in fig. 10, the terminal device 110 may perform step S1001 to transmit an operation request for the user identification list to the content management system 121. The content management system 121 may perform step S1002, and transmit an operation instruction for the user identification list to the database 122 in response to the operation request from the terminal device 110. Here, the operation instruction is, for example, to add a list item in the user identification list, to delete the list, or to modify the list, or the like. In this way, the database 122 may execute step S1003 to modify the user id list according to the operation instruction. The log management server 1231 may send a retrieval request for the user identification list to the database. In this way, the database 122 may perform step S1004 to transmit the user identification list to the log management server 1231. The log management server 1231 may perform step S1005 to transmit the user identification list to the service processing server 131. The service processing server 131 may execute step S1006 when generating the log record, and determine whether the user identifier of the generated log record belongs to the user identifier list. Upon determining that the subscriber identity of the log record does not belong to the subscriber identity list, the service processing server 131 may delete the log record. When determining that the user identifier of the log record belongs to the user identifier list, the service processing server 131 may store the log record locally, and may further perform step S1007. In step S1007, the log record is transmitted to the log management server 1231. In this way, the log management server 1231 may perform step S1008 to store the log record to the storage server. In addition, the database can also store a log management server cluster list. When the log management server 1231 is to exit the service, step S1009 may be performed. In step S1009, a notification message of the withdrawal of the service is transmitted to the database. Thus, the database 122 may execute step S1010 to delete the service processing server 1231 in the log management server cluster list.
FIG. 11 illustrates a block diagram of the components of a computing device. As shown in fig. 11, the computing device includes one or more processors (CPUs) 1102, a communications module 1104, a memory 1106, a user interface 1110, and a communications bus 1108 for interconnecting these components.
The processor 1102 may receive and transmit data via the communication module 1104 to enable network communications and/or local communications.
The user interface 1110 includes one or more output devices 1112, including one or more speakers and/or one or more visual displays. The user interface 1110 also includes one or more input devices 1114. The user interface 1110 may receive, for example, an instruction of a remote controller, but is not limited thereto.
Memory 1106 may be high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices; or non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.
The memory 1106 stores a set of instructions executable by the processor 1102, including:
an operating system 1116, including programs for handling various basic system services and for performing hardware-related tasks;
the application 1118 includes various programs for implementing the above-described perspective adjustment method, and such programs can implement the log management process in the above-described embodiments, and may include, for example, the log management apparatus 700 shown in fig. 7, the log management apparatus 800 shown in fig. 8, and the log management apparatus 900 shown in fig. 9.
In addition, each of the examples of the present application may be realized by a data processing program executed by a data processing apparatus such as a computer. It is clear that a data processing program constitutes the present application.
Further, the data processing program, which is generally stored in one storage medium, is executed by directly reading the program out of the storage medium or by installing or copying the program into a storage device (such as a hard disk and/or a memory) of the data processing device. Such a storage medium therefore also constitutes the present invention. The storage medium may use any type of recording means, such as a paper storage medium (e.g., paper tape, etc.), a magnetic storage medium (e.g., a flexible disk, a hard disk, a flash memory, etc.), an optical storage medium (e.g., a CD-ROM, etc.), a magneto-optical storage medium (e.g., an MO, etc.), and the like.
The present application therefore also discloses a non-volatile storage medium having stored therein a data processing program for executing any one of the embodiments of the log management method described above.
In addition, the method steps described in this application may be implemented by hardware, for example, logic gates, switches, Application Specific Integrated Circuits (ASICs), programmable logic controllers, embedded microcontrollers, and the like, in addition to data processing programs. Such hardware capable of implementing the methods described herein may also constitute the present application.
The above description is only a preferred example of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present application should be included in the scope of the present application.
Claims (15)
1. A method of log management, the method comprising:
obtaining one or more user identification lists, wherein the one or more user identification lists comprise a first user identification list for recording a globally unique identifier;
monitoring a business process operation related to a business process request, the business process request including one or more user identifications including a globally unique identifier related to the business process request;
generating a log record corresponding to the business processing operation, wherein the log record comprises one or more user identifications related to the business processing request; and
when it is determined that at least one of the one or more user identities belongs to the list of one or more user identities, storing the log record, including: storing the log record when it is determined that the globally unique identifier associated with the service processing request belongs to the first subscriber identity list.
2. The method of claim 1, wherein the method further comprises: and when at least one user identifier in the one or more user identifiers is determined to belong to the one or more user identifier lists, sending the log record to a log management server.
3. The method of claim 1, wherein said obtaining one or more lists of user identities comprises: and acquiring the one or more user identification lists from the log management server.
4. The method of claim 1, wherein the method further comprises: and deleting the log record when determining that all the user identifications in the one or more user identifications do not belong to the one or more user identification lists.
5. The method of claim 1, wherein the one or more subscriber identity lists further comprise a second subscriber identity list for recording international mobile equipment identities, the one or more subscriber identities further comprising an international mobile equipment identity associated with the service processing request; the storing the log record when it is determined that at least one of the one or more user identities belongs to the list of one or more user identities, further comprising:
storing the log record when it is determined that the international mobile equipment identity associated with the service processing request belongs to the second subscriber identity list.
6. A method of log management, the method comprising:
obtaining one or more user identification lists, wherein the one or more user identification lists comprise a first user identification list for recording a globally unique identifier;
sending the one or more user identification lists to one or more service processing servers, so that each service processing server monitors service processing operation related to service processing request, the service processing request comprises one or more user identifications, generating a log record corresponding to the service processing operation, and returning the log record corresponding to the one or more user identification lists, wherein the user identification of the log record returned by each service processing server belongs to the one or more user identification lists;
receiving the log record.
7. The method of claim 6, wherein the method further comprises:
acquiring a log management server list, wherein the log management server list is used for describing server nodes contained in a log management server cluster;
selecting a server node used for storing the log in the log management server list according to a screening rule, and taking the selected server node as a current storage server;
storing the log record to the storage server.
8. The method of claim 7, wherein the obtaining the log management server list comprises: the log management server list is obtained from a database.
9. The method as claimed in claim 7, wherein said selecting one of the server nodes in the log management server list for collectively storing the logs according to the filtering rule and using the selected server node as the current storage server comprises:
and sequencing the server nodes according to the internet protocol addresses of the server nodes in the list in the log management server to obtain a sequencing result, and selecting one server node as the storage server according to the sequencing result.
10. The method of claim 7, wherein the storing the log record to the storage server comprises:
when the local server is determined to be the storage server, storing the log record at the local server;
when the local server is determined not to be the storage server, sending the log record to the storage server so that the storage server can store the log record.
11. The method of claim 8, wherein the method further comprises:
when the local server is started, sending a starting notification message indicating the starting of the local server to the database, so that the database adds the local server to the log management server list according to the starting notification message.
12. An apparatus for log management, the apparatus comprising:
a list obtaining unit, configured to obtain one or more user identifier lists, where the one or more user identifier lists include a first user identifier list used for recording a globally unique identifier;
a service monitoring unit, configured to monitor a service processing operation related to a service processing request, where the service processing request includes one or more user identifiers, and the one or more user identifiers include a globally unique identifier related to the service processing request;
a log generating unit, configured to generate a log record corresponding to the service processing operation, where the log record includes one or more user identifiers related to the service processing request; and
a log storage unit, configured to store the log record when it is determined that at least one of the one or more user identifiers belongs to the one or more user identifier lists;
the log storage unit is further configured to store the log record when it is determined that the globally unique identifier associated with the service processing request belongs to the first subscriber identity list.
13. An apparatus for log management, the apparatus comprising:
a list obtaining unit, configured to obtain one or more user identifier lists, where the one or more user identifier lists include a first user identifier list used for recording a globally unique identifier;
a list sending unit, configured to send the one or more user identifier lists to one or more service processing servers, so that each service processing server monitors a service processing operation related to a service processing request, where the service processing request includes one or more user identifiers, generates a log record corresponding to the service processing operation, and returns the log record corresponding to the one or more user identifier lists, where the user identifier of the log record returned by each service processing server belongs to the one or more user identifier lists;
a log receiving unit for receiving the log record.
14. A computing device, comprising:
one or more processors;
a memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any of claims 1-11.
15. A storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the method of any of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810205067.7A CN108427619B (en) | 2018-03-13 | 2018-03-13 | Log management method and device, computing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810205067.7A CN108427619B (en) | 2018-03-13 | 2018-03-13 | Log management method and device, computing equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108427619A CN108427619A (en) | 2018-08-21 |
| CN108427619B true CN108427619B (en) | 2020-10-20 |
Family
ID=63157931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810205067.7A Active CN108427619B (en) | 2018-03-13 | 2018-03-13 | Log management method and device, computing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108427619B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109495562B (en) * | 2018-11-09 | 2022-04-12 | 北京明略昭辉科技有限公司 | Method and device for acquiring equipment identifier |
| CN110110516A (en) * | 2019-01-04 | 2019-08-09 | 北京车和家信息技术有限公司 | Log recording method, apparatus and system |
| CN112783842B (en) * | 2019-11-07 | 2024-03-01 | 北京沃东天骏信息技术有限公司 | Log collection method and device |
| CN111538712B (en) * | 2020-04-30 | 2023-07-21 | 恒生电子股份有限公司 | Log recording method, processing node, electronic equipment and storage medium |
| CN112258298B (en) * | 2020-11-19 | 2024-08-09 | 中国民航信息网络股份有限公司 | Data processing method and device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009266078A (en) * | 2008-03-31 | 2009-11-12 | Nippon Telegr & Teleph Corp <Ntt> | System and method for gathering and utilizing context information |
| CN103440297A (en) * | 2013-08-20 | 2013-12-11 | 苏州迈科网络安全技术股份有限公司 | Method for recording and reading user operation log information |
| CN105100128A (en) * | 2014-04-24 | 2015-11-25 | 北京金山网络科技有限公司 | Server cluster log acquiring and providing methods, log server and node server |
| CN106777049A (en) * | 2016-12-09 | 2017-05-31 | 武汉斗鱼网络科技有限公司 | A kind of processing method and system for avoiding repeating daily record output |
-
2018
- 2018-03-13 CN CN201810205067.7A patent/CN108427619B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009266078A (en) * | 2008-03-31 | 2009-11-12 | Nippon Telegr & Teleph Corp <Ntt> | System and method for gathering and utilizing context information |
| CN103440297A (en) * | 2013-08-20 | 2013-12-11 | 苏州迈科网络安全技术股份有限公司 | Method for recording and reading user operation log information |
| CN105100128A (en) * | 2014-04-24 | 2015-11-25 | 北京金山网络科技有限公司 | Server cluster log acquiring and providing methods, log server and node server |
| CN106777049A (en) * | 2016-12-09 | 2017-05-31 | 武汉斗鱼网络科技有限公司 | A kind of processing method and system for avoiding repeating daily record output |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108427619A (en) | 2018-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427619B (en) | Log management method and device, computing equipment and storage medium | |
| US10805171B1 (en) | Understanding network entity relationships using emulation based continuous learning | |
| CN111131379B (en) | Distributed flow acquisition system and edge calculation method | |
| Zheng et al. | Investigating QoS of real-world web services | |
| US11539590B2 (en) | Detect impact of network maintenance in software defined infrastructure | |
| WO2016184175A1 (en) | Database processing method and apparatus | |
| WO2021151312A1 (en) | Method for determining inter-service dependency, and related apparatus | |
| US20030212778A1 (en) | UML representation of parameter calculation expressions for service monitoring | |
| CN113949707A (en) | OpenResty and K8S-based container cloud service discovery and load balancing method | |
| US20090049458A1 (en) | Interface for application components | |
| CN111556135A (en) | Request scheduling method, system and device and electronic equipment | |
| CN108093036B (en) | Method and device for acquiring resources | |
| EP3937502A1 (en) | Method, apparatus and device for pushing video stream, and storage medium | |
| CN114900449B (en) | Resource information management method, system and device | |
| US20140006554A1 (en) | System management apparatus, system management method, and storage medium | |
| CN117938863B (en) | Cluster-based joint simulation implementation method, system, equipment and storage medium | |
| CN109324892B (en) | Distributed management method, distributed management system and device | |
| CN112994934B (en) | Data interaction method, device and system | |
| CN112448833B (en) | Multi-management-domain communication method and device | |
| US11595471B1 (en) | Method and system for electing a master in a cloud based distributed system using a serverless framework | |
| CN115225645A (en) | Service updating method, device, system and storage medium | |
| CN113778780A (en) | Application stability determination method and device, electronic equipment and storage medium | |
| EP1720285B1 (en) | Apparatus and method for processing messages in a network management system | |
| CN114546979B (en) | Distributed storage system, and management method, device and equipment thereof | |
| CN110740151A (en) | micro-service adjusting method, device, server and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |