CN112527600A - Monitoring log processing method, device, equipment and storage medium - Google Patents

Monitoring log processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN112527600A
CN112527600A CN202011489454.1A CN202011489454A CN112527600A CN 112527600 A CN112527600 A CN 112527600A CN 202011489454 A CN202011489454 A CN 202011489454A CN 112527600 A CN112527600 A CN 112527600A
Authority
CN
China
Prior art keywords
log
monitoring
logs
local array
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011489454.1A
Other languages
Chinese (zh)
Inventor
单鹏飞
万齐帅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Bank Co Ltd
Original Assignee
Ping An Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Bank Co Ltd filed Critical Ping An Bank Co Ltd
Priority to CN202011489454.1A priority Critical patent/CN112527600A/en
Publication of CN112527600A publication Critical patent/CN112527600A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/03Credit; Loans; Processing thereof

Abstract

The invention relates to the technical field of cloud monitoring, is applied to the field of log monitoring, and discloses a monitoring log processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a monitoring log of a service front-end system in real time according to a preset log buried point, and storing the monitoring log into a local array; determining a log type of the monitoring log in the local array, wherein the log type comprises a conventional log; circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log type of the monitoring log in the local array; if the number of the conventional logs in the local array exceeds the preset number, deleting the conventional logs of the local array until the number of the conventional logs in the local array is smaller than the preset number; and when the local array stores abnormal logs, uploading all monitoring logs in the current local array to a log processing server.

Description

Monitoring log processing method, device, equipment and storage medium
Technical Field
The invention relates to the technical field of cloud monitoring, is applied to the field of log monitoring application, and particularly relates to a monitoring log processing method, a monitoring log processing device, monitoring log processing equipment and a storage medium.
Background
At present, with the development of information technology, application scenes of intelligent devices such as computers are increasingly rich, various application programs running on the intelligent devices are also increasingly rich, but errors are inevitable in actual running, and the intelligent devices used by users as front-end devices are very important to guarantee stable and normal running of the intelligent devices. Therefore, developers usually form various monitoring logs by monitoring the operation condition and operation content of the front end through a log system. Taking the bank front end as an example, the log system of the bank front end plays a very important monitoring role in developing and troubleshooting anomalies of the bank front end.
In the prior art, most of various monitoring logs are uploaded to a monitoring log server side, namely a back end, at regular intervals, so that developers can conveniently look up the monitoring logs, and problems are found. However, the uploaded monitoring logs are numerous and numerous, which has a high requirement on the processing capability of the monitoring log server, and is not favorable for saving server resources and traffic.
Disclosure of Invention
The invention relates to the technical field of cloud monitoring, is applied to the field of log monitoring application, and particularly provides a monitoring log processing method, a monitoring log processing device, computer equipment and a computer readable storage medium, so as to solve the problem that the prior art is not beneficial to saving server resources and saving flow.
A monitoring log processing method comprises the following steps:
acquiring a monitoring log of a service front-end system in real time according to a preset log buried point, and storing the monitoring log into a local array;
determining a log type of the monitoring log in the local array, wherein the log type comprises a conventional log;
circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log type of the monitoring log in the local array;
if the number of the conventional logs in the local array exceeds the preset number, deleting the conventional logs of the local array until the number of the conventional logs in the local array is smaller than the preset number;
and when the local array stores abnormal logs, uploading all monitoring logs in the current local array to a log processing server.
A monitoring log processing apparatus comprising:
the acquisition module is used for acquiring the monitoring log of the client in real time according to the preset log burying point;
the storage module is used for storing the monitoring log into a local array created by the client;
the determining module is used for determining the log type of the monitoring log in the local array, wherein the log type comprises a conventional log;
the judging module is used for circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log types of the monitoring logs in the local array;
the deleting module is used for deleting the conventional logs of the local array if the number of the conventional logs in the local array exceeds the preset number until the number of the conventional logs in the local array is smaller than the preset number;
and the uploading module is used for uploading all monitoring logs in the current local array to the log processing server side when the local array stores abnormal logs.
A computer device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, said processor implementing the steps of the above-mentioned monitoring log processing method when executing said computer program.
A computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the steps of the above-described monitoring log processing method.
In one scheme provided by the monitoring log processing method, the monitoring log processing device, the computer equipment and the storage medium, the info type log is discarded in a mode of setting a local array and circularly judging, so that the purpose of saving flow is achieved, the processing amount of the log processing at the log processing service end is reduced, the processing resources of a back-end server are saved, and the investment of calculation is reduced. In addition, when encountering abnormal logs such as war logs or error logs, the abnormal logs are uploaded to the log processing server, so that the flow of the terminal equipment and the processing resources of the log processing server can be reduced, the overall log monitoring operation and maintenance cost is reduced, and the packet loss rate can be reduced to a certain extent due to the small transmission data volume, so that the monitoring efficiency is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of a monitoring log processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a monitoring log processing method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of one embodiment of storing the monitoring log in the local array in step S10 of FIG. 2;
FIG. 4 is a schematic flow chart illustrating a monitoring log processing method according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an embodiment of step S50 of FIG. 2;
FIG. 6 is a schematic diagram of a monitoring log processing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The monitoring log processing method provided by the embodiment of the invention can be applied to an application environment shown in fig. 1, wherein the application environment comprises a developer, a front-end user, terminal equipment, a business front-end system, a log processing server, a Catlog server and a Catlog billboard. The business front-end system is operated on the terminal device, the business front-end system is used for providing business service for a front-end user, and can refer to various business service types, specifically, the business service types are not limited, for example, the business front-end system can provide a certain loan service, a developer refers to a person developing the business front-end system, the front-end user can operate a business page of the business front-end system, the business page can respond according to the operation of the front-end user, for example, a certain area of the business page is clicked to enter another business page, which is a response process of the business front-end system, a related monitoring log can be generated in the process of operating the business page by the front-end user, the terminal device is in communication connection with the log processing service end and interacts data/new signaling, and the terminal device is used for uploading the generated monitoring log to the log processing service end, and the log processing server feeds log acceptance results such as whether the monitoring log is accepted successfully or not back to the terminal equipment, the log processing server can write the monitoring log uploaded by the terminal equipment into a Catlog system of the Catlog server, the Catlog server can respond to the monitoring log writing results in real time, then a developer can check the content of front-end log data through the Catlog board, the Catlog board can inquire the monitoring log data from the Catlog server according to the frequent watching of the developer and return the monitoring log data to the Catlog board, so that the Catlog board feeds the checking response results back to the developer, namely the checking response results are displayed to the developer, and the developer analyzes the monitoring log, thereby analyzing and processing the service condition of the service front-end system.
The terminal device can be but not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the log processing server can be realized by an independent server or a server cluster formed by a plurality of servers.
In an embodiment, as shown in fig. 2, a monitoring log processing method is provided, where the method is applied to the terminal device in fig. 1, and includes the following steps:
s10: and acquiring a monitoring log of a service front-end system in real time according to a preset log buried point, and storing the monitoring log into a local array.
In the embodiment of the invention, the embedded point can be deployed in the service front-end service system application by using the embedded point code mode, so that when a front-end user uses the service front-end system, the embedded point is used for acquiring the corresponding monitoring log.
The monitoring log refers to some log records when the front-end user uses the front-end service system, and the monitoring log includes, but is not limited to, various data, such as: the browsing volume or the click volume of the service page in the front-end service system; the stay time of the front-end user on each service page; the front-end user accesses the service page through what entrance; the behavior triggered by the front-end user in the corresponding service page, and the like.
It should be noted that, in the embodiment of the present invention, a service page refers to any service page used by a front-end service system to provide a service for a front-end user. Taking the loan service as an example, the loan service system may include different service pages such as a loan application page, a loan state page, a loan detail page, and the like, and when the related behaviors of the service pages meet preset buried point triggering conditions, the front-end user records the related behaviors, thereby obtaining a monitoring log.
In this step, after the monitoring log of the service front-end system is obtained in real time according to the preset log burying point, the monitoring log of the service front-end system is stored in the local array. Wherein the local array is the local array created by the terminal device.
S20: determining a log type of the monitoring log in the local array, wherein the log type comprises a regular log.
The monitoring logs have different log types according to different monitoring data, and generally comprise a regular log and an exception log, wherein in an application scene, the regular log comprises an info type log, and the exception log comprises a war type log and an error type log.
In the embodiment of the invention, after the monitoring log is stored in the local array, the log type of the monitoring log in the local array is determined. For example, if there are monitor Log 1, monitor Log 2, monitor Log 1-N, and monitor Log N, then the log types of monitor Logs 1-N are determined, respectively.
S30: and circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log types of the monitoring logs in the local array.
In this step, after determining the log type of the monitoring log in the local array, according to the log type of the monitoring log in the local array, circularly determining whether the number of the regular logs in the local array exceeds a preset number. It should be noted that, because the monitoring logs stored in the local array are stored in real time, the types and the number of the monitoring logs in the local array may be changed all the time, and in this step, it needs to continuously loop to determine whether the number of the regular logs in the local array exceeds the preset number.
The preset number is not specifically limited, and it is verified that in an application scenario, the preset number is optimal within a range of 4-10, for example, it is determined in a loop whether the number of the regular logs in the local array exceeds 4, and it is again described that there is no fixed limit here, and the number of the regular logs may be stored by itself.
S40: and if the number of the conventional logs in the local array exceeds the preset number, deleting the conventional logs of the local array until the number of the conventional logs in the local array is less than the preset number.
For example, if the number of regular logs in the local array is determined to exceed 4, the regular logs in the local array may be randomly deleted until the number of regular logs in the local array is less than 4. It should be noted that, through this step, the purpose of discarding the conventional log can be achieved, and at the same time, a part of the conventional log is retained, because the info-type log is the conventional operation log of the user, but when the problem of war and error occurs, the problem still needs to be checked in context, so that a part of the info-type log is also needed, which is helpful for checking the problem according to the monitoring log in the following without occupying too much space and traffic of the terminal device.
S50: and when the local array stores abnormal logs, uploading all monitoring logs in the current local array to a log processing server.
In the embodiment of the present invention, it is determined whether an abnormal log, for example, the aforementioned wan type log and/or error type log, is stored in the local array, and when the abnormal log is stored in the local array, all monitoring logs in the local array are uploaded to the log processing server.
Therefore, in the embodiment of the present invention, a monitoring log processing method is provided, in which an info type log is discarded in a manner of setting a local array and performing a cyclic judgment, so as to achieve a purpose of saving traffic, reduce a processing amount of a log processing service end for processing logs, save a processing resource of a back-end server, and reduce an input of computational power. In addition, when encountering abnormal logs such as war logs or error logs, the abnormal logs are uploaded to the log processing server, so that the flow of the terminal equipment and the processing resources of the log processing server can be reduced, the overall log monitoring operation and maintenance cost is reduced, and the packet loss rate can be reduced to a certain extent due to the small transmission data volume, so that the monitoring efficiency is effectively improved.
In an embodiment, as shown in fig. 3, in step S10, the storing the monitoring log in a local array specifically includes the following steps:
s11: creating an empty array as the local array to use the local array as a container for log localization.
S12: and configuring a plurality of function functions which are opened to the outside by the local array, wherein different function functions are respectively and correspondingly used as call inlets of monitoring logs of different log types.
For steps S11-S12, before locally storing the monitoring log, the terminal device creates an array with empty contents as a local array in advance, so as to use the local array as a container for log localization. And configuring a plurality of function functions for the local array to be respectively used as call entries of monitoring logs of different log types. Specifically configuring the local array to open a plurality of function functions to the outside can be realized by codes, which are not described in detail herein, and aims to call monitoring logs of different log types to be stored in the local data subsequently to provide corresponding interfaces.
For example, the plurality of function functions include a first function info (), a second function war (), and a third function error (), where the first function info (), the second function war (), and the third function error () respectively serve as entries of info, war, and error type logs. Thus, the info, war and error type logs can be called through error (), info () and war (), respectively.
S13: and when each function is called, converting the entry array object of each function into a target array.
After configuring function functions such as error (), info (), and war () for a local array, the type logs of info, war, and error can be called respectively through the function functions such as error (), info (), and war (), wherein it needs to be explained that the function functions are a calling method and can be called only by corresponding entry parameters. When each function is called through the input parameter, the input parameter array object of each function needs to be converted into a target array.
It will be appreciated that the parameterized class array object entries of each function is an object corresponding to the arguments passed to the function, e.g., when the info () method is called, its function parameterized entries are (123, "prolog", "a:" a ", b:" b ", true), when the corresponding entries are a class array. In a specific application, all elements in the class array definitions can be converted into the target array by a slice method.
For example, the specific implementation code may be array, protocol, slice, application (definitions), so that the class number group of the function may be converted into the target number group.
Example codes are as follows:
Figure BDA0002840310520000091
s14: and carrying out serialization processing on the target array corresponding to each function to obtain a serialized access object corresponding to each function.
S15: and calling the corresponding function through the serialized objects corresponding to the function functions to acquire the corresponding monitoring log serialized data.
S16: and storing the monitoring log serialized data into a local array.
As for steps S14-S15, it can be understood that since the class array definitions of the function do not distinguish the parameter types, the JavaScript character string value, the numeric value, the boolean value, the array, the object and other types can be compatible, and after the class array definitions are converted into the target array, the target array corresponding to each function can be serialized to obtain the serialized parameter objects.
For example, a json. striping () method can be used to perform serialization processing on a target array, and a corresponding function is called through a serialized reference object corresponding to each function to obtain corresponding monitoring log serialized data; and finally, serializing the data of the monitoring log and storing the serialized data into a local array. Therefore, the finally obtained localized monitoring log can be compatible with various data types, the problem of data incompatibility during subsequent uploading of the monitoring log can be avoided, and the compatibility of input data of the development embedded log is improved.
In an embodiment, as shown in fig. 4, in step S10, after the monitoring log of the client is obtained in real time according to the preset log burying point, the method further includes the following steps:
s60: and configuring a unique identification number for the life cycle of the service page which generates the monitoring log at present.
When a front-end user uses different service pages of a service front-end system, different service pages have different life cycles, and the life cycle of the service page refers to the time period from the time when the service page is opened to the time when the service page is closed. In the embodiment of the invention, a unique identification number is configured in the life cycle of the service page which generates the monitoring log at present. That is, for a certain service page, there is a unique identification number.
For example, if service page 1 is opened at 08:45 am and closed at 09:00 am, a unique identification number a1 is configured for service page 1 during the lifetime of 08:45-09: 00; for example, if business page 1 is opened at 14:45 PM and closed at 15:00 PM, a unique identification number A2 is configured for business page 1 during the lifecycle of 14:45-15:00 PM; service page 2 was opened at 09:45 am and closed at 10:00 am, and a unique identification number A3 was configured for service page 2 during the life cycle of 09:45-10:00 am.
S70: and setting a log warning value of a target log type corresponding to the current service page according to the identification mark number.
After a unique identification number is configured in the life cycle of a service page, a log warning value of a target log type corresponding to the current service page can be set according to the identification number. The target log type is generally a war type log or an error type log, and because too many war type logs or error type logs indicate that more errors may exist in the current service page, a corresponding log warning value needs to be set for the war type log or the error type log.
For example, for the service page 1, after the service page 1 is configured with a unique identification number a1 in the life cycle of 08:45-09:00, a warning value of a war type log or an error type generated in the life cycle of 08:45-09:00 of the service page 1 can be set according to the identification number a 1.
For another example, for the service page 1, after the service page 1 is configured with a unique identification number a2 in the life cycle of 14:45-15:00, a warning value of a war type log or an error type generated in the life cycle of 14:45-15:00 of the service page 1 can be set according to the identification number a 1.
For another example, for the service page 2, after the service page 2 is configured with the unique identification number A3 in the lifetime of 09:45-10:00, the warning value of the war type log or error type generated by the service page 2 in the lifetime of 09:45-10:00 can be set according to the identification number a 1.
S80: and judging whether the number of the monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page.
S90: and if the number of the monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page, automatically sending warning information to a related personnel terminal.
For steps S80-S90, the embodiment of the present invention configures different handlers for different types of monitoring logs, so as to handle the abnormality reflected by the monitoring logs. After the log warning value of the target log type corresponding to the current service page is set according to the identification number, whether the number of the monitoring logs corresponding to the target log type generated in the life cycle of the current service page exceeds the log warning value can be judged. And if the number of the monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page, automatically sending warning information to a related personnel terminal so as to facilitate the processing and analysis of related personnel.
For example, for the service page 1, if the number of the war type logs generated in the service page 1 in the life cycle of 08:45-09:00 is judged to exceed the warning value corresponding to the war type logs, and if the number of the war type logs generated in the service page 1 in the life cycle of 08:45-09:00 is judged to exceed the warning value corresponding to the war type logs, warning information is automatically sent to a related personnel terminal; or if the number of error type logs generated in the life cycle of the business page 1 from 08:45 to 09:00 is judged to exceed the warning value corresponding to the error type logs, and if the number of the error type logs generated in the life cycle of the business page 1 from 08:45 to 09:00 is judged to exceed the warning value corresponding to the error type logs, warning information is automatically sent to a related personnel terminal so that related personnel can process and analyze the warning information.
For another example, for the service page 1, if it is determined that the number of the war type logs generated in the life cycle of the service page 1 from 14:45 to 15:00 exceeds the warning value corresponding to the war type logs, and if the number of the war type logs generated in the life cycle of the service page 1 from 14:45 to 15:00 exceeds the warning value corresponding to the war type logs, automatically sending warning information to the related personnel terminal; or if the number of error type logs generated in the life cycle of the service page 1 from 14:45 to 15:00 is judged to exceed the warning value corresponding to the error type logs, and if the number of the error type logs generated in the life cycle of the service page 1 from 14:45 to 15:00 exceeds the warning value corresponding to the error type logs, automatically sending warning information to a related personnel terminal so that related personnel can process and analyze the warning information.
For another example, for the service page 2, if it is determined that the number of the war type logs generated in the life cycle of the service page 2 from 09:45 to 10:00 exceeds the warning value corresponding to the war type logs, and if the number of the war type logs generated in the life cycle of the service page 2 from 09:45 to 10:00 exceeds the warning value corresponding to the war type logs, automatically sending warning information to the related personnel terminal; or if the number of error type logs generated in the life cycle of 09:45-10:00 of the service page 2 is judged to exceed the warning value corresponding to the error type logs, and if the number of the error type logs generated in the life cycle of 09:45-10:00 of the service page 2 exceeds the warning value corresponding to the error type logs, automatically sending warning information to a related personnel terminal so that related personnel can process and analyze the warning information.
It should be noted that the warning values of the same or different monitoring log types of different service pages may be different, and the warning values set for different life cycles of the same service page may also be different, and may be specifically configured according to actual situations, which is not limited in the present invention.
In the embodiment, the identification identifier ID is mainly set for a service page of the terminal device to identify different service pages, and then a warning value, that is, a warning threshold value, of a war type log or an error type log corresponding to the identification identifier ID can be set.
In addition, in an embodiment, if the current service page is closed, the memory corresponding to the service page is released, so as to avoid occupying too much memory space of the terminal device.
In an embodiment, as shown in fig. 5, in step S50, that is, uploading all monitoring logs in the current local array to the log processing server includes the following steps:
s51: and summarizing and counting all monitoring logs in the current local array according to a preset grouping method and grouping.
For example, all monitoring logs in the current local array may be summarized, counted and grouped according to monitoring log levels. It can be understood that in the field of monitoring log processing, the monitoring log level refers to types of logs such as info, war, error and the like, and because all information contained in different logs is different, the monitoring logs need to be summarized, counted and grouped so as to be divided into corresponding persons for processing in the following process, and the processing efficiency is optimized. The summary statistics can be directly summarized according to different monitoring log grade types so as to determine logs of different monitoring log grades. For example, the summary statistics may be performed according to different conditions such as the generation time of the monitoring log and the level of the monitoring log. Can be selected according to actual conditions.
S52: and dividing the counted and grouped monitoring logs into different processing personnel to perform association marking on the groups, wherein the association marking is used for marking the corresponding relation between the groups and the processing personnel.
S53: and uploading the monitoring logs in the grouping to the log processing server according to the association mark.
In the embodiment, through marking and distributing, different processing personnel at all levels can process corresponding problems, the monitoring logs are fully utilized, solutions are reasonably distributed, and the speed of solving the log reflection errors can be increased.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment, a monitoring log processing apparatus is provided, and the monitoring log processing apparatus corresponds to the monitoring log processing methods in the above embodiments one to one. As shown in fig. 6, the monitoring log processing apparatus includes an obtaining module 101, a storing module 102, a determining module 103, a judging module 104, a deleting module 105, and a transmitting module 106. The functional modules are explained in detail as follows:
the acquisition module 101 is configured to acquire a monitoring log of a client in real time according to a preset log burying point;
a storage module 102, configured to store the monitoring log in a local array created by the client;
a determining module 103, configured to determine a log type of the monitoring log in the local array, where the log type includes a regular log;
a judging module 104, configured to circularly judge whether the number of the regular logs in the local array exceeds a preset number according to the log type of the monitoring log in the local array;
a deleting module 105, configured to delete the conventional logs of the local array if the number of the conventional logs in the local array exceeds the preset number until the number of the conventional logs in the local array is smaller than the preset number;
and the uploading module 106 is configured to upload all monitoring logs in the current local array to a log processing server when the local array stores an abnormal log.
In an embodiment, the storage module is specifically configured to:
creating an empty array as the local array to use the local array as a container for log localization;
configuring a plurality of function functions which are opened to the outside by the local array, wherein different function functions are respectively and correspondingly used as call inlets of monitoring logs of different log types;
when each function is called, converting the input-parameter array object of each function into a target array;
carrying out serialization processing on the target array corresponding to each function to obtain a serialized access object corresponding to each function;
calling the corresponding function through the serialized objects corresponding to the function functions to acquire corresponding monitoring log serialized data;
and storing the monitoring log serialized data into a local array.
In one embodiment, the apparatus further comprises a configuration module, a setting module, and a sending module:
the configuration module is used for configuring a unique identification number for the life cycle of the service page which generates the monitoring log at present;
the setting module is used for setting a log warning value of a target log type corresponding to the current service page according to the identification mark number;
the judging module is further configured to judge whether the number of monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page;
and the sending module is used for automatically sending warning information to a related personnel terminal if the number of the monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page.
In one embodiment, the apparatus further comprises a release die; :
and the release module is used for releasing the memory corresponding to the service page if the current service page is closed.
In an embodiment, the upload module is specifically configured to:
summarizing and counting all monitoring logs in the current local array according to a preset grouping method and grouping the monitoring logs;
dividing the counted and grouped monitoring logs into different processing personnel to perform association marking on the groups, wherein the association marking is used for marking the corresponding relation between the groups and the processing personnel;
and uploading the monitoring logs in the grouping to the log processing server according to the association mark.
Therefore, in the embodiment of the present invention, a monitoring log processing apparatus is provided, where an info type log is discarded in a manner of setting a local array and performing a cyclic judgment, so as to achieve a purpose of saving traffic, reduce a processing amount of a log processing service end for processing the log, save a processing resource of a back-end server, and reduce an input of computational power. In addition, when encountering abnormal logs such as war logs or error logs, the abnormal logs are uploaded to the log processing server, so that the flow of the terminal equipment and the processing resources of the log processing server can be reduced, the overall log monitoring operation and maintenance cost is reduced, and the packet loss rate can be reduced to a certain extent due to the small transmission data volume, so that the monitoring efficiency is effectively improved.
For specific limitations of the monitoring log processing apparatus, reference may be made to the above limitations of the monitoring log processing method, which are not described herein again. The modules in the monitoring log processing device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a storage medium and an internal memory. The medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and computer programs in the storage medium to run. The network interface of the computer equipment is used for communicating with an external log processing server through network connection. The computer program is executed by a processor to implement a monitoring log processing method.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring a monitoring log of a service front-end system in real time according to a preset log buried point, and storing the monitoring log into a local array;
determining a log type of the monitoring log in the local array, wherein the log type comprises a conventional log;
circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log type of the monitoring log in the local array;
if the number of the conventional logs in the local array exceeds the preset number, deleting the conventional logs of the local array until the number of the conventional logs in the local array is smaller than the preset number;
and when the local array stores abnormal logs, uploading all monitoring logs in the current local array to a log processing server.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a monitoring log of a service front-end system in real time according to a preset log buried point, and storing the monitoring log into a local array;
determining a log type of the monitoring log in the local array, wherein the log type comprises a conventional log;
circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log type of the monitoring log in the local array;
if the number of the conventional logs in the local array exceeds the preset number, deleting the conventional logs of the local array until the number of the conventional logs in the local array is smaller than the preset number;
and when the local array stores abnormal logs, uploading all monitoring logs in the current local array to a log processing server.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (10)

1. A monitoring log processing method is characterized by comprising the following steps:
acquiring a monitoring log of a service front-end system in real time according to a preset log buried point, and storing the monitoring log into a local array;
determining a log type of the monitoring log in the local array, wherein the log type comprises a conventional log;
circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log type of the monitoring log in the local array;
if the number of the conventional logs in the local array exceeds the preset number, deleting the conventional logs of the local array until the number of the conventional logs in the local array is smaller than the preset number;
and when the local array stores abnormal logs, uploading all monitoring logs in the current local array to a log processing server.
2. The monitoring log processing method of claim 1, wherein the storing the monitoring log into a local array comprises:
creating an empty array as the local array to use the local array as a container for log localization;
configuring a plurality of function functions which are opened to the outside by the local array, wherein different function functions are respectively and correspondingly used as call inlets of monitoring logs of different log types;
when each function is called, converting the input-parameter array object of each function into a target array;
carrying out serialization processing on the target array corresponding to each function to obtain a serialized access object corresponding to each function;
calling the corresponding function through the serialized objects corresponding to the function functions to acquire corresponding monitoring log serialized data;
and storing the monitoring log serialized data into a local array.
3. The monitoring log processing method according to claim 1, wherein after the monitoring log of the client is obtained in real time according to the preset log burying point, the method further comprises:
configuring a unique identification number for the life cycle of a service page which generates the monitoring log at present;
setting a log warning value of a target log type corresponding to the current service page according to the identification mark number;
judging whether the number of monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page;
and if the number of the monitoring logs corresponding to the generated target log type exceeds the log warning value in the life cycle of the current service page, automatically sending warning information to a related personnel terminal.
4. The monitoring log processing method of claim 3, wherein the method further comprises:
and if the current service page is closed, releasing the memory corresponding to the service page.
5. The monitoring log processing method according to any one of claims 1 to 4, wherein the uploading all monitoring logs in the current local array to a log processing server includes:
summarizing and counting all monitoring logs in the current local array according to a preset grouping method and grouping the monitoring logs;
dividing the counted and grouped monitoring logs into different processing personnel to perform association marking on the groups, wherein the association marking is used for marking the corresponding relation between the groups and the processing personnel;
and uploading the monitoring logs in the grouping to the log processing server according to the association mark.
6. The monitor log processing method according to any one of claims 1 to 4, wherein the normal type log includes an info type log, and the exception log includes a war type log and an error type log.
7. A monitoring log processing apparatus, comprising:
the acquisition module is used for acquiring the monitoring log of the client in real time according to the preset log burying point;
the storage module is used for storing the monitoring log into a local array created by the client;
the determining module is used for determining the log type of the monitoring log in the local array, wherein the log type comprises a conventional log;
the judging module is used for circularly judging whether the number of the conventional logs in the local array exceeds a preset number or not according to the log types of the monitoring logs in the local array;
the deleting module is used for deleting the conventional logs of the local array if the number of the conventional logs in the local array exceeds the preset number until the number of the conventional logs in the local array is smaller than the preset number;
and the uploading module is used for uploading all monitoring logs in the current local array to the log processing server side when the local array stores abnormal logs.
8. The monitoring log processing apparatus of claim 7, wherein the storage module is specifically configured to:
creating an empty array as the local array to use the local array as a container for log localization;
configuring a plurality of function functions which are opened to the outside by the local array, wherein different function functions are respectively and correspondingly used as call inlets of monitoring logs of different log types;
when each function is called, converting the input-parameter array object of each function into a target array;
carrying out serialization processing on the target array corresponding to each function to obtain a serialized access object corresponding to each function;
calling the corresponding function through the serialized objects corresponding to the function functions to acquire corresponding monitoring log serialized data;
and storing the monitoring log serialized data into a local array.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the steps of the monitoring log processing method according to any one of claims 1 to 6 are implemented when the processor executes the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the monitoring log processing method according to any one of claims 1 to 6.
CN202011489454.1A 2020-12-16 2020-12-16 Monitoring log processing method, device, equipment and storage medium Pending CN112527600A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011489454.1A CN112527600A (en) 2020-12-16 2020-12-16 Monitoring log processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011489454.1A CN112527600A (en) 2020-12-16 2020-12-16 Monitoring log processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112527600A true CN112527600A (en) 2021-03-19

Family

ID=75000798

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011489454.1A Pending CN112527600A (en) 2020-12-16 2020-12-16 Monitoring log processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112527600A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113391990A (en) * 2021-06-30 2021-09-14 未鲲(上海)科技服务有限公司 System log monitoring method, device, equipment and storage medium
CN116582406A (en) * 2023-07-13 2023-08-11 支付宝(杭州)信息技术有限公司 Method and device for uploading monitoring data

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113391990A (en) * 2021-06-30 2021-09-14 未鲲(上海)科技服务有限公司 System log monitoring method, device, equipment and storage medium
CN116582406A (en) * 2023-07-13 2023-08-11 支付宝(杭州)信息技术有限公司 Method and device for uploading monitoring data
CN116582406B (en) * 2023-07-13 2023-09-26 支付宝(杭州)信息技术有限公司 Method and device for uploading monitoring data

Similar Documents

Publication Publication Date Title
CN110377569B (en) Log monitoring method, device, computer equipment and storage medium
CN109144700B (en) Method and device for determining timeout duration, server and data processing method
CN109284269A (en) Abnormal log analysis method, device, storage medium and server
CN112910945A (en) Request link tracking method and service request processing method
CN110232010A (en) A kind of alarm method, alarm server and monitoring server
CN110851159B (en) Business rule updating method and device, computer equipment and storage medium
CN110647471B (en) Interface test case generation method, electronic device and storage medium
CN109542718B (en) Service call monitoring method and device, storage medium and server
CN110399178B (en) Third-party service calling method and device, electronic equipment and storage medium
CN112527600A (en) Monitoring log processing method, device, equipment and storage medium
CN111651595A (en) Abnormal log processing method and device
CN103176892A (en) Page monitoring method and system
CN114363144B (en) Fault information association reporting method and related equipment for distributed system
CN108509313A (en) A kind of business monitoring method, platform and storage medium
CN112232881A (en) Data detection method and device, electronic equipment and storage medium
CN111881014A (en) System test method, device, storage medium and electronic equipment
CN111669295A (en) Service management method and device
CN114936136A (en) Business process monitoring method and device, computer equipment and storage medium
CN107679423A (en) Partition integrity inspection method and device
CN109559121A (en) Transaction path calls exception analysis method, device, equipment and readable storage medium storing program for executing
CN113360210A (en) Data reconciliation method and device, computer equipment and storage medium
CN108111328B (en) Exception handling method and device
CN112115041A (en) Dynamic point burying method and device for application program, storage medium and computer equipment
CN116431344A (en) Configuration method and device of financial service resource degradation strategy and electronic equipment
CN115454673A (en) Data collection method, device and medium for baseboard management controller

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination