CN109656784A - A kind of log processing method and device - Google Patents
A kind of log processing method and device Download PDFInfo
- Publication number
- CN109656784A CN109656784A CN201811595489.6A CN201811595489A CN109656784A CN 109656784 A CN109656784 A CN 109656784A CN 201811595489 A CN201811595489 A CN 201811595489A CN 109656784 A CN109656784 A CN 109656784A
- Authority
- CN
- China
- Prior art keywords
- log
- application program
- amount
- monitoring period
- default
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000012544 monitoring process Methods 0.000 claims abstract description 121
- 230000005764 inhibitory process Effects 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims description 35
- 238000010586 diagram Methods 0.000 description 10
- 230000000875 corresponding effect Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000012546 transfer Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
Abstract
The present invention provides a kind of log processing method and device, this method comprises: the log amount that monitoring application program generates within the default monitoring period；If the log amount that the application program generates within the default monitoring period is more than preset threshold, log collection inhibition is carried out, to reduce the log amount of acquisition.Using the embodiment of the present invention system stability can be improved to avoid log memory space inadequate caused by a large amount of logs is generated in the application program short time.
Description
Technical field
The present invention relates to network communication technology field more particularly to a kind of log processing methods and device.
Background technique
Diagnostic log (abbreviation log herein) is to assist analysis system failure, the maintainable basic means of safeguards system.
Log is generally held in the media such as persistent file, persistent database.When uninstalling system, it will be deleted whole logs.
Due to running with systems stay, log can continue to increase.Therefore, it is necessary to periodically clear up saved log.If can not
Cleaning log in time, then it is possible that the problem of Insufficient disk space leads to system crash.If timing clear up during, i.e., on
After cleanup action, before cleanup action starting next time, there are a large amount of logs in the short time, then may also lead to disk sky
Between it is insufficient, and then lead to system crash.
Summary of the invention
The present invention provides a kind of log processing method and device, to solve in log processing scheme application program in the short time
It is interior to lead to the problem of a large amount of logs and lead to log memory space inadequate.
According to a first aspect of the embodiments of the present invention, a kind of log processing method is provided, comprising:
The log amount that monitoring application program generates within the default monitoring period；
If the log amount that the application program generates within the default monitoring period is more than preset threshold, log is carried out
Acquisition inhibits, to reduce the log amount of acquisition.
According to a first aspect of the embodiments of the present invention, a kind of log processing method is provided, is applied in log processing system
Any log monitor, include multiple containers in the log processing system, the log monitoring be deployed in each container
Device, which comprises
Monitor the log amount that the application program in itself said container generates within the default monitoring period；
If the log amount that the application program generates within the default monitoring period is more than preset threshold, log is carried out
Acquisition inhibits, to reduce the log amount of acquisition.
According to a third aspect of the embodiments of the present invention, a kind of log processing device is provided, comprising:
Monitoring unit, the log amount generated within the default monitoring period for monitoring application program；
Judging unit, whether the log amount for judging that the application program generates within the default monitoring period is more than default
Threshold value；
Processing unit, if being more than default threshold for the log amount that the application program generates within the default monitoring period
Value, then carry out log collection inhibition, to reduce the log amount of acquisition.
According to a fourth aspect of the embodiments of the present invention, a kind of log processing device is provided, is applied in log processing system
Any log monitor, include multiple containers in the log processing system, the log monitoring be deployed in each container
Device, described device include:
Monitoring unit produces within the default monitoring period for monitoring the application program in the log processor said container
Raw log amount；
Judging unit, whether the log amount for judging that the application program generates within the default monitoring period is more than default
Threshold value；
Processing unit, if being more than default threshold for the log amount that the application program generates within the default monitoring period
Value, then carry out log collection inhibition, to reduce the log amount of acquisition.
Using technical solution disclosed by the invention, the log generated within the default monitoring period by monitoring application program
Amount, and when the log amount that application program generates within the default monitoring period is more than preset threshold, log collection inhibition is carried out, with
The log amount for reducing acquisition, controls log amount in the log collection stage, avoids in the application program short time and generates
Log memory space inadequate caused by a large amount of logs, improves system stability.
Detailed description of the invention
Fig. 1 is a kind of flow diagram of log processing method provided in an embodiment of the present invention；
Fig. 2 is a kind of configuration diagram of concrete application scene provided in an embodiment of the present invention；
Fig. 3 is the process signal of the log processing method under application scenarios shown in a kind of Fig. 2 provided in an embodiment of the present invention
Figure；
Fig. 4 is the configuration diagram of another concrete application scene provided in an embodiment of the present invention；
Fig. 5 is the process signal of the log processing method under application scenarios shown in a kind of Fig. 4 provided in an embodiment of the present invention
Figure；
Fig. 6 is a kind of structural schematic diagram of log processing device provided in an embodiment of the present invention；
Fig. 7 is a kind of structural schematic diagram of log processing device provided in an embodiment of the present invention.
Specific embodiment
Technical solution in embodiment in order to enable those skilled in the art to better understand the present invention, and make of the invention real
The above objects, features, and advantages for applying example can be more obvious and easy to understand, with reference to the accompanying drawing to technical side in the embodiment of the present invention
Case is described in further detail.
It referring to Figure 1, is a kind of flow diagram of log processing method provided in an embodiment of the present invention, as shown in Figure 1,
The log processing method may include:
The log amount that step 101, monitoring application program generate within the default monitoring period.
In the embodiment of the present invention, lead to log memory space in order to avoid application program generates a large amount of logs in a short time
Deficiency, the log amount that can be generated to application program are monitored, and when the log amount that application program generates is excessive, are reduced
The log amount for needing to store.
Correspondingly, in embodiments of the present invention, can monitor application program (can be according to actual field in the default monitoring period
Scape setting, such as 1s, 2s) in generate log amount.
In one example, log amount can include but is not limited to log byte number or/and log message number.
In this example, it is contemplated that the log that application program generates may include the log or/and message class of file type
The log of type.
Log for file type can determine log amount by statistical log byte number；For the day of type of message
Will, on the one hand can be with statistical log byte number, on the other hand, can be with accounting message quantity (referred to herein as log message number).
If the log amount that step 102, application program generate within the default monitoring period is more than preset threshold, log is carried out
Acquisition inhibits, to reduce the log amount of acquisition.
In the embodiment of the present invention, lead to log memory space in order to avoid application program generates a large amount of logs in a short time
Deficiency can control log amount in the log collection stage.
Correspondingly, in embodiments of the present invention, if the log amount that application program generates within the default monitoring period is more than pre-
If threshold value (can be set) according to actual scene, then log collection inhibition can be carried out, to reduce the log amount of acquisition, in turn,
The log amount for needing to store can be reduced.
In one example, the log amount that above-mentioned application program generates within the default monitoring period is more than preset threshold, can
To include:
The log byte number that application program generates within the default monitoring period is more than predetermined word joint number threshold value.
In this example, it can be united according to the log for the file type that application program generates or/and the log of type of message
The log byte number that application program generates is counted, and determines whether is log byte number that application program generates within the default monitoring period
More than predetermined word joint number threshold value.
Wherein, if the log byte number that application program generates within the default monitoring period is more than predetermined word joint number threshold value,
Log collection inhibition can be carried out, to reduce the log amount of acquisition.
In another example, the log amount that above-mentioned application program generates within the default monitoring period is more than preset threshold,
May include:
The log message number that application program generates within the default monitoring period is more than default message number threshold value.
In this example, the log that can be generated according to the log statistic application program for the type of message that application program generates
Message number, and whether the log message number for determining that application program generates within the default monitoring period is more than default message number threshold value.
Wherein, if the log message number that application program generates within the default monitoring period is more than default message number threshold value,
Log collection inhibition can be carried out, to reduce the log amount of acquisition.
It should be noted that in embodiments of the present invention, it can also be simultaneously to application program within the default monitoring period
The log byte number and log message number of generation are monitored, and when application program is in the default log word for monitoring and generating in the period
Joint number is more than predetermined word joint number threshold value, or, carrying out log collection inhibition when log message number is more than default message number threshold value；Or
Person can be more than predetermined word joint number threshold value when the log byte number that application program generates within the default monitoring period, and log report
When literary number is more than default message number threshold value, log collection inhibition is carried out, specific implementation does not repeat them here herein.
In addition, in embodiments of the present invention, if the log amount that application program generates within the default monitoring period be less than it is pre-
If threshold value, then log monitor not will do it log collection inhibition, after the log that application program generates is handled by log processor,
It stores to log memory.
Wherein, log processor can be one operated in the container run on physical server or physical server
Process.Log processor is handled the log received according to log configuration, such as stores the log received to specified
Log memory.
For example, IP address, file directory, the socket service of syslog server can be configured in log configuration
The corresponding relationship of port numbers etc. and specified log rank or Log Types and log memory is held, thus, log processor
The log received can be stored to corresponding log memory according to log configuration and log rank or Log Types.
For example, it is assumed that the log needs that log configuration middle finger determines Log Types A and Log Types B are stored to syslog and are serviced
Device, and it is configured with the IP address of syslog server, when log processor receives the log of Log Types A and Log Types B,
Syslog server can be sent it to according to the IP address of syslog server.
In addition, log processor can be configured with log Prune Policies.For storing to local log, by log
Reason device does timing and clears up, will etc. former days deletion, in order to avoid local log amount increases without limitation, specific implementation is not done superfluous herein
It states.
Further, in embodiments of the present invention, it is contemplated that the importance of log is usually positively correlated with the rank of log, i.e.,
The importance of the higher log of usual rank can be higher, therefore, when log memory space inadequate, should guarantee as far as possible high level
Log can be stored.
Correspondingly, the present invention in one embodiment, above-mentioned carry out log collection inhibition may include:
Improve the log rank that currently comes into force；
Abandon the log that the rank that application program generates is lower than the log rank that currently comes into force.
In this embodiment, in order to reduce the log amount of acquisition, can according to the rank of log to log collected into
Row inhibits.
Wherein, come into force log rank for being filtered to log, i.e. each log can have a rank, log
After monitor sets the log rank currently to come into force, rank is lower than the day come into force before deserving in the log that application program generates
The log of will rank will be abandoned by log monitor, will not be sent to log processor again.The work for the log rank that specifically comes into force
It is illustrated in postpose embodiment.
Correspondingly, in this embodiment, if the log amount that application program generates within the default monitoring period is more than default threshold
The log rank that currently comes into force then can be improved in value, and abandons the rank of application program generation lower than the log rank that currently comes into force
Log.
Wherein, the initial value for the log rank that currently comes into force is the log rank (this paper configured in advance for each application program
In be properly termed as configuration log rank), i.e., application program generate log rank be not less than the configuration log rank.
Further, in this embodiment, it is contemplated that log is to assist analysis system failure, the maintainable base of safeguards system
Therefore this means when log memory space abundance, the log that application program generates is carried out storing to be better achieved setting
Standby maintenance.
Correspondingly, after above-mentioned raising currently comes into force log rank, can also include:
If the log amount that application program generates within the default monitoring period is less than preset threshold, the current effective date is reduced
Will rank.
Specifically, after raising comes into force log rank, if the log amount that application program generates within the default monitoring period
It is less than preset threshold, can reduce and currently come into force log rank, it can be preferably real to guarantee that more logs are stored to
Existing plant maintenance.
In one example, the log amount that above-mentioned application program generates within the default monitoring period is less than preset threshold,
May include:
The log amount that application program generates within the default monitoring period is less than default log byte number threshold value；Or/and
The log message number that application program generates within the default monitoring period is less than default message number threshold value.
It should be noted that in embodiments of the present invention, since the lowest level of the log of application program generation is configuration
Therefore log rank can also be carried out when needing to reduce and currently coming into force log rank by lowest level of configuration log rank
Rank reduces, i.e., the lowest level that log can reduce that currently comes into force is configuration log rank.
But application awareness arrives, in embodiments of the present invention, if configuration log rank is not the minimum journal stage that system is supported
Not, then it needs to reduce when currently coming into force log rank, the currently log that comes into force can also be decreased below into configuration log rank (i.e.
The lowest level that log can reduce that currently comes into force can be lower than configuration log rank), due to the log that currently comes into force after reducing
Rank be lower than configuration log rank, i.e. the rank of log caused by application program can be higher than and currently come into force log rank,
Therefore, the log that application program generates can be acquired all.
Further, in embodiments of the present invention, in order to avoid the excessive log that single application program generates influences other
The acquisition and storage of the log of application program can carry out log for each application program respectively and generate monitoring and acquisition control.
Correspondingly, the present invention in one embodiment, the day that above-mentioned monitoring application program generates in predetermined period
Will amount may include:
The log amount that each application program generates within the default monitoring period is monitored respectively；
If the log amount that above-mentioned application program generates within the default monitoring period is more than preset threshold, log collection is carried out
Inhibit, may include:
For any application program, if the log amount that the application program generates within the default monitoring period is more than default threshold
Value then carries out log collection threshold value for the application program.
In this embodiment it is possible to which each application program is monitored in the log amount that the default monitoring period generates respectively.
For example, each application deployment log monitor can be respectively corresponded, answered by the monitoring of each log monitor is corresponding
The log amount generated with program in the default monitoring period.
For any application program, if the log amount that the application program generates within the default monitoring period is more than default threshold
Value then carries out log collection inhibition for the application program.
In one example, it can be run by running multiple containers in log processing system, and in each container
The mode of log monitor realizes monitoring of the log monitor to the log amount of specified application process.
In this example, log monitor can monitor the application program in itself said container within the default monitoring period
The log amount of generation, and the log amount generated within the default monitoring period according to the application program of itself said container whether be more than
Preset threshold determines whether to carry out log collection inhibition.
Optionally, single application program can be run in each container, thus, the log monitor disposed in each container is equal
The log amount that the single application program run in itself said container generates within the default monitoring period need to be monitored, is avoided single
The excessive log that application program generates influences the acquisition and storage of the log of other applications.
It should be noted that in embodiments of the present invention, when being directed to the progress log amount monitoring of each application program respectively, respectively
The application program corresponding default monitoring period may be the same or different；Similarly, the corresponding preset threshold of each application program can
With identical, can also be different, in this regard, the present invention is without limitation.
In order to make those skilled in the art more fully understand technical solution provided in an embodiment of the present invention, below with reference to specific
Application scenarios are illustrated technical solution provided in an embodiment of the present invention.
Embodiment one
Fig. 2 is referred to, is a kind of configuration diagram of concrete application scene provided in an embodiment of the present invention, as shown in Fig. 2,
In this embodiment it is assumed that operation has application program 211~215 in system, log collector 221~223 is respectively used to acquire
The log that the log that application program 211~213 generates, application program 214 and application program 215 generate is type of message log,
Pass through TCP (Transmission Control Protocol, transmission control protocol)/UDP (User Datagram
Protocol, User Datagram Protocol) or HTTP (HyperText Transfer Protocol, hypertext transfer protocol)/
The side HTTPs (Hyper Text Transfer Protocol over Secure Socket Layer, safe version http protocol)
Formula is transferred to log processor 230；Log processor 230 after handling log for saving to log memory 240.
In this embodiment, in order to realize log collection control, log monitor 250 can be disposed in systems, is used for
The log amount that each application program generates in default monitoring period (by taking 1s as an example) is monitored, and according to the generation per second of each application program
Log amount carry out log collection control.
Based on application scenarios shown in Fig. 2, log processing process provided in an embodiment of the present invention can be as shown in figure 3, it can
With the following steps are included:
Step 301, log monitor 250 monitor the log that application program 211~215 generates.
Step 302, the generation per second of 250 statistics application program of log monitor 211~215 log byte number.If being more than
Predetermined word joint number threshold value, then go to step 305；Otherwise, step 303 is gone to.
Step 303, the generation per second of 250 statistics application program of log monitor 211~215 log message number.If being more than
Default message number threshold value, then go to step 305；Otherwise, step 304 is gone to.
It should be noted that in this embodiment, between the operation recorded in step 302 and step 303 and there is no must
Right sequential relationship, it can the operation recorded in step 302 is first carried out, it is rear to execute the operation recorded in step 303；It can also be with
The operation recorded in step 303 is first carried out, it is rear to execute the operation recorded in step 302；Step 302 and step can also concurrently be executed
The operation recorded in rapid 303.
In this embodiment, log monitor 250 may include counter module, rate limiter block and rank adjusting device
Module；Wherein:
Counter module on the one hand can be with the log byte number of the generation per second of statistics application program 211~215, another party
Face, can be with the log message number of the generation per second of statistics application program 211~215.
Wherein, log monitor 250 often receives a daily record data (including file type log or type of message day
Will), counter module can update the day in the counted monitoring period according to the byte number for the log for including in daily record data
Will byte number；Log monitor 250 often receives a type of message daily record data, and counter module is all by the monitoring counted
Log message number in phase adds 1.
Rate limiter block may determine that whether the log byte number of the generation per second of application program 211~215 is more than predetermined word
Whether joint number threshold value and the message log number of generation per second are more than default message number threshold value.
If the log byte number of the generation per second of application program 211~215 is more than predetermined word joint number threshold value, or/and, production per second
Raw message log number is more than default message number threshold value, then on the one hand rate limiter block can be improved with trigger level regulator module
Currently come into force log rank；On the other hand, the rank of the generation of application program 211~215 can be abandoned lower than the log that currently comes into force
The log of rank.
If the log byte number of the generation per second of application program 211~215 is less than predetermined word joint number threshold value, and generation per second
Message log number be less than default message number threshold value, then rate limiter block will can currently be come into force with trigger level regulator module
Log rank reverts to configuration log rank.
Rank adjusting device module record current come into force log rank and configuration log rank, for according to rate limiter block
The current log rank that comes into force of instruction setting.
Wherein, in this embodiment, system log rank follows syslog (system log) protocol specification, value range 0
To 7.0 indicates highest level, and 7 indicate lowest level.Corresponding rank is successively from 0 to 7 are as follows: system is unavailable, must be into
Row immediately treats, serious error, mistake, warning, normal information, but more important, normal information, Debugging message.
Whether step 304, log monitor 250 the log rank that judges currently to come into force are higher than configuration log rank；If being higher than,
The log rank that currently comes into force then is reverted into configuration log rank, and goes to step 301；Otherwise, step 301 is gone to.
Step 305 improves the log rank that currently comes into force, and abandons the rank of the generation of application program 211~215 lower than current
The log for the log rank that comes into force, goes to step 301.
Embodiment two
Fig. 4 is referred to, is a kind of configuration diagram of concrete application scene provided in an embodiment of the present invention, as shown in figure 4,
In this embodiment it is assumed that operation has application program 411~415 in system, log collector 421~423 is respectively used to acquire
The log that the log that application program 411~413 generates, application program 414 and application program 415 generate is type of message log,
It is transferred to by TCP/UDP or HTTP (HyperText Transfer Protocol, hypertext transfer protocol)/HTTPs mode
Log processor 430；Log processor 430 after handling log for saving to log memory 440.
In this embodiment, in order to realize log collection control, log monitor can be disposed in systems, for monitoring
The log amount that each application program generates in default monitoring period (by taking 1s as an example), and according to the day of each application program generation per second
Will amount carries out log collection control.
In addition, can be directed to respectively in order to avoid the excessive log that single application program generates influences other applications
In the same container, difference is held for each application deployment log monitor, log monitor and corresponding application deployment
Device can be deployed in different hosts respectively, can also be deployed in same host.
As shown in figure 4, application program 411~415 is respectively deployed in container 451~455,461~465 points of log monitor
The log amount of the generation per second of application program 411~415 Yong Yu not be monitored, and according to the log of corresponding application program generation per second
Amount carries out log collection inhibition.
Application scenarios based on shown in Fig. 4, log processing process provided in an embodiment of the present invention can be as shown in figure 5, it can
To include the following steps (by taking application program 411 as an example):
Step 501, log monitor 461 monitor the log that application program 411 generates.
Step 502, the generation per second of 461 statistics application program of log monitor 411 log byte number.If being more than predetermined word
Joint number threshold value, then go to step 505；Otherwise, step 503 is gone to.
Step 503, the generation per second of 461 statistics application program of log monitor 411 log message number.If being more than default report
Literary number threshold value, then go to step 505；Otherwise, step 504 is gone to.
Whether step 504, log monitor 461 the log rank that judges currently to come into force are higher than configuration log rank；If being higher than,
The log rank that currently comes into force then is reverted into configuration log rank, and goes to step 501；Otherwise, step 501 is gone to.
The rank that step 505 improves the current log rank that comes into force, and abandons the generation of application program 411, which is lower than, currently to come into force
The other log of journal stage, goes to step 501.
Through above description as can be seen that in technical solution provided in an embodiment of the present invention, by monitoring application program
The log amount generated within the default monitoring period, and when the log amount that application program generates within the default monitoring period is more than default
When threshold value, log collection inhibition is carried out, to reduce the log amount of acquisition, log amount is controlled in the log collection stage,
It avoids and generates log memory space inadequate caused by a large amount of logs in the application program short time, improve system stability.
Fig. 6 is referred to, is a kind of structural schematic diagram of log processing device provided in an embodiment of the present invention, as shown in fig. 6,
The apparatus may include:
Monitoring unit 610, the log amount generated within the default monitoring period for monitoring application program；
Judging unit 620, for judge log amount that the application program generates within the default monitoring period whether be more than
Preset threshold；
Processing unit 630, if the log amount for the application program to generate within the default monitoring period is more than pre-
If threshold value, then log collection inhibition is carried out, to reduce the log amount of acquisition.
In an alternative embodiment, the log amount includes log byte number or/and log message number；
The judging unit 620, if the log generated within the default monitoring period specifically for the application program
Byte number is more than predetermined word joint number threshold value；Or/and the log message that the application program generates within the default monitoring period
Number is more than default message number threshold value, and the log amount for determining that the application program generates within the default monitoring period is more than default
Threshold value.
In an alternative embodiment, the processing unit 630, specifically for improving the log rank that currently comes into force；Described in discarding
The rank that application program generates is lower than the log for the log rank that currently comes into force.
In an alternative embodiment, the processing unit 630 is also used to after improving the log rank that currently comes into force, if institute
When stating application program the log amount that generates being less than the preset threshold within the default monitoring period, the current effective date is reduced
Will rank.
In an alternative embodiment, the monitoring unit 610, specifically for monitoring each application program respectively in default monitoring week
The log amount generated in phase；
The processing unit 630 is specifically used for for any application program, if the application program is within the default monitoring period
The log amount of generation is more than preset threshold, then carries out log collection inhibition for the application program.
Fig. 7 is referred to, is a kind of structural schematic diagram of log processing device provided in an embodiment of the present invention, wherein the day
Will processing unit can be applied to any log monitor in log processing system, may include more in the log processing system
A container is deployed with log monitor in each container, as shown in fig. 7, the apparatus may include:
Monitoring unit 710, for monitoring the application program in the log processor said container in the default monitoring period
The log amount of interior generation；
Judging unit 720, for judge log amount that the application program generates within the default monitoring period whether be more than
Preset threshold；
Processing unit 730, if the log amount for the application program to generate within the default monitoring period is more than pre-
If threshold value, then log collection inhibition is carried out, to reduce the log amount of acquisition.
In an alternative embodiment, single application program is run in each container.
The function of each unit and the realization process of effect are specifically detailed in the above method and correspond to step in above-mentioned apparatus
Realization process, details are not described herein.
For device embodiment, since it corresponds essentially to embodiment of the method, so related place is referring to method reality
Apply the part explanation of example.The apparatus embodiments described above are merely exemplary, wherein described be used as separation unit
The unit of explanation may or may not be physically separated, and component shown as a unit can be or can also be with
It is not physical unit, it can it is in one place, or may be distributed over multiple network units.It can be according to actual
The purpose for needing to select some or all of the modules therein to realize the present invention program.Those of ordinary skill in the art are not paying
Out in the case where creative work, it can understand and implement.
As seen from the above-described embodiment, the log amount generated within the default monitoring period by monitoring application program, and work as and answer
When being more than preset threshold with the log amount that program generates within the default monitoring period, log collection inhibition is carried out, to reduce acquisition
Log amount, log amount is controlled in the log collection stage, avoids and generates a large amount of logs in the application program short time
Caused log memory space inadequate, improves system stability.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to of the invention its
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the invention, these modifications, purposes or
Person's adaptive change follows general principle of the invention and including the undocumented common knowledge in the art of the present invention
Or conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is limited only by the attached claims.
Claims (14)
1. a kind of log processing method characterized by comprising
The log amount that monitoring application program generates within the default monitoring period；
If the log amount that the application program generates within the default monitoring period is more than preset threshold, log collection is carried out
Inhibit, to reduce the log amount of acquisition.
2. the method according to claim 1, wherein the log amount includes log byte number or/and log report
Literary number；
The log amount that the application program generates within the default monitoring period is more than preset threshold, comprising:
The log byte number that the application program generates within the default monitoring period is more than predetermined word joint number threshold value；Or/and
The log message number that the application program generates within the default monitoring period is more than default message number threshold value.
3. the method according to claim 1, wherein the progress log collection inhibition, comprising:
Improve the log rank that currently comes into force；
Abandon the log that the rank that the application program generates is lower than the log rank that currently comes into force.
4. according to the method described in claim 3, it is characterized in that, the raising currently comes into force after log rank, further includes:
If the log amount that the application program generates within the default monitoring period is less than the preset threshold, reduction is worked as
Before the journal stage that comes into force it is other.
5. the method according to claim 1, wherein the monitoring application program generates within the default monitoring period
Log amount, comprising:
The log amount that each application program generates within the default monitoring period is monitored respectively；
If the log amount that the application program generates within the default monitoring period is more than preset threshold, log is carried out
Acquisition inhibits, comprising:
For any application program, if the log amount that the application program generates within the default monitoring period is more than preset threshold,
Log collection inhibition is carried out for the application program.
6. a kind of log processing method, which is characterized in that applied to any log monitor in log processing system, the day
Include multiple containers in will processing system, be deployed with the log monitor in each container, which comprises
Monitor the log amount that the application program in itself said container generates within the default monitoring period；
If the log amount that the application program generates within the default monitoring period is more than preset threshold, log collection is carried out
Inhibit, to reduce the log amount of acquisition.
7. according to the method described in claim 6, it is characterized in that, running single application program in each container.
8. a kind of log processing device characterized by comprising
Monitoring unit, the log amount generated within the default monitoring period for monitoring application program；
Judging unit, whether the log amount for judging that the application program generates within the default monitoring period is more than default threshold
Value；
Processing unit, if the log amount for the application program to generate within the default monitoring period is more than preset threshold,
Log collection inhibition is carried out, then to reduce the log amount of acquisition.
9. device according to claim 8, which is characterized in that the log amount includes log byte number or/and log report
Literary number；
The judging unit, if the log byte number generated within the default monitoring period specifically for the application program is super
Cross predetermined word joint number threshold value；Or/and the log message number that the application program generates within the default monitoring period is more than pre-
If message number threshold value, the log amount for determining that the application program generates within the default monitoring period is more than preset threshold.
10. device according to claim 8, which is characterized in that
The processing unit, specifically for improving the log rank that currently comes into force；The rank that the application program generates is abandoned to be lower than
The log for the log rank that currently comes into force.
11. device according to claim 10, which is characterized in that
The processing unit is also used to after improving the log rank that currently comes into force, if the application program is in the default prison
When the log amount generated in the control period is less than the preset threshold, the log rank that currently comes into force is reduced.
12. device according to claim 8, which is characterized in that
The monitoring unit, the log amount generated within the default monitoring period specifically for monitoring each application program respectively；
The processing unit is specifically used for for any application program, if what the application program generated within the default monitoring period
Log amount is more than preset threshold, then carries out log collection inhibition for the application program.
13. a kind of log processing device, which is characterized in that described applied to any log monitor in log processing system
Include multiple containers in log processing system, is deployed with the log monitor in each container, described device includes:
Monitoring unit is generated within the default monitoring period for monitoring the application program in the log processor said container
Log amount；
Judging unit, whether the log amount for judging that the application program generates within the default monitoring period is more than default threshold
Value；
Processing unit, if the log amount for the application program to generate within the default monitoring period is more than preset threshold,
Log collection inhibition is carried out, then to reduce the log amount of acquisition.
14. device according to claim 13, which is characterized in that run single application program in each container.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811595489.6A CN109656784A (en) | 2018-12-25 | 2018-12-25 | A kind of log processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811595489.6A CN109656784A (en) | 2018-12-25 | 2018-12-25 | A kind of log processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109656784A true CN109656784A (en) | 2019-04-19 |
Family
ID=66116332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811595489.6A Pending CN109656784A (en) | 2018-12-25 | 2018-12-25 | A kind of log processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109656784A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581060A (en) * | 2020-05-11 | 2020-08-25 | 金蝶软件（中国）有限公司 | Prometheus-based log alarm system and method and related equipment |
| CN111654412A (en) * | 2020-05-15 | 2020-09-11 | 华青融天(北京)软件股份有限公司 | Data acquisition and transmission method and device and electronic equipment |
| WO2021096346A1 (en) * | 2019-11-15 | 2021-05-20 | Mimos Berhad | A computer-implemented system for management of container logs and its method thereof |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838659A (en) * | 2014-02-17 | 2014-06-04 | 大唐移动通信设备有限公司 | Method and device for controlling system logs |
| CN106484593A (en) * | 2016-09-29 | 2017-03-08 | 北京小米移动软件有限公司 | Log processing method, device and electronic equipment |
| CN107622120A (en) * | 2017-09-22 | 2018-01-23 | 新华三技术有限公司 | System journal method for cleaning and device |
| CN108897663A (en) * | 2018-06-19 | 2018-11-27 | 新华三技术有限公司 | log output control method and device |
-
2018
- 2018-12-25 CN CN201811595489.6A patent/CN109656784A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838659A (en) * | 2014-02-17 | 2014-06-04 | 大唐移动通信设备有限公司 | Method and device for controlling system logs |
| CN106484593A (en) * | 2016-09-29 | 2017-03-08 | 北京小米移动软件有限公司 | Log processing method, device and electronic equipment |
| CN107622120A (en) * | 2017-09-22 | 2018-01-23 | 新华三技术有限公司 | System journal method for cleaning and device |
| CN108897663A (en) * | 2018-06-19 | 2018-11-27 | 新华三技术有限公司 | log output control method and device |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021096346A1 (en) * | 2019-11-15 | 2021-05-20 | Mimos Berhad | A computer-implemented system for management of container logs and its method thereof |
| CN111581060A (en) * | 2020-05-11 | 2020-08-25 | 金蝶软件（中国）有限公司 | Prometheus-based log alarm system and method and related equipment |
| CN111581060B (en) * | 2020-05-11 | 2024-03-12 | 金蝶软件（中国）有限公司 | Prometaus-based log alarm system, method and related equipment |
| CN111654412A (en) * | 2020-05-15 | 2020-09-11 | 华青融天(北京)软件股份有限公司 | Data acquisition and transmission method and device and electronic equipment |
| CN111654412B (en) * | 2020-05-15 | 2022-02-01 | 华青融天(北京)软件股份有限公司 | Data acquisition and transmission method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106777371B (en) | Log collection system and method | |
| CN108255641B (en) | CDP disaster recovery method based on cloud platform | |
| CN109656784A (en) | A kind of log processing method and device | |
| EP3335120B1 (en) | Method and system for resource scheduling | |
| WO2020248507A1 (en) | Container cloud-based system resource monitoring method and related device | |
| CN103023984B (en) | Terminal application server and application log filtering method thereof | |
| CN105005521A (en) | Test method and apparatus | |
| CN103761309A (en) | Operation data processing method and system | |
| CN104022902A (en) | Method and system of monitoring server cluster | |
| CN109391655A (en) | Service gray scale dissemination method, device, system and storage medium | |
| CN106470123A (en) | Log collecting method, client, server and electronic equipment | |
| WO2014053313A1 (en) | Data logs management in a multi-client architecture | |
| Xia et al. | Performance and availability modeling of ITSystems with data backup and restore | |
| CN112350854B (en) | Flow fault positioning method, device, equipment and storage medium | |
| CN101183979A (en) | Method and device of configuration backup | |
| CN104932978A (en) | System running fault self-detection and self-recovery method and system | |
| CN108304296A (en) | A kind of server monitoring method, system, equipment and computer readable storage medium | |
| CN106383668A (en) | Information storage method, storage management device and client | |
| CN111694518A (en) | Method, device and equipment for automatically migrating data after cluster expansion or contraction | |
| CN110989935A (en) | Data processing and transmitting method and device of flash system | |
| CN104618162A (en) | Management method, device and system of system docking | |
| CN107566480B (en) | User activity information acquisition method and device for mobile terminal application and storage medium | |
| CN116560889A (en) | Data link management method, device, computer equipment and storage medium | |
| JP2005018120A (en) | Method for collecting apparatus information in network management system | |
| CN208046653U (en) | A kind of electric power monitoring system network security monitoring main website plateform system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190419 |