CN102385549A - Log processing system, log processing method and log storage sub-system - Google Patents
Log processing system, log processing method and log storage sub-system Download PDFInfo
- Publication number
- CN102385549A CN102385549A CN2010102718155A CN201010271815A CN102385549A CN 102385549 A CN102385549 A CN 102385549A CN 2010102718155 A CN2010102718155 A CN 2010102718155A CN 201010271815 A CN201010271815 A CN 201010271815A CN 102385549 A CN102385549 A CN 102385549A
- Authority
- CN
- China
- Prior art keywords
- log
- subsystem
- log information
- message queue
- store
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a log processing system, a log processing method and a log storage sub-system. The log processing system comprises an application sub-system, the log storage sub-system and a log analysis sub-system, wherein the application sub-system is used for generating log information and particularly sending the log information to the log storage sub-system, the log storage sub-system is independent to the application sub-system and is used for storing the log information and particularly storing the log information in a message queue mode, and the log analysis sub-system is used for analyzing the log information and particularly reading the log information from the log storage sub-system and analyzing the log information. With the adoption of the embodiment of the invention, storage resources of the application sub-system in the log processing system are not occupied to a great extent during log processing, so that the influence on the application sub-system is reduced.
Description
Technical field
The present invention relates to data processing technique, relate in particular to the log processing technology.
Background technology
Existing a kind of log processing system is as shown in Figure 1.Log processing system adopts the distributed design mode, and the logger module in each application subsystem is log information in the application subsystem that belongs to separately all, and regularly recorded logs information is issued the log analysis subsystem.After the log analysis subsystem obtains log information, log information such as is analyzed at processing.
The inventor finds after scrutinizing above-mentioned prior art; Logger module in the application subsystem is in application subsystem during log information; Need log information be stored in the application subsystem, like this, log information just need take the storage resources of application system.If logger module recorded logs information is many more, the occupied storage resources of application subsystem will be many more so, will influence the work of application subsystem like this.
Summary of the invention
The present invention provides log processing system, log processing method and log store subsystem, in order in the log processing process, does not take the storage resources of the application subsystem in the log processing system as far as possible, thereby reduces using the influence of subsystem.
A kind of log processing system comprises:
Generate the application subsystem of log information;
Be independent of the log store subsystem of application subsystem and storing daily record information;
Analyze the log analysis subsystem of log information;
Application subsystem specifically is used for log information is issued the log store subsystem; The log store subsystem specifically is used to use the stored in form log information of message queue, and the log analysis subsystem specifically is used for reading log information and analyzing log information from the log store subsystem.
Preferably, be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
The log information that application subsystem is issued the log store subsystem carries the sign that is used to represent the log information type;
The log store subsystem specifically is used for according to said sign, with log information be stored in the represented corresponding message queue of log information type of said sign in.
Preferably, the log analysis subsystem specifically is used for according to said sign, reads the log information that needs from the log store subsystem.
Preferably, the log analysis subsystem specifically is used for timing and monitors in the message queue of log store subsystem whether log information is arranged, if having, then reads log information.
Preferably, also comprise the log management subsystem, be used to be provided for analyzing the analysis rule of log information;
The log analysis subsystem specifically is used for according to the analysis rule of log management subsystem setting log information being analyzed.
A kind of log processing method is applied to above-mentioned log processing system, and said method comprises:
Application subsystem generates log information;
Application subsystem is issued the log store subsystem stores with the log information that generates.
A kind of application subsystem is applied to above-mentioned log processing system, and said application subsystem comprises:
Generation unit is used to generate log information;
Transmitting element is used for the log information that generation unit generates is issued the log store subsystem stores.
A kind of log processing method is applied to above-mentioned log processing system, and said method comprises:
The log store subsystem obtains the log information that application subsystem is sent;
The log store subsystem is with the stored in form log information of message queue;
When the daily record storage subsystem obtain that the log analysis subsystem sends the request of log information is provided the time, the log store subsystem sends to the log analysis subsystem with the log information in the message queue.
Preferably, be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
The log information that the application subsystem that the log store subsystem obtains is sent carries the sign that is used to represent the log information type;
The log store subsystem comprises with the stored in form log information of message queue: the log store subsystem is according to said sign, with log information be stored in the represented corresponding message queue of log information type of said sign in.
Preferably, the request that log information is provided of sending of the log analysis subsystem that obtains of log store subsystem carries the sign of the log information type that is used to represent needs;
The log store subsystem according to the said sign that is used to represent the log information type of needs, was searched the message queue corresponding with the log information type of said needs before the log information in the message queue is sent to the log analysis subsystem;
If in the message queue that finds log information is arranged, then the log information in the message queue that finds is offered the log analysis subsystem.
A kind of log store subsystem is applied to above-mentioned log processing system, and said log store subsystem comprises:
Log information obtains the unit, is used to obtain the log information that application subsystem is sent;
Storage unit is used for the stored in form log information with message queue;
Request obtains the unit, is used to obtain the request that log information is provided that the log analysis subsystem sends;
Transmitting element, be used for when request obtain that the unit obtains that the log analysis subsystem sends the request of log information is provided the time, the log information in the message queue is sent to the log analysis subsystem.
Preferably, be provided with a plurality of message queues in the storage unit, all corresponding a kind of log information type of each message queue;
The log information that log information obtains the application subsystem transmission of unit acquisition carries the sign that is used to represent the log information type;
Storage unit specifically is used for according to said sign, with log information be stored in the represented corresponding message queue of log information type of said sign in.
Preferably, request obtains the sign that the request that log information is provided that log analysis subsystem that the unit obtains sends carries the log information type that is used to represent needs;
Said log store subsystem also comprises:
Search the unit, be used for before transmitting element sends to the log analysis subsystem with the log information of message queue,, search the message queue corresponding with the log information type of said needs according to the said sign that is used to represent the log information type of needs;
Judging unit is used for after searching the unit and finding the message queue corresponding with the log information type of said needs, judging in the message queue that finds whether log information is arranged;
Transmitting element specifically is used for after judging unit confirms that the message queue that finds has log information, the log information in the message queue that finds being offered the log analysis subsystem.
A kind of log processing method is applied to above-mentioned log processing system, and said method comprises:
The log analysis subsystem reads log information from the log store subsystem;
The log analysis subsystem is analyzed the log information that reads.
Preferably, be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
The log analysis subsystem reads log information and comprises from the log store subsystem: the log analysis subsystem reads log information according to the sign that is used to represent the log information type in the message queue from the log store subsystem.
A kind of log analysis subsystem is applied to above-mentioned log processing system, and said log analysis subsystem comprises:
Reading unit is used for reading log information from the log store subsystem;
Analytic unit is used for the log information that reads is analyzed.
Preferably, be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
Reading unit specifically is used for reading log information according to the sign that is used to represent the log information type in the message queue from the log store subsystem.
In the present invention, the application subsystem in the log processing system is issued log information the log store subsystem that is independent of application subsystem after generating log information; The log analysis subsystem can read log information and analyze log information from the log store subsystem; Like this, in the log processing process, just do not need application subsystem to keep in log information; Thereby need not take the storage resources of application subsystem, reduce using the influence of subsystem.
In the prior art; Be provided with logger module in the application subsystem, carry out between logger module and the log analysis subsystem mutual, the alter operation if logger module is upgraded etc.; Will have influence on the operate as normal of log analysis subsystem so; So, in the prior art, application subsystem and the coupling of log analysis subsystem height.In the present invention; The log store subsystem is independent of application subsystem; The log analysis subsystem reads log information from the log store subsystem, and the alter operation even application subsystem is upgraded etc. also can have influence on the operate as normal of log store subsystem and log analysis subsystem hardly; So the degree of coupling between log store subsystem and log analysis subsystem and the application subsystem is low.
In the prior art; The log analysis subsystem obtains the log information that a plurality of application subsystems are sent; Afterwards, the log analysis subsystem also need be put in order these log informations, if the quantity and the type of the log information that obtains are many more; The log analysis subsystem will be more complicated to the arrangement of log information so, and workload will be big more.In the present invention; The log store subsystem like this, is just put log information in order in the log store subsystem with the stored in form log information of message queue; Need not the log analysis subsystem be put in order by log information, reduced the workload of log analysis subsystem.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art; To do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below; Obviously, the accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills; Under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the synoptic diagram of existing a kind of log processing system;
Fig. 2 is the synoptic diagram of log processing system provided by the invention;
Fig. 3 is the synoptic diagram of log processing method provided by the invention;
Fig. 4 is the synoptic diagram of application subsystem provided by the invention;
Fig. 5 is the synoptic diagram of another kind of log processing method provided by the invention;
Fig. 6 is the synoptic diagram of log store subsystem provided by the invention;
Fig. 7 is the synoptic diagram of another log processing method provided by the invention;
Fig. 8 is the synoptic diagram of log analysis subsystem provided by the invention.
Embodiment
To combine the accompanying drawing in the embodiment of the invention below, the technical scheme in the embodiment of the invention is carried out clear, intactly description, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills are not making the every other embodiment that is obtained under the creative work prerequisite, all belong to the scope of the present invention's protection.
More clearly understand technical scheme of the present invention for ease of those skilled in the art, at first introduce the applied network environment of technical scheme of the present invention.
Technical scheme of the present invention can be applied to log processing system shown in Figure 2.As shown in Figure 2, log processing system can comprise application subsystem, log store subsystem and log analysis subsystem.Application subsystem can generate log information, and log information is issued the log store subsystem.The log store subsystem can use the stored in form log information of message queue.The log analysis subsystem is analyzed log information.
In the present invention, the log store subsystem is independent of application subsystem physically, that is to say; The log store subsystem does not coexist on the server with application subsystem; Application subsystem can adopt asynchronous mode that log information is issued the log store subsystem, after application subsystem generates log information like this, just log information is issued the log store subsystem; Thereby the storage of daily record and application subsystem just can be worked simultaneously, can when the stores processor of daily record, not block to live.Whether log store subsystem and log analysis subsystem be separate physically, can be designed according to actual needs by those skilled in the art, and the present invention does not do qualification.
In the present invention, log processing system can adopt the distributed design form, that is, log processing system can comprise at least one application subsystem, at least one log store subsystem and at least one log analysis subsystem.If log processing system comprises a plurality of application subsystems and a plurality of log store subsystem; Can all stipulate corresponding log store subsystem for each application subsystem so; For each log store subsystem is all stipulated corresponding application subsystem, each application subsystem can only corresponding log store subsystem, also can corresponding a plurality of log store subsystems; Each log store subsystem is application subsystem of correspondence only; Also can corresponding a plurality of application subsystems, concrete distributed frame can be designed by those skilled in the art according to actual needs, repeats no more here.
Describe in the face of a kind of log processing system of the present invention down.As shown in Figure 2, log processing system comprises: the application subsystem that generates log information; Be independent of the log store subsystem of application subsystem and storing daily record information; Analyze the log analysis subsystem of log information.Application subsystem specifically is used for log information is issued the log store subsystem; The log store subsystem specifically is used to use the stored in form log information of message queue, and the log analysis subsystem specifically is used for reading log information and analyzing log information from the log store subsystem.
Application subsystem just can be issued the log store subsystem with log information after generating log information, and like this, log information just needn't be temporarily stored in the application subsystem.
Can be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue, the log information type can be by being used to represent that the sign of log information type representes.A message queue can only be stored the log information of corresponding log information type.Message queue can be corresponding at least one be used to represent the sign of log information type.A plurality of message queues can the corresponding same sign that is used to represent the log information type.
The log information that application subsystem is issued the log store subsystem can carry the sign that is used to represent the log information type; Like this; The above-mentioned sign that the log store subsystem can carry according to log information, with log information be stored in the represented corresponding message queue of log information type of above-mentioned sign in.In other words, the log store subsystem is searched the message queue corresponding with above-mentioned sign according to above-mentioned sign, and log information is stored in the message queue that finds.The log analysis subsystem can read the log information that needs according to the sign that is used to represent the log information type from the log store subsystem.Concrete; If the log analysis subsystem need be analyzed the log information of certain log information type; The log analysis subsystem can send a request to the log store subsystem so; This request expression needs the log store subsystem that log information is provided, and this request can carry the sign of this log information type that is used to represent the needs analysis.The log store subsystem is searched the message queue corresponding with above-mentioned sign according to the above-mentioned sign of carrying in the request after obtaining this request.If in the message queue that finds log information is arranged, the log store subsystem is issued the log analysis subsystem with the log information in the message queue that finds so.
In the present invention, in the message queue of log analysis subsystem in can monitoring log store subsystem regularly whether log information is arranged,, then read log information if having.If there is not log information in the message queue, so just continue monitoring.Concrete, monitoring can be adopted the mode of inquiry.For example; The log analysis subsystem can regularly send a request to the log store subsystem, and whether be used for the apply for information formation has log information, if reply sure response message in the log store subsystem; Then representing has log information in the message queue; If reply the response message negating in the log store subsystem, perhaps in official hour, do not reply message, then representing does not have log information in the message queue.
Need to prove; If in the log analysis subsystem is issued the request that log information is provided of log store subsystem, do not carry the sign of expression log information type; We can say that then tomorrow, the will analyzing subsystem was not specified the log information that needs which kind of type; In this case; The log store subsystem can be issued the log analysis subsystem with the log information in any one message queue, also can the log information in a plurality of message queues be issued the log analysis subsystem, can also the log information in all message queues be issued the log analysis subsystem.
In like manner; If whether the sign of not carrying expression log information type in the request of log information is arranged in the log analysis subsystem is issued the apply for information formation of log store subsystem; We can say that then tomorrow, the will analyzing subsystem was not specified the log information that needs which kind of type; In this case; If the log store subsystem is determined in the rare message queue log information is arranged, the log store subsystem can be replied sure response message to the log analysis subsystem so, if the log store subsystem confirms all there is not log information in all message queues; The log store subsystem can be replied the response message negating to the log analysis subsystem so, does not perhaps reply message.
Log processing system shown in Figure 2 can also comprise the log management subsystem, and the log management subsystem is used to be provided for analyzing the analysis rule of log information.The log management subsystem can be issued the log analysis subsystem with analysis rule, and the log analysis subsystem can be analyzed log information according to analysis rule.
Log processing system shown in Figure 2 can also comprise the analysis result application subsystem, and the analysis result application subsystem is used for the result that the application log analyzing subsystem is analyzed log information.For example, the analysis result application subsystem can be performance analysis subsystem, Monitor And Control Subsystem etc.
Based on above-mentioned log processing system, the present invention provides multiple log processing method, application subsystem, log store subsystem and log analysis subsystem respectively from the angle of application subsystem, log store subsystem and log analysis subsystem.
At first introduce a kind of log processing method and application subsystem from the angle of application subsystem.
As shown in Figure 3, a kind of log processing method comprises:
S301: application subsystem generates log information;
S302: application subsystem is issued the log store subsystem stores with the log information that generates.
Application subsystem just can be issued the log store subsystem with log information after generating log information, and like this, log information just needn't be temporarily stored in the application subsystem.
The log information that application subsystem is issued the log store subsystem can carry the sign that is used to represent the log information type; Like this; The above-mentioned sign that the log store subsystem can carry according to log information, with log information be stored in the represented corresponding message queue of log information type of above-mentioned sign in.
As shown in Figure 4, a kind of application subsystem comprises: generation unit 401 is used to generate log information; Transmitting element 402 is used for the log information that generation unit 401 generates is issued the log store subsystem stores.
After generation unit 401 generated log information, transmitting element 402 just can be issued the log store subsystem with log information, and like this, log information just needn't be temporarily stored in the application subsystem.
The log information that transmitting element 402 is issued the log store subsystem can carry the sign that is used to represent the log information type; Like this; The above-mentioned sign that the log store subsystem can carry according to log information, with log information be stored in the represented corresponding message queue of log information type of above-mentioned sign in.
Introduce a kind of log processing method and log store subsystem from the angle of log store subsystem below.
As shown in Figure 5, a kind of log processing method comprises:
S501: the log store subsystem obtains the log information that application subsystem is sent;
S502: the log store subsystem is with the stored in form log information of message queue;
S503: when the daily record storage subsystem obtain that the log analysis subsystem sends the request of log information is provided the time, the log store subsystem sends to the log analysis subsystem with the log information in the message queue.
Can be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue.The log information that the application subsystem that the log store subsystem obtains is sent can carry the sign that is used to represent the log information type.The log store subsystem can be according to above-mentioned sign, with log information be stored in the represented corresponding message queue of log information type of above-mentioned sign in.
The request that log information is provided that the log analysis subsystem that the log store subsystem obtains sends can carry the sign of the log information type that is used to represent needs.The log store subsystem can be searched the message queue corresponding with the log information type of above-mentioned needs according to the above-mentioned sign that is used to represent the log information type of needs before the log information in the message queue is sent to the log analysis subsystem.If in the message queue that finds log information is arranged, then the log information in the message queue that finds is offered the log analysis subsystem.
As shown in Figure 6, a kind of log store subsystem comprises: log information obtains unit 601, is used to obtain the log information that application subsystem is sent; Storage unit 602 is used for the stored in form log information with message queue; Request obtains unit 603, is used to obtain the request that log information is provided that the log analysis subsystem sends; Transmitting element 604, be used for when request obtain that unit 603 obtains that the log analysis subsystems send the request of log information is provided the time, the log information in the message queue is sent to the log analysis subsystem.
Can be provided with a plurality of message queues in the storage unit 602, all corresponding a kind of log information type of each message queue.The log information that log information obtains the application subsystem transmission of unit 601 acquisitions can carry the sign that is used to represent the log information type.Storage unit 602 can be according to above-mentioned sign, with log information be stored in the represented corresponding message queue of log information type of above-mentioned sign in.
Request obtains the sign that the request that log information is provided that log analysis subsystem that unit 603 obtains sends can carry the log information type that is used to represent needs.
Log store subsystem shown in Figure 6 can also comprise: search unit 605; Be used for before transmitting element 604 sends to the log analysis subsystem with the log information of message queue; According to the above-mentioned sign that is used to represent the log information type of needs, search the message queue corresponding with the log information type of above-mentioned needs; Judging unit 606 is used for after searching unit 605 and finding the message queue corresponding with the log information type of above-mentioned needs, judging in the message queue that finds whether log information is arranged; Transmitting element 604 offers the log analysis subsystem with the log information in the message queue that finds after in the message queue that judging unit 606 is confirmed to find, log information being arranged.
The specific descriptions of above-mentioned log processing method and log store subsystem can repeat no more referring to the associated description among the above-mentioned log processing system embodiment here.
Introduce a kind of log processing method and log analysis subsystem from the angle of log analysis subsystem below.
As shown in Figure 7, a kind of log processing method comprises:
S701: the log analysis subsystem reads log information from the log store subsystem;
S702: the log analysis subsystem is analyzed the log information that reads.
Can be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue.The log analysis subsystem can read log information in the message queue from the log store subsystem according to the sign that is used to represent the log information type.
In the message queue of log analysis subsystem in can monitoring log store subsystem regularly whether log information is arranged,, then read log information if having.If there is not log information in the message queue, so just continue monitoring.
As shown in Figure 8, a kind of log analysis subsystem comprises: reading unit 801 is used for reading log information from the log store subsystem; Analytic unit 802 is used for the log information that reads is analyzed.
Can be provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue.Reading unit 801 can read log information in the message queue from the log store subsystem according to the sign that is used to represent the log information type.
The specific descriptions of above-mentioned log processing method and log analysis subsystem can repeat no more referring to the associated description among the above-mentioned log processing system embodiment here.
In the present invention, the application subsystem in the log processing system is issued log information the log store subsystem that is independent of application subsystem after generating log information; The log analysis subsystem can read log information and analyze log information from the log store subsystem; Like this, in the log processing process, just do not need application subsystem to keep in log information; Thereby need not take the storage resources of application subsystem, reduce using the influence of subsystem.
In the prior art; Be provided with logger module in the application subsystem, carry out between logger module and the log analysis subsystem mutual, the alter operation if logger module is upgraded etc.; Will have influence on the operate as normal of log analysis subsystem so; So, in the prior art, application subsystem and the coupling of log analysis subsystem height.In the present invention; The log store subsystem is independent of application subsystem; The log analysis subsystem reads log information from the log store subsystem, and the alter operation even application subsystem is upgraded etc. also can have influence on the operate as normal of log store subsystem and log analysis subsystem hardly; So the degree of coupling between log store subsystem and log analysis subsystem and the application subsystem is low.
In the prior art; The log analysis subsystem obtains the log information that a plurality of application subsystems are sent; Afterwards, the log analysis subsystem also need be put in order these log informations, if the quantity and the type of the log information that obtains are many more; The log analysis subsystem will be more complicated to the arrangement of log information so, and workload will be big more.In the present invention; The log store subsystem like this, is just put log information in order in the log store subsystem with the stored in form log information of message queue; Need not the log analysis subsystem be put in order by log information, reduced the workload of log analysis subsystem.
One of ordinary skill in the art will appreciate that all or part of flow process that realizes in the foregoing description method; Be to instruct relevant hardware to accomplish through computer program; Described program can be stored in the computer read/write memory medium; This program can comprise the flow process like the embodiment of above-mentioned each side method when carrying out.Wherein, described storage medium can be magnetic disc, CD, read-only storage memory body (Read-Only Memory, ROM) or at random store memory body (Random Access Memory, RAM) etc.
The above only is a preferred implementation of the present invention; Should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; Can also make some improvement and retouching, these improvement and retouching also should be regarded as protection scope of the present invention.
Claims (17)
1. a log processing system is characterized in that, comprising:
Generate the application subsystem of log information;
Be independent of the log store subsystem of application subsystem and storing daily record information;
Analyze the log analysis subsystem of log information;
Application subsystem specifically is used for log information is issued the log store subsystem; The log store subsystem specifically is used to use the stored in form log information of message queue, and the log analysis subsystem specifically is used for reading log information and analyzing log information from the log store subsystem.
2. the system of claim 1 is characterized in that, is provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
The log information that application subsystem is issued the log store subsystem carries the sign that is used to represent the log information type;
The log store subsystem specifically is used for according to said sign, with log information be stored in the represented corresponding message queue of log information type of said sign in.
3. system as claimed in claim 2 is characterized in that, the log analysis subsystem specifically is used for according to said sign, reads the log information that needs from the log store subsystem.
4. the system of claim 1 is characterized in that, the log analysis subsystem specifically is used for timing and monitors in the message queue of log store subsystem whether log information is arranged, if having, then reads log information.
5. the system of claim 1 is characterized in that, also comprises the log management subsystem, is used to be provided for analyzing the analysis rule of log information;
The log analysis subsystem specifically is used for according to the analysis rule of log management subsystem setting log information being analyzed.
6. a log processing method is characterized in that, is applied to any described log processing system of claim 1-5, and said method comprises:
Application subsystem generates log information;
Application subsystem is issued the log store subsystem stores with the log information that generates.
7. an application subsystem is characterized in that, is applied to any described log processing system of claim 1-5, and said application subsystem comprises:
Generation unit is used to generate log information;
Transmitting element is used for the log information that generation unit generates is issued the log store subsystem stores.
8. a log processing method is characterized in that, is applied to any described log processing system of claim 1-5, and said method comprises:
The log store subsystem obtains the log information that application subsystem is sent;
The log store subsystem is with the stored in form log information of message queue;
When the daily record storage subsystem obtain that the log analysis subsystem sends the request of log information is provided the time, the log store subsystem sends to the log analysis subsystem with the log information in the message queue.
9. method as claimed in claim 8 is characterized in that, is provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
The log information that the application subsystem that the log store subsystem obtains is sent carries the sign that is used to represent the log information type;
The log store subsystem comprises with the stored in form log information of message queue: the log store subsystem is according to said sign, with log information be stored in the represented corresponding message queue of log information type of said sign in.
10. method as claimed in claim 9 is characterized in that, the request that log information is provided that the log analysis subsystem that the log store subsystem obtains sends carries the sign of the log information type that is used to represent needs;
The log store subsystem according to the said sign that is used to represent the log information type of needs, was searched the message queue corresponding with the log information type of said needs before the log information in the message queue is sent to the log analysis subsystem;
If in the message queue that finds log information is arranged, then the log information in the message queue that finds is offered the log analysis subsystem.
11. a log store subsystem is characterized in that, is applied to any described log processing system of claim 1-5, said log store subsystem comprises:
Log information obtains the unit, is used to obtain the log information that application subsystem is sent;
Storage unit is used for the stored in form log information with message queue;
Request obtains the unit, is used to obtain the request that log information is provided that the log analysis subsystem sends;
Transmitting element, be used for when request obtain that the unit obtains that the log analysis subsystem sends the request of log information is provided the time, the log information in the message queue is sent to the log analysis subsystem.
12. log store subsystem as claimed in claim 11 is characterized in that, is provided with a plurality of message queues in the storage unit, all corresponding a kind of log information type of each message queue;
The log information that log information obtains the application subsystem transmission of unit acquisition carries the sign that is used to represent the log information type;
Storage unit specifically is used for according to said sign, with log information be stored in the represented corresponding message queue of log information type of said sign in.
13. log store subsystem as claimed in claim 12 is characterized in that, request obtains the sign that the request that log information is provided that log analysis subsystem that the unit obtains sends carries the log information type that is used to represent needs;
Said log store subsystem also comprises:
Search the unit, be used for before transmitting element sends to the log analysis subsystem with the log information of message queue,, search the message queue corresponding with the log information type of said needs according to the said sign that is used to represent the log information type of needs;
Judging unit is used for after searching the unit and finding the message queue corresponding with the log information type of said needs, judging in the message queue that finds whether log information is arranged;
Transmitting element specifically is used for after judging unit confirms that the message queue that finds has log information, the log information in the message queue that finds being offered the log analysis subsystem.
14. a log processing method is characterized in that, is applied to any described log processing system of claim 1-5, said method comprises:
The log analysis subsystem reads log information from the log store subsystem;
The log analysis subsystem is analyzed the log information that reads.
15. method as claimed in claim 14 is characterized in that, is provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
The log analysis subsystem reads log information and comprises from the log store subsystem: the log analysis subsystem reads log information according to the sign that is used to represent the log information type in the message queue from the log store subsystem.
16. a log analysis subsystem is characterized in that, is applied to any described log processing system of claim 1-5, said log analysis subsystem comprises:
Reading unit is used for reading log information from the log store subsystem;
Analytic unit is used for the log information that reads is analyzed.
17. log analysis subsystem as claimed in claim 16 is characterized in that, is provided with a plurality of message queues in the log store subsystem, all corresponding a kind of log information type of each message queue;
Reading unit specifically is used for reading log information according to the sign that is used to represent the log information type in the message queue from the log store subsystem.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102718155A CN102385549A (en) | 2010-09-02 | 2010-09-02 | Log processing system, log processing method and log storage sub-system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102718155A CN102385549A (en) | 2010-09-02 | 2010-09-02 | Log processing system, log processing method and log storage sub-system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102385549A true CN102385549A (en) | 2012-03-21 |
Family
ID=45824979
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010102718155A Pending CN102385549A (en) | 2010-09-02 | 2010-09-02 | Log processing system, log processing method and log storage sub-system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102385549A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838624A (en) * | 2013-12-16 | 2014-06-04 | 浪潮电子信息产业股份有限公司 | Log processing method based on Linux message queue |
| CN104253835A (en) * | 2013-06-27 | 2014-12-31 | 阿里巴巴集团控股有限公司 | User behavior data acquisition method and system |
| CN104937886A (en) * | 2013-01-30 | 2015-09-23 | 日本电信电话株式会社 | Log analysis device, information processing method and program |
| CN106027595A (en) * | 2016-04-25 | 2016-10-12 | 乐视控股（北京）有限公司 | Access log processing method and system for CDN node |
| CN106105112A (en) * | 2014-03-19 | 2016-11-09 | 日本电信电话株式会社 | Analysis rule adjusting apparatus, analysis rule adjust system, analysis rule method of adjustment and analysis rule adjustment programme |
| CN106169959A (en) * | 2016-07-21 | 2016-11-30 | 柳州龙辉科技有限公司 | A kind of log processing device |
| CN108052675A (en) * | 2017-12-28 | 2018-05-18 | 惠州Tcl家电集团有限公司 | Blog management method, system and computer readable storage medium |
| CN108170578A (en) * | 2018-01-08 | 2018-06-15 | 武汉斗鱼网络科技有限公司 | log collecting method and device |
| CN108459952A (en) * | 2017-02-21 | 2018-08-28 | 网易（杭州）网络有限公司 | The processing method and system of program exception |
| CN103729380B (en) * | 2012-10-16 | 2018-11-20 | 阿里巴巴集团控股有限公司 | A kind of data processing method, system and device |
| CN109299042A (en) * | 2018-11-12 | 2019-02-01 | 郑州云海信息技术有限公司 | A kind of log processing method and system |
| CN111224807A (en) * | 2018-11-27 | 2020-06-02 | 中国移动通信集团江西有限公司 | Distributed log processing method, device, equipment and computer storage medium |
| CN114978729A (en) * | 2022-05-27 | 2022-08-30 | 重庆长安汽车股份有限公司 | Vehicle-mounted intrusion detection method and system based on CAN bus and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1642104A (en) * | 2004-01-05 | 2005-07-20 | 华为技术有限公司 | Method and device for realizing system journal |
| CN1677931A (en) * | 2004-04-02 | 2005-10-05 | 鸿富锦精密工业（深圳）有限公司 | Network daily-record data management system and method |
| CN101267338A (en) * | 2008-04-23 | 2008-09-17 | 杭州思福迪信息技术有限公司 | High-performance log and behavior auditing system |
| CN101594252A (en) * | 2009-06-01 | 2009-12-02 | 中兴通讯股份有限公司 | A kind of massive logs storage management system and method |
-
2010
- 2010-09-02 CN CN2010102718155A patent/CN102385549A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1642104A (en) * | 2004-01-05 | 2005-07-20 | 华为技术有限公司 | Method and device for realizing system journal |
| CN1677931A (en) * | 2004-04-02 | 2005-10-05 | 鸿富锦精密工业（深圳）有限公司 | Network daily-record data management system and method |
| CN101267338A (en) * | 2008-04-23 | 2008-09-17 | 杭州思福迪信息技术有限公司 | High-performance log and behavior auditing system |
| CN101594252A (en) * | 2009-06-01 | 2009-12-02 | 中兴通讯股份有限公司 | A kind of massive logs storage management system and method |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729380B (en) * | 2012-10-16 | 2018-11-20 | 阿里巴巴集团控股有限公司 | A kind of data processing method, system and device |
| CN104937886A (en) * | 2013-01-30 | 2015-09-23 | 日本电信电话株式会社 | Log analysis device, information processing method and program |
| CN104937886B (en) * | 2013-01-30 | 2017-10-24 | 日本电信电话株式会社 | Log analysis device, information processing method |
| US9860278B2 (en) | 2013-01-30 | 2018-01-02 | Nippon Telegraph And Telephone Corporation | Log analyzing device, information processing method, and program |
| CN104253835B (en) * | 2013-06-27 | 2018-11-16 | 阿里巴巴集团控股有限公司 | User behavior data acquisition method and its system |
| CN104253835A (en) * | 2013-06-27 | 2014-12-31 | 阿里巴巴集团控股有限公司 | User behavior data acquisition method and system |
| CN103838624A (en) * | 2013-12-16 | 2014-06-04 | 浪潮电子信息产业股份有限公司 | Log processing method based on Linux message queue |
| CN106105112A (en) * | 2014-03-19 | 2016-11-09 | 日本电信电话株式会社 | Analysis rule adjusting apparatus, analysis rule adjust system, analysis rule method of adjustment and analysis rule adjustment programme |
| CN106105112B (en) * | 2014-03-19 | 2019-08-27 | 日本电信电话株式会社 | Analysis rule adjusts device, analysis rule adjustment system and analysis rule method of adjustment |
| CN106027595A (en) * | 2016-04-25 | 2016-10-12 | 乐视控股（北京）有限公司 | Access log processing method and system for CDN node |
| CN106169959A (en) * | 2016-07-21 | 2016-11-30 | 柳州龙辉科技有限公司 | A kind of log processing device |
| CN108459952A (en) * | 2017-02-21 | 2018-08-28 | 网易（杭州）网络有限公司 | The processing method and system of program exception |
| CN108459952B (en) * | 2017-02-21 | 2024-05-24 | 网易（杭州）网络有限公司 | Program exception processing method and system |
| CN108052675A (en) * | 2017-12-28 | 2018-05-18 | 惠州Tcl家电集团有限公司 | Blog management method, system and computer readable storage medium |
| CN108170578A (en) * | 2018-01-08 | 2018-06-15 | 武汉斗鱼网络科技有限公司 | log collecting method and device |
| CN108170578B (en) * | 2018-01-08 | 2021-05-28 | 武汉斗鱼网络科技有限公司 | Log collection method and device |
| CN109299042A (en) * | 2018-11-12 | 2019-02-01 | 郑州云海信息技术有限公司 | A kind of log processing method and system |
| CN111224807A (en) * | 2018-11-27 | 2020-06-02 | 中国移动通信集团江西有限公司 | Distributed log processing method, device, equipment and computer storage medium |
| CN111224807B (en) * | 2018-11-27 | 2023-08-01 | 中国移动通信集团江西有限公司 | Distributed log processing method, device, equipment and computer storage medium |
| CN114978729A (en) * | 2022-05-27 | 2022-08-30 | 重庆长安汽车股份有限公司 | Vehicle-mounted intrusion detection method and system based on CAN bus and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102385549A (en) | Log processing system, log processing method and log storage sub-system | |
| CN103716174A (en) | Test log acquisition system and method for the same | |
| CN102467408A (en) | Method and device for accessing data of virtual machine | |
| CN102043686A (en) | Disaster tolerance method, backup server and system of memory database | |
| CN112347165B (en) | Log processing method and device, server and computer readable storage medium | |
| CN114090479B (en) | Access report system, method, device, chip and electronic equipment | |
| CN108334556A (en) | A kind of method and system of analysis internet finance massive logs | |
| CN103678149A (en) | Data processing method and device | |
| EP3444722B1 (en) | Installation of operating system | |
| CN103955441A (en) | Equipment management system, equipment management method and IO (Input/Output) expansion interface | |
| CN105205167A (en) | Log data system | |
| WO2007027211A3 (en) | System and method for scanning memory for pestware | |
| CN105207832A (en) | Server stress testing method and device | |
| CN107357804A (en) | The analysis system and method for internet finance massive logs | |
| EP0924596A3 (en) | A storage subsystem having a plurality of interfaces conforming to a plurality of data formats | |
| CN108205482B (en) | File mount restoration methods | |
| CN104077082A (en) | Network voting data storage method and device | |
| CN104079600A (en) | File storage method, file storage device, file access client and metadata server system | |
| CN102129619A (en) | Service data processing method and system based on cloud storage | |
| CN102681790A (en) | Data dump device and method and system for data dump by same | |
| CN110297810A (en) | A kind of stream data processing method, device and electronic equipment | |
| CN103236938A (en) | Method and system for user action collection based on cache memory and asynchronous processing technology | |
| CN113656364A (en) | Sensor data processing method, device and computer readable storage medium | |
| CN105468502A (en) | Log collection method, device and system | |
| CN114422253B (en) | Distributed vulnerability scanning system, method and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120321 |