CN104580310A - Log processing method and server - Google Patents

Log processing method and server Download PDF

Info

Publication number
CN104580310A
CN104580310A CN201310496795.5A CN201310496795A CN104580310A CN 104580310 A CN104580310 A CN 104580310A CN 201310496795 A CN201310496795 A CN 201310496795A CN 104580310 A CN104580310 A CN 104580310A
Authority
CN
China
Prior art keywords
log information
daily record
data
record data
log
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201310496795.5A
Other languages
Chinese (zh)
Inventor
王吉业
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201310496795.5A priority Critical patent/CN104580310A/en
Publication of CN104580310A publication Critical patent/CN104580310A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/069Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications

Abstract

The invention discloses a log processing method and a server. The method comprises the following steps: receiving a first type of log message; searching a log message structure corresponding to the received first type of log message in a configuration file, wherein the log message structure corresponding to each of a plurality of log types including the first type is recorded in the configuration file, and the log message structure comprises a plurality of preset data attributes; acquiring the log data corresponding to each of the plurality of data attributes in the log message according to the searched log message structure. By virtue of the log processing method and the server, the technical problem of high maintenance cost of the large-platform-based log service caused by adopting a hard coding manner in the existing log message processing logic is solved; the technical effect of reducing the maintenance cost of the log service is realized.

Description

Log processing method and server
Technical field
The present invention relates to the communications field, in particular to a kind of log processing method and server.
Background technology
In the current communications field, along with the application on site of access log services platform, such as game application is more and more, different games log or different types of log-structured all different with a game, this causes the developer of log services platform to need to write different handling procedures to the day of dissimilar game.In this course, log server needs to identify the log information received, and by be kept at corresponding to the hard coded in log server log information processing logic different log informations is processed, thus every corresponding a kind of Log Types, then need the corresponding hard coded of interpolation one section so that process the log information of the type, thus bring the increase of development amount and the higher problem of maintenance cost.
For above-mentioned problem, at present effective solution is not yet proposed.
Summary of the invention
Embodiments provide a kind of log processing method and server, at least to solve the technical matters higher based on the log services maintenance cost of large platform because existing log information processing logic adopts hard coded mode to cause.
According to an aspect of the embodiment of the present invention, provide a kind of log processing method, comprising: the log information receiving the first kind; The log information structure that the log information of the above-mentioned first kind received is corresponding is searched in configuration file, wherein, record in above-mentioned configuration file and each the corresponding log information structure in the multiple Log Types comprising the above-mentioned first kind, above-mentioned log information structure comprises default multiple data attributes; Obtain from above-mentioned log information and each the corresponding daily record data in above-mentioned multiple data attribute according to the above-mentioned log information structure found.
According to the another aspect of the embodiment of the present invention, additionally provide a kind of log processing method, comprise: in configuration file, search the log information structure that the log information of the first kind is corresponding, wherein, record in above-mentioned configuration file and each the corresponding log information structure in the multiple Log Types comprising the above-mentioned first kind, above-mentioned log information structure comprises default multiple data attributes; Above-mentioned log information structure according to finding obtains and each the corresponding daily record data in above-mentioned multiple data attribute; Above-mentioned daily record data according to above-mentioned log information structure and acquisition generates log information; The above-mentioned log information generated is sent to log server.
According to the another aspect of the embodiment of the present invention, additionally provide a kind of server, comprising: receiving element, for receiving the log information of the first kind; Search unit, for the log information structure that the log information of searching the above-mentioned first kind received in configuration file is corresponding, wherein, record in above-mentioned configuration file and each the corresponding log information structure in the multiple Log Types comprising the above-mentioned first kind, above-mentioned log information structure comprises default multiple data attributes; Acquiring unit, for obtaining and each the corresponding daily record data in above-mentioned multiple data attribute according to the above-mentioned log information structure found from above-mentioned log information.
According to the another aspect of the embodiment of the present invention, additionally provide a kind of server, comprise: search unit, for the log information structure that the log information of searching the first kind in configuration file is corresponding, wherein, record in above-mentioned configuration file and each the corresponding log information structure in the multiple Log Types comprising the above-mentioned first kind, above-mentioned log information structure comprises default multiple data attributes; Acquiring unit, for obtaining and each the corresponding daily record data in above-mentioned multiple data attribute according to the above-mentioned log information structure found; Generation unit, for generating log information according to the above-mentioned daily record data of above-mentioned log information structure and acquisition; Transmitting element, for being sent to log server by the above-mentioned log information generated.
In embodiments of the present invention, be different from prior art, processing logic for log information can be considered as being divided in order to two parts, wherein a part is same or analogous actuating logic, another part for be recorded in configuration file for differentiation Log Types corresponding to the descriptor of different log information structures, thus for the log processing method provided according to the embodiment of the present invention, its executive agent can simply by searching and calling log information structure corresponding with the first kind in configuration file, and perform same or analogous actuating logic, namely the parsing to log information and process can be realized, thus reach the technique effect of the maintenance cost reducing log services, and then the technical matters higher based on the log services maintenance cost of large platform solved because existing log information processing logic adopts hard coded mode to cause.
Accompanying drawing explanation
Accompanying drawing described herein is used to provide a further understanding of the present invention, and form a application's part, schematic description and description of the present invention, for explaining the present invention, does not form inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the schematic diagram of a kind of log processing method according to the embodiment of the present invention;
Fig. 2 is the schematic diagram of the another kind of log processing method according to the embodiment of the present invention;
Fig. 3 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Fig. 4 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Fig. 5 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Fig. 6 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Fig. 7 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Fig. 8 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Fig. 9 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Figure 10 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Figure 11 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Figure 12 is the schematic diagram of another log processing method according to the embodiment of the present invention;
Figure 13 is the schematic diagram of a kind of server according to the embodiment of the present invention;
Figure 14 is the schematic diagram of the another kind of server according to the embodiment of the present invention;
Figure 15 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 16 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 17 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 18 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 19 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 20 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 21 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 22 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 23 is the schematic diagram of another server according to the embodiment of the present invention;
Figure 24 is the schematic diagram of another server according to the embodiment of the present invention.
Embodiment
The present invention program is understood better in order to make those skilled in the art person, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the embodiment of a part of the present invention, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, should belong to the scope of protection of the invention.
It should be noted that, term " first ", " second " etc. in instructions of the present invention and claims and above-mentioned accompanying drawing are for distinguishing similar object, and need not be used for describing specific order or precedence.Should be appreciated that the data used like this can be exchanged in the appropriate case, so as embodiments of the invention described herein can with except here diagram or describe those except order implement.In addition, term " comprises " and " having " and their any distortion, intention is to cover not exclusive comprising, such as, contain those steps or unit that the process of series of steps or unit, method, system, product or equipment is not necessarily limited to clearly list, but can comprise clearly do not list or for intrinsic other step of these processes, method, product or equipment or unit.
Embodiment 1
According to the embodiment of the present invention, provide a kind of log processing method, as shown in Figure 1, the method comprises:
S102: the log information receiving the first kind;
S104: search the log information structure that the log information of the first kind received is corresponding in configuration file, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes;
S106: obtain from log information and each the corresponding daily record data in multiple data attribute according to the log information structure found.
Will be clear that, one of technical solution of the present invention problem to be solved is to provide a kind of method, to process the message including log content received, wherein, for ease of statement, the above-mentioned message including log content can be defined as log information.
In embodiments of the present invention, the concrete manifestation form of above-mentioned log information can be HTML (Hypertext Markup Language) http (Hypertext Transfer Protocol) message, but this and do not mean that restriction constituted to the present invention, such as, in some embodiments of the invention, first message also can show as file transfer protocol (FTP) ftp(File Transfer Protocol) message, or other feasible message meeting File Transfer form or messages etc.
Generally speaking, for receiving this log information message and processing the log information received further, can be special log server.But in some embodiments of the invention, main body for performing above-mentioned log processing method also can be embodied in the third-party server being arranged on Operation Server, log server or other types, or the functional module etc. that server is installed, the present invention is not construed as limiting this.
According to the embodiment of the present invention provide log processing method, first can receive above-mentioned log information by step S102, can having that its concrete method of reseptance is known to those skilled in the art is multiple, and the present invention does not do tired stating at this.Further, in embodiments of the present invention, log information can be divided into polytype.Such as, for the daily record data platform towards multiple application on site while of, this platform can simultaneously for receiving the log information from above-mentioned multiple different application on site, wherein, due to type and/or the content difference of application, the log information that the log information generated in operation process and the server by application place send out also is had nothing in common with each other.On the other hand, even if for same application, the type of the log information generated in its operation process also can have multiple, such as, logining by user, publish the daily record data that the sequence of events triggered in process feeds back to the Operation Server of application, such as user ID, login time, log in the log information that IP address etc. can form a type, and the sequence of events triggered by user behavior in the operation process of application feeds back to the daily record data of Operation Server, the order number that such as user operation is corresponding, the concrete object of user operation, amounts etc. can form the log information of another kind of type, Deng.
Certainly, be more than some examples, and should not be construed as limitation of the invention.Usually, in embodiments of the present invention, the processing mode of the log information for any one type in above-mentioned polytype log information being described, wherein, for ease of describing, the log information of the type being designated as the log information of the first kind.Wherein, it should be noted that in embodiments of the present invention, the log information of the first kind should have the log information form corresponding with this first kind, and the detailed description for this form will provide in the examples below.
According to the log processing method that the embodiment of the present invention provides, in step S104, the log information structure that the log information of the first kind received is corresponding can be searched in configuration file, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes.
In embodiments of the present invention, log information structure can represent the descriptor for being described the structure of log information, thus find the log information structure corresponding with the log information of the first kind in step S104 after, can resolve the log information received according to this log information structure in step s 106 further.
Particularly, above-mentioned log information structure can be realized by Multi-encoding mode, such as, as one preferred embodiment, this log information structure can utilize xml encode write, log information structured documents can:
<struct name=" AccountLogout " id=" " desc=" account publishes daily record " >
<entry name=" iEventId " type=" ubigint " size=" " desc=" event id "/>
<entry name=" iUin " type=" uint " size=" " desc=" user ID "/>
<entry name=" dtEventTime " type=" datetime " size=" " desc=" time of exiting "/>
<entry name=" dtLoginTime " type=" datetime " size=" " desc=" time of logining "/>
<entry name=" vClientIp " type=" varchar " size=" 24 " desc=" when user logs in client place IP "/>
<entry name=" iZoneId " type=" int " size=" " desc=" channel number "/>
<entry name=" iOnlineTime " type=" int " size=" " desc=" this logs in online hours "/>
Wherein, the first row <struct name=" ... " > may be used for the type defining log information corresponding to this log information structure, such as, the typonym of this type can as defined abovely be " AccountLogout ", wherein, the practical significance of this type can as in xml document description part " desc=... " in record shown in, for representing the daily record of " account publishes daily record " this type.
In fact, on the other hand, <struct name=" ... " the above-mentioned type title defined in > can also be used as the identification item of this log information structure, serve the searching log information structure described in step S104, such as, alternatively, as shown in Figure 2, in embodiments of the present invention, step S104 can comprise:
S202: obtain the default identification item corresponding with the first kind from log information;
S204: search the log information structure that default identification item is corresponding in configuration file, and using log information structure corresponding for default identification item as log information structure corresponding to log information, wherein, record in configuration file and each the corresponding default identification item in the multiple Log Types comprising the first kind.
Under this scene, as a part for the preset format for log information, log information is carrying outside daily record data, the above-mentioned default identification item for identifying log information structure can also be carried, such as, for the log information structure shown in the xml document provided in above-mentioned example, its typonym is " AccountLogout ", if the type name is referred to as default identification item, then can be corresponding with this log information structure by the log information carrying the character string of " AccountLogout ", wherein, alternatively, this character string can be arranged on the head of the message body in log information, thus the message body of this log information can comprise content as follows:
AccountLogout|12345|678|2013-9-2516:00:00|2013-9-2510:00:00|192.168.13.10|…
In other words, when receiving this log information, the log information structure comprising " AccountLogout " this default identification item then can be searched in configuration file, and then the log information received is resolved, to obtain the daily record data that correspond to different pieces of information attribute carried in this log information according to the log information structure corresponding with log information found in step s 106.
Wherein, in above-mentioned xml document below the second row <struct name=" ... " > may be used for defining multiple data attribute, such as, the title of the data attribute of this two row of document definition can be " iEventId ", wherein, the practical significance of this data attribute can as describe part " desc=... " in record shown in, for representing " event id ", that is, daily record data in the log information that the data attribute " iEventId " defined in log information structure as shown in above-mentioned xml document is corresponding can be the ID of event, such as this daily record data can be " 12345 " in above-mentioned message body.
Similarly, can be defined multiple data attribute by log information structure, the multiple data attributes such as, defined in above-mentioned xml document: " iEventId ", " iUin ", " dtEventTime ", " dtLoginTime ", " vClientIp ", " iZoneId ", " iOnlineTime ", for describing the daily record data with following practical significance carried in log information respectively: " event id ", " user ID ", " time of exiting ", " time of logining ", " when user logs in client place IP ", " channel number ", " this logs in online hours ", also be, log information structure comprises multiple data attribute, wherein, these data attributes are for describing dissimilar daily record data, and to the sorting of the daily record data carried in log information in subsequent processes, the operations such as storage.
On basis described above, as the preferred mode of one, as shown in Figure 3, in embodiments of the present invention, before step S102, above-mentioned log processing method can also comprise:
S302: receive log information structure corresponding to the first kind and default identification item corresponding to the first kind;
S304: the log information structure received and default identification item are added in configuration file.
Wherein, this log information structure and default identification item can come from operator or the exploitation side of the application generating log information.Under above-mentioned scene, for log server or service platform, each interpolation service object, such as during a application on site, all can receive from the operator of this application or exploitation side with this application generate log information structure corresponding to each type the log information of the first kind or more eurypalynous log information and default identification item, thus for log server or service platform, as long as the log information structure received and default identification item are added in configuration file, namely newly-increased service object can be served under the prerequisite of the processing logic being stored in this locality not being modified, this changes an open service platform by making this log server or service platform into by a closed service platform.On the other hand, existing service object is needed to the situation that Log Types is upgraded, also can receive the log information structure after upgrading by log server similarly, thus significantly reduce the maintenance cost of log services.
By above-described embodiment, detailed elaboration is carried out to log information structure alleged in technical solution of the present invention, but above description is only for the understanding to technical solution of the present invention, should not be considered as constituting any unnecessary restriction to the present invention, such as, in embodiments of the present invention, the corresponding relation adopting default identification item to set up between log information and log information structure is non-essential, this default identification item also can not be set to character string, but the mapping mode etc. adopting other feasible, the present invention is not construed as limiting this.
And then according to the log processing method that the embodiment of the present invention provides, in step s 106, can obtain from log information and each the corresponding daily record data in multiple data attribute according to the log information structure found.
In embodiments of the present invention, step S106 can realize in several ways, and wherein, generally speaking, as one preferred embodiment, as shown in Figure 4, in embodiments of the present invention, step S106 can comprise:
S402: obtain daily record data in the position corresponding with data attribute from log information, wherein, log information structure also comprises: the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute.
Under above-mentioned scene, multiple data attribute is not only recorded in log information structure, the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute can also be recorded, wherein, this positional information can be not limited only to the corresponding particular location of daily record data in log information, such as, between the location point represented with byte number or lane place, also the relative position of daily record data in log information can be represented, the location point such as identified by identifier, or the in turn Rankine-Hugoniot relations etc. of daily record data in log information that multiple data attribute is corresponding.
Be described in detail by the implementation of some embodiments to step S402 below.
Wherein, as the optional embodiment of one, as shown in Figure 5, in embodiments of the present invention, step S402 can comprise:
S502: the data slot in acquisition log information between N number of default separator and N+1 default separator is as the daily record data corresponding with the N number of data attribute in log information structure, N>=1, wherein, log information carries multiple daily record data, and the default separator between before being arranged on multiple daily record data and in multiple daily record data adjacent two, and put in order with multiple daily record data corresponding data attribute the putting in order in log information structure of multiple daily record data in log information is identical, wherein, log information structure also comprises: preset separator.
For the form of the log information provided in previous embodiment, wherein, this log information can be:
AccountLogout|12345|678|2013-9-2516:00:00|2013-9-2510:00:00|192.168.13.10|…
Wherein, character " | " namely can be considered as default separator, and this default separator can be recorded in log information structure in the lump, thus in this log information, and adjacent two in multiple daily record data are separated by default separator " | ".Under this scene, namely the daily record data separated by default separator can be identified by log server and be sequentially read, and this reading order can be corresponding with the data attribute recorded in log information structure, such as, daily record data in this log information: " 12345 ", " 678 ", " 2013-9-2516:00:00 ", " 2013-9-2510:00:00 ", " 192.168.13.10 " can correspond respectively to the data attribute of sequential definition in above-mentioned xml document: " iEventId ", " iUin ", " dtEventTime ", " dtLoginTime ", " vClientIp ", and be respectively used to refer to: " event id ", " user ID ", " time of exiting ", " time of logining ", " when user logs in client place IP ".
Certainly, it is more than a kind of example, and do not mean that any restriction is constituted to the present invention, such as this default separator also can be written in the processing logic that the hard coded in log server embodies, no matter thus for the daily record of which kind of type or log information, also can all adopt same default separator as the reference identified the data attribute of daily record data and correspondence thereof.
Further, in some embodiments of the invention, due to problems such as Signal transmissions faults, still there is the possibility of makeing mistakes in the daily record data obtained from adjacent two separators described in step S502, thus alternatively, as shown in Figure 6, in embodiments of the present invention, step S502 can comprise:
S602: judge whether data slot belongs to the data type of daily record data corresponding to N number of data attribute, if belong to, then using data slot as the daily record data corresponding with the N number of data attribute in log information structure, wherein, log information structure also comprises: the data type of the daily record data of each correspondence in multiple data attribute.
Wherein, the mode verified by data type, can promote the reliability of log services further, more efficiently avoids possibility misdata imported in daily record data table of follow-up execution.
As the optional embodiment of another kind, as shown in Figure 7, in embodiments of the present invention, step S402 also can comprise:
S702: obtain the data slot of the designated length be positioned at after location identifier corresponding to the first attribute as the daily record data corresponding with the first attribute from log information, wherein, first attribute is any one in multiple data attribute, log information structure also comprises: the data type of the daily record data of each correspondence in the location identifier of each correspondence in multiple data attribute and multiple data attribute, wherein, the data type of the daily record data that designated length is corresponding with the first attribute matches.
Wherein, outside multiple data attribute, the data type of the daily record data of each correspondence in the location identifier of each correspondence in multiple data attribute and multiple data attribute can also be recorded in log information structure, thus the data slot of the designated length be positioned at after location identifier corresponding to the first attribute can be obtained as the daily record data corresponding with the first attribute from log information.In this way, the daily record data corresponding with data attribute can be obtained from log information equally.
Should be appreciated that the above embodiment provided is only for the understanding to technical solution of the present invention, wherein, in embodiments of the present invention, can also have multiple according to log information structure to the mode that log information is resolved, therefore not to repeat here.
In the prior art, log server identifies to the log information received, and by be kept at corresponding to the hard coded in log server log information processing logic different log informations is processed, thus every corresponding a kind of Log Types, then need the corresponding hard coded of interpolation one section so that process the log information of the type.In embodiments of the present invention, be different from prior art, processing logic for log information can be considered as being divided in order to two parts, wherein a part is same or analogous actuating logic, another part for be recorded in configuration file for differentiation Log Types corresponding to the descriptor of different log information structures, thus for the log processing method provided according to the embodiment of the present invention, its executive agent can simply by searching and calling log information structure corresponding with the first kind in configuration file, and perform same or analogous actuating logic, namely the parsing to log information and process can be realized, thus reach the technique effect of the maintenance cost reducing log services, and then the technical matters higher based on the log services maintenance cost of large platform solved because existing log information processing logic adopts hard coded mode to cause.
Further, in some embodiments of the invention, the regulation of homogenization can also be carried out to the form of dissimilar log information, also be, in step s 102, described log information can carry the daily record data be carried according to preset format in described log information, wherein, dissimilar log information can have dissimilar, the daily record data of different pieces of information attribute, also difference can be there is in the quantity of the daily record data included by it, but the arrangement mode of these daily record datas in log information can meet preset format, such as this preset format can be: put in order with daily record data corresponding data attribute the putting in order in log information structure of daily record data in log information is identical, thus in step s 106, when the daily record data corresponding to data attribute obtains, its obtain manner can be simplified further, such as it can carry out order acquisition by the order of the data attribute comprised in daily record message structure to the daily record data in log information in homogenization ground.Under this scene, namely the log information structure recorded in above-mentioned configuration file can have the form of homogenization, thus has further facilitated writing and maintenance of log information structure, can reach the technique effect reducing log services maintenance cost further.
After obtaining the daily record data corresponding with predefined data attribute by step S106, the daily record data can further these with relevant data attribute is stored in tables of data as data record, wherein, the field in tables of data can be set to these data attributes.
Wherein, alternatively, as shown in Figure 8, in embodiments of the present invention, after step s 106, above-mentioned log processing method can also comprise:
S802: be stored in tables of data by the daily record data of acquisition, wherein, also records the corresponding relation between the data attribute corresponding to daily record data and daily record data in tables of data.
Such as, for the log information received in previous embodiment, this log information be stored into formed in tables of data data line record can for as shown in table 1:
Table 1
Wherein, as shown in table 1, this tables of data can also be used for storage and belong to same Log Types, such as, belong to many log informations of the first kind.Wherein, as the optional embodiment of one, as shown in Figure 9, in embodiments of the present invention, step S102 can comprise:
S902: receive log information in preset time period; Wherein,
Step S802 can comprise:
S904: the daily record data of acquisition is stored in temporary log file, wherein, temporary log file store receive within the time cycle belong to the daily record data obtained all log informations of the first kind; After preset time period terminates, the daily record data stored in temporary log file batch is imported in the tables of data corresponding with the first kind; Empty temporary log file.
Under above-mentioned scene, for the receiving element for receiving log information, can be separated from each other with the importing module for importing daily record data, also be namely therebetween decoupling zero, thus the sluggishness that both can not produce because of the importing of the process of log information and daily record data has influence on the timely reception for log information, also can not lower in the processing pressure that receiving element receives data time the wasting of resources of importing module importing daily record data for batch that causes, thus improve the efficiency of the bulk treatment process of log information.
Embodiment 2
According to the embodiment of the present invention, provide a kind of log processing method, as shown in Figure 10, the method comprises:
S1002: search the log information structure that the log information of the first kind is corresponding in configuration file, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes;
S1004: obtain and each the corresponding daily record data in multiple data attribute according to the log information structure found;
S1006: the daily record data according to log information structure and acquisition generates log information;
S1008: the log information of generation is sent to log server.
Corresponding with the log processing method provided in embodiment 1, one of problem to be solved of the log processing method that the embodiment of the present invention provides is to provide a kind of method, to process daily record data, and the log information belonging to a certain type generated as described in example 1 above, and by step S1006, this log information is sent to log server.Wherein, for ease of statement, type belonging to the log information that will send in the embodiment of the present invention also can be designated as the first kind, this first kind can be identical with the first kind described in embodiment 1, also can be different, should be understood to any one in Log Types, and should not be considered as limitation of the invention.
According to the log processing method that the embodiment of the present invention provides, in step S1002, the log information structure that the log information of the first kind is corresponding can be searched in configuration file, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes.
Be similar to the concept of configuration file and the log information structure proposed in embodiment 1, in embodiments of the present invention, configuration file is equally for log message structure, wherein, this configuration file both can record a kind of log information structure, also multiple log information structure can be recorded, to match with polytype log information.
Generally speaking, in embodiments of the present invention, the executive agent of above-mentioned log processing method can be the Operation Server at one or more application place, but the present invention is not construed as limiting this.Wherein, the configuration file preserved in this Operation Server can record multiple log information structures corresponding to the log information of multiple types that one or more application that this Operation Server is runed produce usually, wherein, each Log Types can corresponding a kind of log information structure.
Log information structure alleged in the embodiment of the present invention can be corresponding consistent with the log information structure described in embodiment 1, such as, same Log Types, the concrete form of such as, in the log processing method that the log information of the first kind provides in the embodiment of the present invention corresponding log information structure, in the log processing method performed by log server of the log information of the first kind that can send with the log processing method provided for receiving the embodiment of the present invention, desired seek is identical with the concrete form of the log information structure called.Certainly, in some embodiments of the invention, also can adjust with carrying out correspondence log information structure, should be considered as within protection scope of the present invention equally.
By quoting the log information structure shown in the xml document that provides in embodiment 1, technical solution of the present invention is described below.
Be similar to the search procedure described in step S104 in embodiment 1, in step S1002, the default label entry that the first kind is corresponding can be utilized equally, such as xml document by the first row <struct name=" ... " the typonym " AccountLogout " defined in >, realizes searching this log information structure corresponding to the log information of the first kind.And then in step S1004, can obtain and each the corresponding daily record data in multiple data attribute according to the log information structure found, wherein, the concept of above-mentioned multiple data attribute can be consistent with the concept of the data attribute proposed in embodiment 1, the multiple data attributes defined in such as xml document: " iEventId ", " iUin ", " dtEventTime ", " dtLoginTime ", " vClientIp ", " iZoneId ", " iOnlineTime ".By step S1004, Operation Server can obtain and each the corresponding daily record data in above-mentioned multiple data attribute from the application of operation, and then in step S1006, the daily record data got can be encapsulated as the log information with preset format according to log information structure, and by step S1008, this log information be sent to log server.
On basis described above, log server as the take over party of this log information can utilize identical log information structure to resolve this log information that log server receives, and then acquisition is encapsulated into the above-mentioned daily record data in log information and the corresponding relation between these daily record datas and multiple data attribute again, wherein, before step S1002, above-mentioned log processing method can also comprise: above-mentioned log information structure is sent to log server, wherein, the log information structure sent can comprise the default label entry for identifying this log information structure.By with upper type, the technique effect of the maintenance cost reducing log services can be reached, and then solve the technical matters higher based on the log services maintenance cost of large platform because existing log information processing logic adopts hard coded mode to cause
Alternatively, as shown in figure 11, in embodiments of the present invention, step S1006 can comprise:
S1102: the daily record data of acquisition is inserted on the position corresponding to data attribute corresponding with daily record data in log information, wherein, log information structure also comprises: the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute.
Wherein, corresponding with the step S402 described in embodiment 1, in embodiments of the present invention, outside multiple data attribute, the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute can also be recorded in log information structure, and then by step S1102, can according to the record of this positional information, can the daily record data of acquisition be inserted on the position corresponding to data attribute corresponding with daily record data in log information, thus for log server, from log information, this daily record data can be obtained in the position corresponding with data attribute by step S402.
Further, as shown in figure 12, in embodiments of the present invention, step S1102 can comprise:
S1202: alternately add daily record data and default separator in log information, default separator is provided with between two of arbitrary neighborhood in log information daily record datas for making, and put in order with daily record data corresponding data attribute the putting in order in log information structure of daily record data in log information is identical, wherein, log information structure also comprises: preset separator.
Under above-mentioned scene, the message body in the log information generated by step S1202 can be content as follows:
AccountLogout|12345|678|2013-9-2516:00:00|2013-9-2510:00:00|192.168.13.10|…
Wherein, the form of this log information is consistent with the form of a kind of feasible log information provided in embodiment 1, thus for log server, can be resolved log information by the mode described in step S502 or step S602, with default separator for reference, obtain the daily record data corresponding with data attribute from the assigned address log information.
It should be noted that, for aforesaid each embodiment of the method, in order to simple description, therefore it is all expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not by the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in instructions all belongs to preferred embodiment, and involved action and module might not be that the present invention is necessary.
Through the above description of the embodiments, those skilled in the art can be well understood to the mode that can add required general hardware platform by software according to the log processing method of above-described embodiment and realize, hardware can certainly be passed through, but in a lot of situation, the former is better embodiment.Based on such understanding, technical scheme of the present invention can embody with the form of software product the part that prior art contributes in essence in other words, this computer software product is stored in a storage medium (as ROM/RAM, magnetic disc, CD), comprising some instructions in order to make a station terminal equipment (can be mobile phone, computing machine, server, or the network equipment etc.) perform method described in each embodiment of the present invention.
Embodiment 3
According to the embodiment of the present invention, additionally provide a kind of server for implementing the log processing method described in above-described embodiment 1, as shown in figure 13, this server comprises:
1) receiving element 1302, for receiving the log information of the first kind;
2) unit 1304 is searched, for the log information structure that the log information of searching the first kind received in configuration file is corresponding, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes;
3) acquiring unit 1306, for obtaining and each the corresponding daily record data in multiple data attribute according to the log information structure found from log information.
Will be clear that, one of technical solution of the present invention problem to be solved is to provide a kind of server, to process the message including log content received, wherein, for ease of statement, the above-mentioned message including log content can be defined as log information.
In embodiments of the present invention, the concrete manifestation form of above-mentioned log information can be HTML (Hypertext Markup Language) http (Hypertext Transfer Protocol) message, but this and do not mean that restriction constituted to the present invention, such as, in some embodiments of the invention, first message also can show as file transfer protocol (FTP) ftp(File Transfer Protocol) message, or other feasible message meeting File Transfer form or messages etc.
Generally speaking, for receiving this log information message and processing the log information received further, can be special log server.But in some embodiments of the invention, for performing the main body of aforesaid operations, also the server that namely embodiment of the present invention provides also can be embodied in the third-party server etc. of Operation Server, log server or other types, and the present invention is not construed as limiting this.
According to the embodiment of the present invention provide server, first can receive above-mentioned log information by receiving element 1302, can having that its concrete method of reseptance is known to those skilled in the art is multiple, and the present invention does not do tired stating at this.Further, in embodiments of the present invention, log information can be divided into polytype.Such as, for the daily record data platform towards multiple application on site while of, this platform can simultaneously for receiving the log information from above-mentioned multiple different application on site, wherein, due to type and/or the content difference of application, the log information that the log information generated in operation process and the server by application place send out also is had nothing in common with each other.On the other hand, even if for same application, the type of the log information generated in its operation process also can have multiple, such as, logining by user, publish the daily record data that the sequence of events triggered in process feeds back to the Operation Server of application, such as user ID, login time, log in the log information that IP address etc. can form a type, and the sequence of events triggered by user behavior in the operation process of application feeds back to the daily record data of Operation Server, the order number that such as user operation is corresponding, the concrete object of user operation, amounts etc. can form the log information of another kind of type, Deng.
Certainly, be more than some examples, and should not be construed as limitation of the invention.Usually, in embodiments of the present invention, the processing mode of the log information for any one type in above-mentioned polytype log information being described, wherein, for ease of describing, the log information of the type being designated as the log information of the first kind.Wherein, it should be noted that in embodiments of the present invention, the log information of the first kind should have the log information form corresponding with this first kind, and the detailed description for this form will provide in the examples below.
According to the server that the embodiment of the present invention provides, searching in unit 1304, the log information structure that the log information of the first kind received is corresponding can be searched in configuration file, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes.
In embodiments of the present invention, log information structure can represent the descriptor for being described the structure of log information, thus after searching in unit 1304 and finding the log information structure corresponding with the log information of the first kind, can resolve the log information received according to this log information structure in acquiring unit 1306 further.
Particularly, above-mentioned log information structure can be realized by Multi-encoding mode, such as, as one preferred embodiment, this log information structure can utilize xml encode write, log information structured documents can:
<struct name=" AccountLogout " id=" " desc=" account publishes daily record " >
<entry name=" iEventId " type=" ubigint " size=" " desc=" event id "/>
<entry name=" iUin " type=" uint " size=" " desc=" user ID "/>
<entry name=" dtEventTime " type=" datetime " size=" " desc=" time of exiting "/>
<entry name=" dtLoginTime " type=" datetime " size=" " desc=" time of logining "/>
<entry name=" vClientIp " type=" varchar " size=" 24 " desc=" when user logs in client place IP "/>
<entry name=" iZoneId " type=" int " size=" " desc=" channel number "/>
<entry name=" iOnlineTime " type=" int " size=" " desc=" this logs in online hours "/>
Wherein, the first row <struct name=" ... " > may be used for the type defining log information corresponding to this log information structure, such as, the typonym of this type can as defined abovely be " AccountLogout ", wherein, the practical significance of this type can as in xml document description part " desc=... " in record shown in, for representing the daily record of " account publishes daily record " this type.
In fact, on the other hand, <struct name=" ... " the above-mentioned type title defined in > can also be used as the identification item of this log information structure, serve the searching log information structure of searching described in unit 1304, such as, alternatively, as shown in figure 14, in embodiments of the present invention, search unit 1304 can comprise:
1) the second acquisition module 1402, for obtaining the default identification item corresponding with the first kind from log information;
2) module 1404 is searched, for searching log information structure corresponding to default identification item in configuration file, and using log information structure corresponding for default identification item as log information structure corresponding to log information, wherein, record in configuration file and each the corresponding default identification item in the multiple Log Types comprising the first kind.
Under this scene, as a part for the preset format for log information, log information is carrying outside daily record data, the above-mentioned default identification item for identifying log information structure can also be carried, such as, for the log information structure shown in the xml document provided in above-mentioned example, its typonym is " AccountLogout ", if the type name is referred to as default identification item, then can be corresponding with this log information structure by the log information carrying the character string of " AccountLogout ", wherein, alternatively, this character string can be arranged on the head of the message body in log information, thus the message body of this log information can comprise content as follows:
AccountLogout|12345|678|2013-9-2516:00:00|2013-9-2510:00:00|192.168.13.10|…
In other words, when receiving this log information, the log information structure comprising " AccountLogout " this default identification item then can be searched in configuration file, and then the log information received is resolved, to obtain the daily record data that correspond to different pieces of information attribute carried in this log information according to the log information structure corresponding with log information found in acquiring unit 1306.
Wherein, in above-mentioned xml document below the second row <struct name=" ... " > may be used for defining multiple data attribute, such as, the title of the data attribute of this two row of document definition can be " iEventId ", wherein, the practical significance of this data attribute can as describe part " desc=... " in record shown in, for representing " event id ", that is, daily record data in the log information that the data attribute " iEventId " defined in log information structure as shown in above-mentioned xml document is corresponding can be the ID of event, such as this daily record data can be " 12345 " in above-mentioned message body.
Similarly, can be defined multiple data attribute by log information structure, the multiple data attributes such as, defined in above-mentioned xml document: " iEventId ", " iUin ", " dtEventTime ", " dtLoginTime ", " vClientIp ", " iZoneId ", " iOnlineTime ", for describing the daily record data with following practical significance carried in log information respectively: " event id ", " user ID ", " time of exiting ", " time of logining ", " when user logs in client place IP ", " channel number ", " this logs in online hours ", also be, log information structure comprises multiple data attribute, wherein, these data attributes are for describing dissimilar daily record data, and to the sorting of the daily record data carried in log information in subsequent processes, the operations such as storage.
On basis described above, as the preferred mode of one, as shown in figure 15, in embodiments of the present invention, above-mentioned server can also comprise:
1) processing unit 1502, for receiving log information structure corresponding to the first kind and default identification item corresponding to the first kind, and adds to the log information structure received and default identification item in configuration file.
Wherein, this log information structure and default identification item can come from operator or the exploitation side of the application generating log information.Under above-mentioned scene, for log server or service platform, each interpolation service object, such as during a application on site, all can receive from the operator of this application or exploitation side with this application generate log information structure corresponding to each type the log information of the first kind or more eurypalynous log information and default identification item, thus for log server or service platform, as long as the log information structure received and default identification item are added in configuration file, namely newly-increased service object can be served under the prerequisite of the processing logic being stored in this locality not being modified, this changes an open service platform by making this log server or service platform into by a closed service platform.On the other hand, existing service object is needed to the situation that Log Types is upgraded, also can receive the log information structure after upgrading by log server similarly, thus significantly reduce the maintenance cost of log services.
By above-described embodiment, detailed elaboration is carried out to log information structure alleged in technical solution of the present invention, but above description is only for the understanding to technical solution of the present invention, should not be considered as constituting any unnecessary restriction to the present invention, such as, in embodiments of the present invention, the corresponding relation adopting default identification item to set up between log information and log information structure is non-essential, this default identification item also can not be set to character string, but the mapping mode etc. adopting other feasible, the present invention is not construed as limiting this.
And then according to the server that the embodiment of the present invention provides, in acquiring unit 1306, can obtain from log information and each the corresponding daily record data in multiple data attribute according to the log information structure found.
In embodiments of the present invention, acquiring unit 1306 can realize in several ways, and wherein, generally speaking, as one preferred embodiment, as shown in figure 16, in embodiments of the present invention, acquiring unit 1306 can comprise:
1) the first acquisition module 1602, for obtaining daily record data in the position corresponding with data attribute from log information, wherein, log information structure also comprises: the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute.
Under above-mentioned scene, multiple data attribute is not only recorded in log information structure, the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute can also be recorded, wherein, this positional information can be not limited only to the corresponding particular location of daily record data in log information, such as, between the location point represented with byte number or lane place, also the relative position of daily record data in log information can be represented, the location point such as identified by identifier, or the in turn Rankine-Hugoniot relations etc. of daily record data in log information that multiple data attribute is corresponding.
Be described in detail by the implementation of some embodiments to the first acquisition module 1602 below.
Wherein, as the optional embodiment of one, as shown in figure 17, in embodiments of the present invention, the first acquisition module 1602 can comprise:
1) first submodule 1702 is obtained, for obtaining data slot in log information between a N number of default separator and N+1 default separator as the daily record data corresponding with the N number of data attribute in log information structure, N>=1, wherein, log information carries multiple daily record data, and the default separator between before being arranged on multiple daily record data and in multiple daily record data adjacent two, and put in order with multiple daily record data corresponding data attribute the putting in order in log information structure of multiple daily record data in log information is identical, wherein, log information structure also comprises: preset separator.
For the form of the log information provided in previous embodiment, wherein, this log information can be:
AccountLogout|12345|678|2013-9-2516:00:00|2013-9-2510:00:00|192.168.13.10|…
Wherein, character " | " namely can be considered as default separator, and this default separator can be recorded in log information structure in the lump, thus in this log information, and adjacent two in multiple daily record data are separated by default separator " | ".Under this scene, namely the daily record data separated by default separator can be identified by log server and be sequentially read, and this reading order can be corresponding with the data attribute recorded in log information structure, such as, daily record data in this log information: " 12345 ", " 678 ", " 2013-9-2516:00:00 ", " 2013-9-2510:00:00 ", " 192.168.13.10 " can correspond respectively to the data attribute of sequential definition in above-mentioned xml document: " iEventId ", " iUin ", " dtEventTime ", " dtLoginTime ", " vClientIp ", and be respectively used to refer to: " event id ", " user ID ", " time of exiting ", " time of logining ", " when user logs in client place IP ".
Certainly, it is more than a kind of example, and do not mean that any restriction is constituted to the present invention, such as this default separator also can be written in the processing logic that the hard coded in log server embodies, no matter thus for the daily record of which kind of type or log information, also can all adopt same default separator as the reference identified the data attribute of daily record data and correspondence thereof.
Further, in some embodiments of the invention, due to problems such as Signal transmissions faults, first obtains the daily record data obtained from adjacent two separators described in submodule 1702 still exists the possibility of makeing mistakes, thus alternatively, as shown in figure 18, in embodiments of the present invention, the first acquisition submodule 1702 can comprise:
1) implementation sub-module 1802, during for belonging to the data type of daily record data corresponding to N number of data attribute at data slot, then using data slot as the daily record data corresponding with the N number of data attribute in log information structure, wherein, log information structure also comprises: the data type of the daily record data of each correspondence in multiple data attribute.
Wherein, the mode verified by data type, can promote the reliability of log services further, more efficiently avoids possibility misdata imported in daily record data table of follow-up execution.
As the optional embodiment of another kind, as shown in figure 19, in embodiments of the present invention, the first acquisition module 1602 also can comprise:
1) second submodule 1902 is obtained, for obtaining the data slot of the designated length be positioned at after location identifier corresponding to the first attribute as the daily record data corresponding with the first attribute from log information, wherein, first attribute is any one in multiple data attribute, log information structure also comprises: the data type of the daily record data of each correspondence in the location identifier of each correspondence in multiple data attribute and multiple data attribute, wherein, the data type of the daily record data that designated length is corresponding with the first attribute matches.
Wherein, outside multiple data attribute, the data type of the daily record data of each correspondence in the location identifier of each correspondence in multiple data attribute and multiple data attribute can also be recorded in log information structure, thus the data slot of the designated length be positioned at after location identifier corresponding to the first attribute can be obtained as the daily record data corresponding with the first attribute from log information.In this way, the daily record data corresponding with data attribute can be obtained from log information equally.
Should be appreciated that the above embodiment provided is only for the understanding to technical solution of the present invention, wherein, in embodiments of the present invention, can also have multiple according to log information structure to the mode that log information is resolved, therefore not to repeat here.
In the prior art, log server identifies to the log information received, and by be kept at corresponding to the hard coded in log server log information processing logic different log informations is processed, thus every corresponding a kind of Log Types, then need the corresponding hard coded of interpolation one section so that process the log information of the type.In embodiments of the present invention, be different from prior art, processing logic for log information can be considered as being divided in order to two parts, wherein a part is same or analogous actuating logic, another part for be recorded in configuration file for differentiation Log Types corresponding to the descriptor of different log information structures, thus for the server provided according to the embodiment of the present invention, its executive agent can simply by searching and calling log information structure corresponding with the first kind in configuration file, and perform same or analogous actuating logic, namely the parsing to log information and process can be realized, thus reach the technique effect of the maintenance cost reducing log services, and then the technical matters higher based on the log services maintenance cost of large platform solved because existing log information processing logic adopts hard coded mode to cause.
Further, in some embodiments of the invention, the regulation of homogenization can also be carried out to the form of dissimilar log information, also be, in receiving element 1302, described log information can carry the daily record data be carried according to preset format in described log information, wherein, dissimilar log information can have dissimilar, the daily record data of different pieces of information attribute, also difference can be there is in the quantity of the daily record data included by it, but the arrangement mode of these daily record datas in log information can meet preset format, such as this preset format can be: put in order with daily record data corresponding data attribute the putting in order in log information structure of daily record data in log information is identical, thus in acquiring unit 1306, when the daily record data corresponding to data attribute obtains, its obtain manner can be simplified further, such as it can carry out order acquisition by the order of the data attribute comprised in daily record message structure to the daily record data in log information in homogenization ground.Under this scene, namely the log information structure recorded in above-mentioned configuration file can have the form of homogenization, thus has further facilitated writing and maintenance of log information structure, can reach the technique effect reducing log services maintenance cost further.
After obtaining the daily record data corresponding with predefined data attribute by acquiring unit 1306, the daily record data can further these with relevant data attribute is stored in tables of data as data record, wherein, the field in tables of data can be set to these data attributes.
Wherein, alternatively, as shown in figure 20, in embodiments of the present invention, be coupled with acquiring unit 1306 ground, and above-mentioned server can also comprise:
1) storage unit 2002, for being stored in tables of data by the daily record data of acquisition, wherein, also records the corresponding relation between the data attribute corresponding to daily record data and daily record data in tables of data.
Such as, for the log information received in previous embodiment, this log information be stored into formed in tables of data data line record can for executing shown in the table 1 in example 1 strictly according to the facts:
Wherein, as shown in table 1, this tables of data can also be used for storage and belong to same Log Types, such as, belong to many log informations of the first kind.Wherein, as the optional embodiment of one, as shown in figure 21, in embodiments of the present invention,
Receiving element 1302 can comprise:
1) receiver module 2102, for receiving log information in preset time period; Wherein,
Storage unit 2002 can comprise:
1) memory module 2104, for the daily record data of acquisition is stored in temporary log file, wherein, temporary log file store receive within the time cycle belong to the daily record data obtained all log informations of the first kind;
2) import module 2106, for after preset time period terminates, the daily record data stored in temporary log file batch is imported in the tables of data corresponding with the first kind;
3) module 2108 is emptied, for emptying temporary log file.
Under above-mentioned scene, for the receiving element for receiving log information, can be separated from each other with the importing module for importing daily record data, also be namely therebetween decoupling zero, thus the sluggishness that both can not produce because of the importing of the process of log information and daily record data has influence on the timely reception for log information, also can not lower in the processing pressure that receiving element receives data time the wasting of resources of importing module importing daily record data for batch that causes, thus improve the efficiency of the bulk treatment process of log information.
Embodiment 4
According to the embodiment of the present invention, additionally provide a kind of server for implementing the log processing method described in above-described embodiment 2, as shown in figure 22, this server comprises:
1) unit 2202 is searched, for the log information structure that the log information of searching the first kind in configuration file is corresponding, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes;
2) acquiring unit 2204, for obtaining and each the corresponding daily record data in multiple data attribute according to the log information structure found;
3) generation unit 2206, generates log information for the daily record data according to log information structure and acquisition;
4) transmitting element 2208, for being sent to log server by the log information of generation.
Corresponding with the server provided in embodiment 2, one of problem to be solved of the server that the embodiment of the present invention provides is to provide a kind of server, to process daily record data, and the log information belonging to a certain type generated as described in example 2 above, and by generation unit 2206, this log information is sent to log server.Wherein, for ease of statement, type belonging to the log information that will send in the embodiment of the present invention also can be designated as the first kind, this first kind can be identical with the first kind described in embodiment 2, also can be different, should be understood to any one in Log Types, and should not be considered as limitation of the invention.
According to the server that the embodiment of the present invention provides, searching in unit 2202, the log information structure that the log information of the first kind is corresponding can be searched in configuration file, wherein, record in configuration file and each the corresponding log information structure in the multiple Log Types comprising the first kind, log information structure comprises default multiple data attributes.
Be similar to the concept of configuration file and the log information structure proposed in embodiment 2, in embodiments of the present invention, configuration file is equally for log message structure, wherein, this configuration file both can record a kind of log information structure, also multiple log information structure can be recorded, to match with polytype log information.
Generally speaking, in embodiments of the present invention, above-mentioned server can be the Operation Server at one or more application place, but the present invention is not construed as limiting this.Wherein, the configuration file preserved in this Operation Server can record multiple log information structures corresponding to the log information of multiple types that one or more application that this Operation Server is runed produce usually, wherein, each Log Types can corresponding a kind of log information structure.
Log information structure alleged in the embodiment of the present invention can be corresponding consistent with the log information structure described in embodiment 2, such as, same Log Types, the concrete form of such as, in the server that the log information of the first kind provides in the embodiment of the present invention corresponding log information structure, the log server desired seek of the log information of the first kind that can send with the server provided for receiving the embodiment of the present invention is identical with the concrete form of the log information structure called.Certainly, in some embodiments of the invention, also can adjust with carrying out correspondence log information structure, should be considered as within protection scope of the present invention equally.
By quoting the log information structure shown in the xml document that provides in embodiment 2, technical solution of the present invention is described below.
Be similar in embodiment 2 and search the search procedure described in unit 1304, searching in unit 2202, the default label entry that the first kind is corresponding can be utilized equally, such as xml document by the first row <struct name=" ... " the typonym " AccountLogout " defined in >, realizes searching this log information structure corresponding to the log information of the first kind.And then in acquiring unit 2204, can obtain and each the corresponding daily record data in multiple data attribute according to the log information structure found, wherein, the concept of above-mentioned multiple data attribute can be consistent with the concept of the data attribute proposed in embodiment 2, the multiple data attributes defined in such as xml document: " iEventId ", " iUin ", " dtEventTime ", " dtLoginTime ", " vClientIp ", " iZoneId ", " iOnlineTime ".By acquiring unit 2204, Operation Server can obtain and each the corresponding daily record data in above-mentioned multiple data attribute from the application of operation, and then in generation unit 2206, the daily record data got can be encapsulated as the log information with preset format according to log information structure, and by transmitting element 2208, this log information be sent to log server.
On basis described above, log server as the take over party of this log information can utilize identical log information structure to resolve this log information that log server receives, and then acquisition is encapsulated into the above-mentioned daily record data in log information and the corresponding relation between these daily record datas and multiple data attribute again, wherein, before searching unit 2202, above-mentioned server can also comprise: above-mentioned log information structure is sent to log server, wherein, the log information structure sent can comprise the default label entry for identifying this log information structure.By with upper type, the technique effect of the maintenance cost reducing log services can be reached, and then solve the technical matters higher based on the log services maintenance cost of large platform because existing log information processing logic adopts hard coded mode to cause
Alternatively, as shown in figure 23, in embodiments of the present invention, generation unit 2206 can comprise:
1) insert module 2302, for the daily record data of acquisition being inserted into the position corresponding to data attribute corresponding with daily record data in log information, wherein, log information structure also comprises: the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute.
Wherein, corresponding with the first acquisition module 1602 described in embodiment 2, in embodiments of the present invention, outside multiple data attribute, the positional information of daily record data in the log information corresponding to log information structure of each correspondence in multiple data attribute can also be recorded in log information structure, and then by insert module 2302, can according to the record of this positional information, can the daily record data of acquisition be inserted on the position corresponding to data attribute corresponding with daily record data in log information, thus for log server, from log information, this daily record data can be obtained in the position corresponding with data attribute by the first acquisition module 1602.
Further, as shown in figure 24, in embodiments of the present invention, insert module 2302 can comprise:
1) intron module 2402, for alternately adding daily record data and default separator in log information, default separator is provided with between two of arbitrary neighborhood in log information daily record datas for making, and put in order with daily record data corresponding data attribute the putting in order in log information structure of daily record data in log information is identical, wherein, log information structure also comprises: preset separator.
Under above-mentioned scene, the message body in the log information generated by intron module 2402 can be content as follows:
AccountLogout|12345|678|2013-9-2516:00:00|2013-9-2510:00:00|192.168.13.10|…
Wherein, the form of this log information is consistent with the form of a kind of feasible log information provided in embodiment 2, thus for log server, the mode described in submodule 1702 or implementation sub-module 1802 can be obtained by first to resolve log information, with default separator for reference, obtain the daily record data corresponding with data attribute from the assigned address log information.
The invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.
In the above embodiment of the present invention, the description of each embodiment is all emphasized particularly on different fields, in certain embodiment, there is no the part described in detail, can see the associated description of other embodiments.
In several embodiments that the application provides, should be understood that, disclosed server, the mode by other realizes.Wherein, device embodiment described above is only schematic, the such as division of described unit, be only a kind of logic function to divide, actual can have other dividing mode when realizing, such as multiple unit or assembly can in conjunction with or another system can be integrated into, or some features can be ignored, or do not perform.Another point, shown or discussed coupling each other or direct-coupling or communication connection can be by some interfaces, and the indirect coupling of unit or module or communication connection can be electrical or other form.
The described unit illustrated as separating component or can may not be and physically separates, and the parts as unit display can be or may not be physical location, namely can be positioned at a place, or also can be distributed in multiple network element.Some or all of unit wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, also can be that the independent physics of unit exists, also can two or more unit in a unit integrated.Above-mentioned integrated unit both can adopt the form of hardware to realize, and the form of SFU software functional unit also can be adopted to realize.
If described integrated unit using the form of SFU software functional unit realize and as independently production marketing or use time, can be stored in a computer read/write memory medium.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words or all or part of of this technical scheme can embody with the form of software product, this computer software product is stored in a storage medium, comprises all or part of step of some instructions in order to make a computer equipment (can be personal computer, server or the network equipment etc.) perform method described in each embodiment of the present invention.And aforesaid storage medium comprises: USB flash disk, ROM (read-only memory) (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), portable hard drive, magnetic disc or CD etc. various can be program code stored medium.
The above is only the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.

Claims (24)

1. a log processing method, is characterized in that, comprising:
Receive the log information of the first kind;
The log information structure that the log information of the described first kind received is corresponding is searched in configuration file, wherein, record in described configuration file and each the corresponding log information structure in the multiple Log Types comprising the described first kind, described log information structure comprises default multiple data attributes;
Obtain from described log information and each the corresponding daily record data in described multiple data attribute according to the described log information structure found.
2. method according to claim 1, is characterized in that, the described log information structure that described basis finds obtains and comprises with each the corresponding daily record data in described multiple data attribute from described log information:
Described daily record data is obtained in the position corresponding with described data attribute from described log information, wherein, described log information structure also comprises: the positional information of daily record data in the log information corresponding to described log information structure of each correspondence in described multiple data attribute.
3. method according to claim 2, is characterized in that, described in position acquisition corresponding with described data attribute the described described log information from receiving, daily record data comprises:
Obtain data slot in described log information between a N number of default separator and N+1 described default separator as the daily record data corresponding with the N number of data attribute in described log information structure, N>=1, wherein, described log information carries multiple daily record data, and the described default separator between before being arranged on described multiple daily record data and in described multiple daily record data adjacent two, and put in order with described multiple daily record data corresponding data attribute the putting in order in described log information structure of described multiple daily record data in described log information is identical, wherein, described log information structure also comprises: described default separator.
4. method according to claim 3, it is characterized in that, the data slot in the described log information of described acquisition between N number of default separator and N+1 described default separator comprises as the daily record data corresponding with the N number of data attribute recorded in described log information structure:
Judge whether described data slot belongs to the data type of daily record data corresponding to described N number of data attribute, if belong to, then using described data slot as the daily record data corresponding with the N number of data attribute in described log information structure, wherein, described log information structure also comprises: the data type of the daily record data of each correspondence in described multiple data attribute.
5. method according to claim 2, is characterized in that, describedly from described log information, obtains described daily record data in the position corresponding with described data attribute comprise:
The data slot of the designated length be positioned at after location identifier corresponding to the first attribute is obtained as the daily record data corresponding with described first attribute from described log information, wherein, described first attribute is any one in described multiple data attribute, described log information structure also comprises: the data type of the daily record data of each correspondence in the location identifier of each correspondence in described multiple data attribute and described multiple data attribute, wherein, the data type of the daily record data that described designated length is corresponding with described first attribute matches.
6. method according to any one of claim 1 to 5, is characterized in that, log information structure corresponding to described log information of searching the described first kind received in configuration file comprises:
The default identification item corresponding with the described first kind is obtained from described log information;
The log information structure that described default identification item is corresponding is searched in configuration file, and using log information structure corresponding for described default identification item as log information structure corresponding to described log information, wherein, record in described configuration file and each the corresponding default identification item in the multiple Log Types comprising the described first kind.
7. method according to claim 6, is characterized in that, before the log information of the described reception first kind, also comprises:
Receive described log information structure corresponding to the described first kind and described default identification item corresponding to the described first kind, and the described log information structure received and described default identification item are added in described configuration file.
8. method according to any one of claim 1 to 5, is characterized in that, the described log information structure found in described basis obtains with after each the corresponding daily record data in described multiple data attribute from described log information, also comprises:
The described daily record data obtained is stored in tables of data, wherein, in described tables of data, also records the corresponding relation between described daily record data and the data attribute corresponding to described daily record data.
9. method according to claim 8, is characterized in that,
The log information of the described reception first kind comprises: in preset time period, receive described log information;
The described described daily record data by acquisition is stored into tables of data and comprises: be stored in temporary log file by the described daily record data obtained, wherein, described temporary log file store receive within the described time cycle belong to the daily record data obtained all log informations of the described first kind; After described preset time period terminates, the daily record data stored in described temporary log file batch is imported in the described tables of data corresponding with the described first kind; Empty described temporary log file.
10. a log processing method, is characterized in that, comprising:
The log information structure that the log information of the first kind is corresponding is searched in configuration file, wherein, record in described configuration file and each the corresponding log information structure in the multiple Log Types comprising the described first kind, described log information structure comprises default multiple data attributes;
Described log information structure according to finding obtains and each the corresponding daily record data in described multiple data attribute;
Described daily record data according to described log information structure and acquisition generates log information;
The described log information generated is sent to log server.
11. methods according to claim 10, is characterized in that, the described described daily record data generation log information according to described log information structure and acquisition comprises:
The described daily record data obtained is inserted on the position corresponding to data attribute corresponding with described daily record data in described log information, wherein, described log information structure also comprises: the positional information of daily record data in the log information corresponding to described log information structure of each correspondence in described multiple data attribute.
12. methods according to claim 11, is characterized in that, the described position be inserted into by the described daily record data obtained corresponding to data attribute corresponding with described daily record data in described log information comprises:
Described daily record data and default separator is alternately added in described log information, described default separator is provided with between two of arbitrary neighborhood in described log information daily record datas for making, and put in order with described daily record data corresponding data attribute the putting in order in described log information structure of described daily record data in described log information is identical, wherein, described log information structure also comprises: described default separator.
13. 1 kinds of servers, is characterized in that, comprising:
Receiving element, for receiving the log information of the first kind;
Search unit, for the log information structure that the log information of searching the described first kind received in configuration file is corresponding, wherein, record in described configuration file and each the corresponding log information structure in the multiple Log Types comprising the described first kind, described log information structure comprises default multiple data attributes;
Acquiring unit, for obtaining and each the corresponding daily record data in described multiple data attribute according to the described log information structure found from described log information.
14. servers according to claim 13, is characterized in that, described acquiring unit comprises:
First acquisition module, for obtaining described daily record data in the position corresponding with described data attribute from described log information, wherein, described log information structure also comprises: the positional information of daily record data in the log information corresponding to described log information structure of each correspondence in described multiple data attribute.
15. servers according to claim 14, is characterized in that, described first acquisition module comprises:
First obtains submodule, for obtaining data slot in described log information between a N number of default separator and N+1 described default separator as the daily record data corresponding with the N number of data attribute in described log information structure, N>=1, wherein, described log information carries multiple daily record data, and the described default separator between before being arranged on described multiple daily record data and in described multiple daily record data adjacent two, and put in order with described multiple daily record data corresponding data attribute the putting in order in described log information structure of described multiple daily record data in described log information is identical, wherein, described log information structure also comprises: described default separator.
16. servers according to claim 15, is characterized in that, described first obtains submodule comprises:
Implementation sub-module, during for belonging to the data type of daily record data corresponding to described N number of data attribute at described data slot, then using described data slot as the daily record data corresponding with the N number of data attribute in described log information structure, wherein, described log information structure also comprises: the data type of the daily record data of each correspondence in described multiple data attribute.
17. servers according to claim 14, is characterized in that, described first acquisition module comprises:
Second obtains submodule, for obtaining the data slot of the designated length be positioned at after location identifier corresponding to the first attribute as the daily record data corresponding with described first attribute from described log information, wherein, described first attribute is any one in described multiple data attribute, described log information structure also comprises: the data type of the daily record data of each correspondence in the location identifier of each correspondence in described multiple data attribute and described multiple data attribute, wherein, the data type of the daily record data that described designated length is corresponding with described first attribute matches.
18., according to claim 13 to the server according to any one of 17, is characterized in that, described in search unit and comprise:
Second acquisition module, for obtaining the default identification item corresponding with the described first kind from described log information;
Search module, for searching log information structure corresponding to described default identification item in configuration file, and using log information structure corresponding for described default identification item as log information structure corresponding to described log information, wherein, record in described configuration file and each the corresponding default identification item in the multiple Log Types comprising the described first kind.
19. servers according to claim 18, is characterized in that, also comprise:
Processing unit, for receiving described log information structure corresponding to the described first kind and described default identification item corresponding to the described first kind, and adds to the described log information structure received and described default identification item in described configuration file.
20., according to claim 13 to the server according to any one of 17, is characterized in that, also comprise:
Storage unit, for being stored in tables of data by the described daily record data obtained, wherein, also records the corresponding relation between described daily record data and the data attribute corresponding to described daily record data in described tables of data.
21. servers according to claim 20, is characterized in that,
Described receiving element comprises:
Receiver module, for receiving described log information in preset time period;
Described storage unit comprises:
Memory module, for by obtain described daily record data be stored in temporary log file, wherein, described temporary log file store receive within the described time cycle belong to the daily record data obtained all log informations of the described first kind;
Import module, for after described preset time period terminates, the daily record data stored in described temporary log file batch is imported in the described tables of data corresponding with the described first kind;
Empty module, for emptying described temporary log file.
22. 1 kinds of servers, is characterized in that, comprising:
Search unit, for the log information structure that the log information of searching the first kind in configuration file is corresponding, wherein, record in described configuration file and each the corresponding log information structure in the multiple Log Types comprising the described first kind, described log information structure comprises default multiple data attributes;
Acquiring unit, for obtaining and each the corresponding daily record data in described multiple data attribute according to the described log information structure found;
Generation unit, for generating log information according to the described daily record data of described log information structure and acquisition;
Transmitting element, for being sent to log server by the described log information generated.
23. servers according to claim 22, is characterized in that, described generation unit comprises:
Insert module, for the described daily record data obtained being inserted into the position corresponding to data attribute corresponding with described daily record data in described log information, wherein, described log information structure also comprises: the positional information of daily record data in the log information corresponding to described log information structure of each correspondence in described multiple data attribute.
24. servers according to claim 23, is characterized in that, described insert module comprises:
Intron module, for alternately adding described daily record data and default separator in described log information, described default separator is provided with between two of arbitrary neighborhood in described log information daily record datas for making, and put in order with described daily record data corresponding data attribute the putting in order in described log information structure of described daily record data in described log information is identical, wherein, described log information structure also comprises: described default separator.
CN201310496795.5A 2013-10-21 2013-10-21 Log processing method and server Pending CN104580310A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310496795.5A CN104580310A (en) 2013-10-21 2013-10-21 Log processing method and server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310496795.5A CN104580310A (en) 2013-10-21 2013-10-21 Log processing method and server

Publications (1)

Publication Number Publication Date
CN104580310A true CN104580310A (en) 2015-04-29

Family

ID=53095483

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310496795.5A Pending CN104580310A (en) 2013-10-21 2013-10-21 Log processing method and server

Country Status (1)

Country Link
CN (1) CN104580310A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528566A (en) * 2015-09-11 2017-03-22 北京国双科技有限公司 Log file output method, server and client
CN106649729A (en) * 2016-12-23 2017-05-10 深圳市金证科技股份有限公司 Log running method and system, client and server
CN107066538A (en) * 2017-03-06 2017-08-18 北京安博通科技股份有限公司 A kind of method and device of data statistics
CN107426039A (en) * 2017-09-18 2017-12-01 武汉虹信通信技术有限责任公司 The method and system that a kind of daily record is reported and received
CN110362547A (en) * 2018-04-02 2019-10-22 阿里巴巴集团控股有限公司 Coding, parsing, storage method and the device of journal file
CN111651481A (en) * 2020-05-29 2020-09-11 泰康保险集团股份有限公司 Log processing method, device, equipment and computer readable storage medium
CN113407508A (en) * 2021-06-11 2021-09-17 济南浪潮数据技术有限公司 Method, system, equipment and medium for compressing log file
WO2021232217A1 (en) * 2020-05-19 2021-11-25 Arris Enterprises Llc Automatic adjustment of logging level of electronic device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1633080A (en) * 2003-12-24 2005-06-29 华为技术有限公司 Method for implementing log in network management system
CN101931562A (en) * 2010-09-29 2010-12-29 杭州华三通信技术有限公司 Web log processing method and device
CN102768636A (en) * 2011-05-05 2012-11-07 阿里巴巴集团控股有限公司 Log analysis method and log analysis device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1633080A (en) * 2003-12-24 2005-06-29 华为技术有限公司 Method for implementing log in network management system
CN101931562A (en) * 2010-09-29 2010-12-29 杭州华三通信技术有限公司 Web log processing method and device
CN102768636A (en) * 2011-05-05 2012-11-07 阿里巴巴集团控股有限公司 Log analysis method and log analysis device

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528566A (en) * 2015-09-11 2017-03-22 北京国双科技有限公司 Log file output method, server and client
CN106649729A (en) * 2016-12-23 2017-05-10 深圳市金证科技股份有限公司 Log running method and system, client and server
CN107066538A (en) * 2017-03-06 2017-08-18 北京安博通科技股份有限公司 A kind of method and device of data statistics
CN107066538B (en) * 2017-03-06 2020-09-01 北京安博通科技股份有限公司 Data statistics method and device
CN107426039A (en) * 2017-09-18 2017-12-01 武汉虹信通信技术有限责任公司 The method and system that a kind of daily record is reported and received
CN110362547A (en) * 2018-04-02 2019-10-22 阿里巴巴集团控股有限公司 Coding, parsing, storage method and the device of journal file
CN110362547B (en) * 2018-04-02 2023-10-03 杭州阿里巴巴智融数字技术有限公司 Method and device for encoding, analyzing and storing log file
WO2021232217A1 (en) * 2020-05-19 2021-11-25 Arris Enterprises Llc Automatic adjustment of logging level of electronic device
US11847042B2 (en) 2020-05-19 2023-12-19 Arris Enterprises Llc Automatic adjustment of a logging level of an electronic device
CN111651481A (en) * 2020-05-29 2020-09-11 泰康保险集团股份有限公司 Log processing method, device, equipment and computer readable storage medium
CN111651481B (en) * 2020-05-29 2023-08-22 泰康保险集团股份有限公司 Log processing method, device, equipment and computer readable storage medium
CN113407508A (en) * 2021-06-11 2021-09-17 济南浪潮数据技术有限公司 Method, system, equipment and medium for compressing log file

Similar Documents

Publication Publication Date Title
CN104580310A (en) Log processing method and server
CN111953776A (en) Application service message pushing method and device, computer equipment and computer storage medium
CN104253741B (en) A kind of method for sending information, relevant apparatus and system
CN102421062B (en) Method and system for pushing application information
CN103533530A (en) Cross-device user corresponding and user tracking methods and systems
CN103838867A (en) Log processing method and device
CN105338124A (en) Resource propagating tracking method and apparatus, and resource propagating system
CN103563414A (en) Mobile network operator identification
CN103347258A (en) Downloading method, system and server of application program and terminal equipment
CN104022940A (en) Information processing method, device and system based on instant messaging (IM)
CN104850546A (en) Mobile media information display method and system
CN103259802A (en) Media file downloading method and device and client device
CN104105007A (en) Video loading method of mobile terminal, devices and system
CN105430534A (en) Intelligent device reporting data method and system
RU2670596C2 (en) Data label carrier information application and processing system and method
CA2931525A1 (en) Data stream identifying method and device
CN105141653A (en) Method, equipment and system for data interaction of service platform
CN104618410A (en) Resource push method and resource push device
CN107590682B (en) Advertisement putting method, server, hotspot device and system
CN105550179A (en) Webpage collection method and browser plug-in
CN102984676B (en) Method and terminal for processing message session
CN103139049B (en) Information interacting method
CN107423446A (en) New media based on cloud storage automates credible deployment system and method
CN109213955B (en) Data processing method and related equipment
CN104298416A (en) Method and device for generating auto-response messages in instant messaging

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20150429

RJ01 Rejection of invention patent application after publication