CN107153602A - Handle the device and method of the log data of aol server - Google Patents

Handle the device and method of the log data of aol server Download PDF

Info

Publication number
CN107153602A
CN107153602A CN201710245653.XA CN201710245653A CN107153602A CN 107153602 A CN107153602 A CN 107153602A CN 201710245653 A CN201710245653 A CN 201710245653A CN 107153602 A CN107153602 A CN 107153602A
Authority
CN
China
Prior art keywords
log data
log
aol server
processing
set form
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710245653.XA
Other languages
Chinese (zh)
Inventor
牛兵奇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nubia Technology Co Ltd
Original Assignee
Nubia Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nubia Technology Co Ltd filed Critical Nubia Technology Co Ltd
Priority to CN201710245653.XA priority Critical patent/CN107153602A/en
Publication of CN107153602A publication Critical patent/CN107153602A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems

Abstract

The invention discloses a kind of device for the log data for handling aol server, including:Acquisition module, the daily record log data for obtaining aol server;Processing module, is handled, the log data after being handled for the log data to the aol server;Memory module, for by the log data Cun Chudao databases after the processing, and to the log data setup time field index after the processing;Module is presented, for being presented from the database log data in the range of query time by external interface.The present invention discloses a kind of method for the log data for handling aol server.Can be by by the real-time log data syn-chronizations of aol server to the other equipment in addition to aol server, realizing the purpose for the log data for obtaining aol server in real time, and then the reparation progress of aol server is improved, improve operating efficiency.

Description

Handle the device and method of the log data of aol server
Technical field
The present invention relates to Internet technical field, more particularly to a kind of log data for handling aol server device and Method.
Background technology
Server can produce a large amount of useful daily record (log) data during operation, and these log data are mainly used to Some access informations are recorded, so as to which reason can be judged according to log data when situation occurs in server, developer are helped Preferably position and repair problem;Or, can also by understanding the analysis of log data the behavior of user so that according to The hobby of user provides the user accurately personalized recommendation.
In daily Server development work, because Server development personnel and O&M engineer can not enter in real time Row communication, after aol server goes wrong, developer asks for log data firstly the need of to O&M engineer, after And carry out case study;This mode can not obtain log data in real time for developer, be taken so as to extend in repair line The progress for device problem of being engaged in, reduces operating efficiency.
The content of the invention
In view of this, the embodiment of the present invention is expected to provide a kind of device and method for the log data for handling aol server, To realize the purpose for the log data for obtaining aol server in real time, the reparation progress of aol server is improved, and then improve work Efficiency.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way:
The present invention provides a kind of device for the log data for handling aol server, and described device includes:
Acquisition module, the daily record log data for obtaining aol server;
Processing module, is handled, the log data after being handled for the log data to the aol server;
Memory module, for by the log data Cun Chudao databases after the processing, and to the log after the processing Data setup time field index;
Module is presented, for being presented from the database log data in the range of query time by external interface.
In such scheme, the acquisition module, specifically for obtaining described by alive messages queue activemq systems The log data of aol server, the log data of the aol server are the File System Watcher machines by the aol server What aol server described in inotify client controls processed was obtained.
In such scheme, the processing module, specifically for inciting somebody to action described by distributed real time computation system storm systems The log data of aol server are filtered, and the log data of the aol server after filtering are generated into set form Log data, the log data of the set form are defined as the log data after the processing;
The set form is the form of the affiliated application module+log specifying informations of time+log rank+log.
In such scheme, described device also includes:
Merging module, the log data for merging the set form according to preparatory condition by the storm systems, The log data of set form after being merged;
The preparatory condition includes at least one following:Prefixed time interval, log data-levels, belonging to log data should With;
The processing module, the processing is defined as also particularly useful for by the log data of the set form after the merging Log data afterwards.
In such scheme, the memory module, specifically for the log data after the processing are used into distributed field system Unite HDFS storage into the log data forms table_log of extra large dupp database Hbase databases, and to the processing after Log data setup time field index.
The present invention provides a kind of method for the log data for handling aol server, and methods described includes:
Obtain the daily record log data of aol server;
Log data to the aol server are handled, the log data after being handled;
By in the log data Cun Chudao databases after the processing, and to the log data setup time words after the processing Segment index;
The log data in the range of query time are presented from the database by external interface.
In such scheme, the log data for obtaining aol server, including:
The log data of the aol server, the online service are obtained by alive messages queue activemq systems The log data of device are the online services as described in the File System Watcher mechanism inotify client controls of the aol server What device was obtained.
In such scheme, the log data to the aol server are handled, the log data after being handled, Including:
The log data of the aol server are filtered by distributed real time computation system storm systems, incited somebody to action The log data of the aol server after filter generate the log data of set form, and the log data of the set form are true It is set to the log data after the processing;
The set form is the form of the affiliated application module+log specifying informations of time+log rank+log.
In such scheme, the log numbers of set form are generated in the log data of the aol server by after filtering After, methods described also includes:
Merge the log data of the set form, consolidating after being merged according to preparatory condition by the storm systems The log data for the formula that fixes;
The preparatory condition includes at least one following:Prefixed time interval, log data-levels, belonging to log data should With;
It is described that the log data of the set form are defined as the log data after the processing, including:
The log data of set form after the merging are defined as to the log data after the processing.
In such scheme, in the log data Cun Chudao databases by after the processing, and to the processing after Log data setup time field index, including:
Log data after the processing are used into distributed file system HDFS storages to extra large dupp database Hbase numbers According in the log data forms table_log in storehouse, and to the log data setup time field index after the processing.
The device and method of the log data for the processing aol server that the embodiment of the present invention is provided, by obtaining on line The log data of server, obtain the real-time log data of aol server;Then the log data to the aol server are carried out Processing, is fixed the log data of form, and is defined as the log data after processing;Again by the log numbers after the processing According to storage into database, to realize the real-time log data syn-chronizations of aol server to other in addition to aol server The purpose of equipment;And to the log data setup time field index after the processing, to facilitate the follow-up lookup to log data; The log data in the range of query time are presented from the database finally by external interface;So just can be by inciting somebody to action The real-time log data syn-chronizations of aol server are realized to the other equipment in addition to aol server and obtain online service in real time The purpose of the log data of device, and then the reparation progress of aol server is improved, improve operating efficiency.
Brief description of the drawings
Fig. 1 is the hardware architecture diagram for an optional mobile terminal for realizing each embodiment of the invention;
Fig. 2 is the communication system architecture schematic diagram that mobile terminal provided in an embodiment of the present invention can be operated;
Fig. 3 handles the flow chart of the embodiment of the method one of the log data of aol server for the present invention;
Fig. 4 handles the flow chart of the embodiment of the method two of the log data of aol server for the present invention;
Fig. 5 handles the log of aol server in the embodiment of the method two for the log data of present invention processing aol server The Organization Chart of data;
Fig. 6 obtains log in the embodiment of the method two for the log data of present invention processing aol server by page URL The schematic diagram of data;
Fig. 7 handles the structural representation of the device embodiment of the log data of aol server for the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation is described.
The mobile terminal of realization each embodiment of the invention is described referring now to accompanying drawing 1.In follow-up description, make With the suffix of such as " module ", " part " or " unit " for representing element only for being conducive to the explanation of the present invention, its Body does not have specific meaning.Therefore, " module " can be used mixedly with " part ".
Mobile terminal can be implemented in a variety of manners.For example, the terminal described in the present invention can include such as moving It is phone, smart phone, notebook computer, digit broadcasting receiver, personal digital assistant (PDA), tablet personal computer (PAD), portable The mobile terminal of formula multimedia player (PMP), guider etc. and such as fixation of numeral TV, desktop computer etc. Terminal.Hereinafter it is assumed that terminal is mobile terminal.However, it will be understood by those skilled in the art that, except being used in particular for moving mesh Element outside, construction according to the embodiment of the present invention can also apply to the terminal of fixed type.
Fig. 1 illustrates for the hardware configuration of the mobile terminal of realization each embodiment of the invention.
Mobile terminal 1 00 can include wireless communication unit 110, audio/video (A/V) input block 120, user's input Unit 130, sensing unit 140, output unit 150, memory 160, interface unit 170, controller 180 and power subsystem 190 Etc..Fig. 1 shows the mobile terminal with various assemblies, it should be understood that being not required for implementing all groups shown Part, can alternatively implement more or less components, the element of mobile terminal will be discussed in more detail below.
Wireless communication unit 110 generally includes one or more assemblies, and it allows mobile terminal 1 00 and wireless communication system Or the radio communication between network.For example, wireless communication unit can include broadcasting reception module 111, mobile communication module 112nd, at least one in wireless Internet module 113, short range communication module 114 and location information module 115.
Broadcasting reception module 111 receives broadcast singal and/or broadcast via broadcast channel from external broadcast management server Relevant information.Broadcast channel can include satellite channel and/or terrestrial channel.Broadcast management server can be generated and sent The broadcast singal and/or broadcast related information generated before the server or reception of broadcast singal and/or broadcast related information And send it to the server of terminal.Broadcast singal can include TV broadcast singals, radio signals, data broadcasting Signal etc..Moreover, broadcast singal may further include the broadcast singal combined with TV or radio signals.Broadcast phase Closing information can also provide via mobile communications network, and in this case, broadcast related information can be by mobile communication mould Block 112 is received.Broadcast singal can exist in a variety of manners, for example, it can be with DMB (DMB) electronics The form of program guide (EPG), the electronic service guidebooks (ESG) of digital video broadcast-handheld (DVB-H) etc. and exist.Broadcast Receiving module 111 can receive signal broadcast by using various types of broadcast systems.Especially, broadcasting reception module 111 Can be wide by using such as multimedia broadcasting-ground (DMB-T), DMB-satellite (DMB-S), digital video Broadcast-hand-held (DVB-H), the Radio Data System of forward link media (MediaFLO@), received terrestrial digital broadcasting integrated service Etc. (ISDB-T) digit broadcasting system receives digital broadcasting.Broadcasting reception module 111, which may be constructed such that, to be adapted to provide for extensively Broadcast the various broadcast systems and above-mentioned digit broadcasting system of signal.Via broadcasting reception module 111 receive broadcast singal and/ Or broadcast related information can be stored in memory 160 (or other types of storage medium).
Mobile communication module 112 sends radio signals to base station (for example, access point, node B etc.), exterior terminal And in server at least one and/or receive from it radio signal.Such radio signal can be logical including voice Talk about signal, video calling signal or the various types of data for sending and/or receiving according to text and/or Multimedia Message.
Wireless Internet module 113 supports the Wi-Fi (Wireless Internet Access) of mobile terminal.The module can be internally or externally It is couple to terminal.Wi-Fi (Wireless Internet Access) technology involved by the module can include WLAN (WLAN) (Wi-Fi), Wibro (WiMAX), Wimax (worldwide interoperability for microwave accesses), HSDPA (high-speed downlink packet access) etc..
Short range communication module 114 is the module for supporting junction service.Some examples of short-range communication technology include indigo plant Tooth TM, radio frequency identification (RFID), Infrared Data Association (IrDA), ultra wide band (UWB), purple honeybee TM etc..
Location information module 115 is the module for checking or obtaining the positional information of mobile terminal.Location information module 115 typical case is global positioning system (GPS).According to current technology, GPS location information module 115, which is calculated, to be come from The range information and correct time information of three or more satellites and for the Information application triangulation of calculating, so that Calculate according to longitude, latitude and highly accurately three-dimensional current location information.Currently, for calculating the side of position and temporal information Position and the error of temporal information that method is calculated using three satellites and by using other satellite correction.This Outside, GPS location information module 115 can be by Continuous plus current location information in real time come calculating speed information.
A/V input blocks 120 are used to receive audio or video signal.A/V input blocks 120 can include the He of camera 121 Microphone 122, the static images that 121 pairs of camera is obtained in video acquisition mode or image capture mode by image capture apparatus Or the view data of video is handled.Picture frame after processing may be displayed on display unit 151.Handled through camera 121 Picture frame afterwards can be stored in memory 160 (or other storage mediums) or be sent out via wireless communication unit 110 Send, two or more cameras 121 can be provided according to the construction of mobile terminal.Microphone 122 can be in telephone calling model, note Sound (voice data) is received via microphone 122 in record pattern, speech recognition mode etc. operational mode, and can be by this The acoustic processing of sample is voice data.Audio (voice) data after processing can be converted in the case of telephone calling model The form output of mobile communication base station can be sent to via mobile communication module 112.Microphone 122 can be implemented various types of Noise eliminate (or suppress) algorithm with eliminate noise that (or suppression) produce during receiving and sending audio signal or Interference.
The order that user input unit 130 can be inputted according to user generates key input data to control each of mobile terminal Plant operation.User input unit 130 allows user to input various types of information, and can include keyboard, metal dome, touch Plate (for example, detection due to being touched caused by resistance, pressure, electric capacity etc. change sensitive component), roller, rocking bar etc. Deng.Especially, when touch pad is superimposed upon on display unit 151 in the form of layer, touch-screen can be formed.
Sensing unit 140 detects the current shooting state of mobile terminal 1 00, (for example, beating for mobile terminal 1 00 is opened or closed Close shooting state), the position of mobile terminal 1 00, user for the presence or absence of contact (that is, touch input) of mobile terminal 1 00, move The acceleration or deceleration movement of the orientation, mobile terminal 1 00 of dynamic terminal 100 and direction etc., and generate mobile whole for controlling The order of the operation at end 100 or signal.For example, when mobile terminal 1 00 is embodied as sliding-type mobile phone, sensing unit 140 It is opening or closing that the sliding-type phone, which can be sensed,.In addition, sensing unit 140 can detect whether power subsystem 190 carries Whether power of powering or interface unit 170 couple with external device (ED).Sensing unit 140 can include proximity transducer 141 will be This is described with reference to touch-screen.
Interface unit 170 is connected the interface that can pass through as at least one external device (ED) with mobile terminal 1 00.For example, External device (ED) can include wired or wireless head-band earphone port, external power source (or battery charger) port, wired or nothing Line FPDP, memory card port, the port for connecting the device with identification module, audio input/output (I/O) end Mouth, video i/o port, ear port etc..Identification module can be that storage is used to verify that user uses each of mobile terminal 1 00 Plant information and subscriber identification module (UIM), client identification module (SIM), Universal Subscriber identification module (USIM) can be included Etc..In addition, the device (hereinafter referred to as " identifying device ") with identification module can take the form of smart card, therefore, know Other device can be connected via port or other attachment means with mobile terminal 1 00.Interface unit 170 can be used for reception and come from The input (for example, data message, electric power etc.) of external device (ED) and the input received is transferred in mobile terminal 1 00 One or more elements can be used for transmitting data between mobile terminal and external device (ED).
In addition, when mobile terminal 1 00 is connected with external base, interface unit 170 may be used as allowing by it by electricity Power provides to the path of mobile terminal 1 00 from base or may be used as allowing passing through it from the various command signals that base is inputted It is transferred to the path of mobile terminal.The various command signals or electric power inputted from base may be used as being used to recognize that mobile terminal is The no signal being accurately fitted within base.Output unit 150 is configured to provide defeated with vision, audio and/or tactile manner Go out signal (for example, audio signal, vision signal, alarm signal, vibration signal etc.).Output unit 150 can include display Unit 151, dio Output Modules 152, alarm unit 153 etc..
Display unit 151 may be displayed on the information handled in mobile terminal 1 00.For example, when mobile terminal 1 00 is in electricity When talking about call mode, display unit 151 can be shown with conversing or other communicating (for example, text messaging, multimedia file Download etc.) related user interface (UI) or graphic user interface (GUI).When mobile terminal 1 00 is in video calling pattern Or during image capture mode, display unit 151 can show the image of capture and/or the image of reception, show video or figure UI or GUI of picture and correlation function etc..
Meanwhile, when display unit 151 and touch pad in the form of layer it is superposed on one another to form touch-screen when, display unit 151 may be used as input unit and output device.Display unit 151 can include liquid crystal display (LCD), thin film transistor (TFT) In LCD (TFT-LCD), Organic Light Emitting Diode (OLED) display, flexible display, three-dimensional (3D) display etc. at least It is a kind of.Some in these displays may be constructed such that transparence to allow user to be watched from outside, and this is properly termed as transparent Display, typical transparent display can be, for example, TOLED (transparent organic light emitting diode) display etc..According to specific Desired embodiment, mobile terminal 1 00 can include two or more display units (or other display devices), for example, moving Dynamic terminal can include outernal display unit (not shown) and inner display unit (not shown).Touch-screen can be used for detection to touch Input pressure and touch input position and touch input area.
Dio Output Modules 152 can mobile terminal be in call signal reception pattern, call mode, logging mode, It is that wireless communication unit 110 is received or in memory 160 when under the isotypes such as speech recognition mode, broadcast reception mode The voice data transducing audio signal of middle storage and it is output as sound.Moreover, dio Output Modules 152 can be provided and movement The audio output (for example, call signal receives sound, message sink sound etc.) for the specific function correlation that terminal 100 is performed. Dio Output Modules 152 can include loudspeaker, buzzer etc..
Alarm unit 153 can provide output to notify event to mobile terminal 1 00.Typical event can be with Including calling reception, message sink, key signals input, touch input etc..In addition to audio or video is exported, alarm unit 153 can provide output in a different manner with the generation of notification event.For example, alarm unit 153 can be in the form of vibration Output is provided, when receiving calling, message or some other entrance communications (Incoming Communication), alarm list Member 153 can provide tactile output (for example, vibration) to notify to user.Exported by providing such tactile, even if When the mobile phone of user is in the pocket of user, user also can recognize that the generation of various events.Alarm unit 153 The output of the generation of notification event can also be provided via display unit 151 or dio Output Modules 152.
Memory 160 can store software program of the processing performed by controller 180 and control operation etc., Huo Zheke Temporarily to store the data that has exported or will export (for example, telephone directory, message, still image, video etc.).And And, memory 160 can store the vibration of various modes on being exported when touching and being applied to touch-screen and audio signal Data.
Memory 160 can include the storage medium of at least one type, and the storage medium includes flash memory, hard disk, many Media card, card-type memory (for example, SD or DX memories etc.), random access storage device (RAM), static random-access storage Device (SRAM), read-only storage (ROM), Electrically Erasable Read Only Memory (EEPROM), programmable read only memory (PROM), magnetic storage, disk, CD etc..Moreover, mobile terminal 1 00 can be with performing memory by network connection The network storage device cooperation of 160 store function.
The overall operation of the generally control mobile terminal of controller 180.For example, controller 180 is performed and voice call, data Communication, video calling etc. related control and processing.In addition, controller 180 can include being used to reproduce (or playback) many matchmakers The multi-media module 181 of volume data, multi-media module 181 can be constructed in controller 180, or can be structured as and control Device 180 is separated.Controller 180 can be with execution pattern identifying processing, by the handwriting input performed on the touchscreen or picture Draw input and be identified as character or image.
Power subsystem 190 receives external power or internal power under the control of controller 180 and provides operation each member Appropriate electric power needed for part and component.
Various embodiments described herein can be with use such as computer software, hardware or its any combination of calculating Machine computer-readable recording medium is implemented.Implement for hardware, embodiment described herein can be by using application-specific IC (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), scene can Programming gate array (FPGA), processor, controller, microcontroller, microprocessor, it is designed to perform function described herein At least one of electronic unit is implemented, and in some cases, such embodiment can be implemented in controller 180. For software implementation, the embodiment of such as process or function can be with allowing to perform the single of at least one function or operation Software module is implemented.Software code can by the software application (or program) write with any appropriate programming language Lai Implement, software code can be stored in memory 160 and be performed by controller 180.
So far, mobile terminal is described according to its function.Below, for the sake of brevity, will description such as folded form, Slide type mobile terminal in various types of mobile terminals of board-type, oscillating-type, slide type mobile terminal etc. is as showing Example.Therefore, the present invention can be applied to any kind of mobile terminal, and be not limited to slide type mobile terminal.
Mobile terminal 1 00 as shown in Figure 1 may be constructed such that using via frame or packet transmission data it is all if any Line and wireless communication system and satellite-based communication system are operated.
The communication system for describing wherein be operated according to the mobile terminal of the present invention referring now to Fig. 2.
Such communication system can use different air interfaces and/or physical layer.For example, used by communication system Air interface includes such as frequency division multiple access (FDMA), time division multiple acess (TDMA), CDMA (CDMA) and universal mobile communications system System (UMTS) (especially, Long Term Evolution (LTE)), global system for mobile communications (GSM) etc..As non-limiting example, under The description in face is related to cdma communication system, but such teaching is equally applicable to other types of system.
With reference to Fig. 2, cdma wireless communication system can include multiple mobile terminal 1s 00, multiple base stations (BS) 270, base station Controller (BSC) 275 and mobile switching centre (MSC) 280.MSC280 is configured to and Public Switched Telephony Network (PSTN) 290 form interface.MSC280 is also structured to the BSC275 formation interfaces with that can be couple to base station 270 via back haul link. Back haul link can be constructed according to any of some known interfaces, the interface include such as E1/T1, ATM, IP, PPP, frame relay, HDSL, ADSL or xDSL.It will be appreciated that system can include multiple BSC275 as shown in Figure 2.
Each BS270 can service one or more subregions (or region), by multidirectional antenna or the day of sensing specific direction Each subregion of line covering is radially away from BS270.Or, each subregion can be by two or more for diversity reception Antenna is covered.Each BS270 may be constructed such that the multiple frequency distribution of support, and each frequency distribution has specific frequency spectrum (for example, 1.25MHz, 5MHz etc.).
What subregion and frequency were distributed, which intersects, can be referred to as CDMA Channel.BS270 can also be referred to as base station transceiver System (BTS) or other equivalent terms.In this case, term " base station " can be used for broadly representing single BSC275 and at least one BS270.Base station can also be referred to as " cellular station ".Or, specific BS270 each subregion can be claimed For multiple cellular stations.
As shown in Figure 2, broadcast singal is sent to the mobile terminal operated in system by broadcsting transmitter (BT) 295 100.Broadcasting reception module 111 as shown in Figure 1 is arranged at mobile terminal 1 00 to receive the broadcast sent by BT295 Signal.In fig. 2 it is shown that several global positioning system (GPS) satellites 300.Satellite 300 helps to position multiple mobile terminals At least one in 100.
In fig. 2, multiple satellites 300 are depicted, it is understood that be, it is possible to use any number of satellite obtains useful Location information.GPS as shown in Figure 1 location information module 115 is generally configured to coordinate to be thought with satellite 300 The location information wanted.Substitute GPS tracking techniques or outside GPS tracking techniques, can use can track mobile terminal Other technologies of position.In addition, at least one gps satellite 300 can optionally or additionally handle satellite dmb transmission.
As a typical operation of wireless communication system, BS270 receives the reverse link from various mobile terminal 1s 00 Signal.Mobile terminal 1 00 generally participates in call, information receiving and transmitting and other types of communication.It is each anti-that certain base station 270 is received Handled to link signal in specific BS270.The data of acquisition are forwarded to the BSC275 of correlation.BSC provides call Resource allocation and the mobile management function of coordination including the soft switching process between BS270.BSC275 is also by the number received According to MSC280 is routed to, it is provided for the extra route service with PSTN290 formation interfaces.Similarly, PSTN290 with MSC280 formation interfaces, MSC280 and BSC275 formation interfaces, and BSC275 correspondingly controls BS270 to believe forward link Number it is sent to mobile terminal 1 00.
Based on above-mentioned mobile terminal hardware configuration and communication system, each embodiment of the inventive method is proposed.
Embodiment one
Fig. 3 handles the flow chart of the embodiment of the method one of the log data of aol server for the present invention;As shown in figure 3, this The method of the log data for the processing aol server that inventive embodiments are provided may include steps of:
Step 301:Obtain the daily record log data of aol server.
In actual applications, can be by the File System Watcher mechanism inotify clients of aol server to being taken on line The log data of business device are monitored in real time;Then, by alive messages queue (active message queue, Activemq) system obtains the real-time log data for the aol server that the inotify client controls are arrived.
Step 302:Log data to the aol server are handled, the log data after being handled.
After the log data of aol server are got by activemq systems, it is by distributed calculate in real time System storm systems are filtered to the log data got, that is, delete useless log data and wrong log data, so The log data after filtering are generated to the log data of set form afterwards, the log data of the set form of generation are to obtain Processing after log data.
Wherein, the set form of the log data of the set form can be:
" the affiliated application module+log specifying informations of time+log rank+log ".
" log ranks " therein can be " alarm (warn) ", " announcement information (info) ", " mistake (error) " grade Not, " the affiliated application modules of log " be used for illustrate the log data be from which application log data, such as, and for from The log data of " weather " application, the affiliated application modules of log are just " weather ".
After by filtering log data generation set form log data after, can also by storm systems according to Preparatory condition is merged to the log data of the set form, the log data of the set form after being merged, to reduce number According to the amount of storage in storehouse;Preparatory condition therein includes at least one following:Prefixed time interval, log data-levels, log data Affiliated application.
For example, by after filtering log data generate set form log data after, can be by storm systems to every Merged every the log data of the set form in T seconds time, the log data of the set form after being merged.
Step 303:By in the log data Cun Chudao databases after the processing, and the log data after the processing are built Field index between immediately.
Handle, after the log data after being handled, this can be handled in the log data to aol server Log data afterwards use distributed file system (Hadoop Distributed File System, HDFS) storage to sea Du In log data forms table_log in general database (Hadoop Database, Hbase), and to the log numbers after the processing According to setup time field index, to facilitate the follow-up lookup to log data.
The time field of the log data after the processing is carried out in descending order for example, can directly invoke and set up index order With regard to line index, it is easiest to be queried to the log data for ensureing newest.
Step 304:The log data in the range of query time are presented from the database by external interface.
In the log data Cun Chudao databases after by processing, and to the log data setup time field ropes after the processing After drawing, the log data in the range of query time can be showed from database by external interface.
, then can be by log log servers from Hbase for example, the log data that will be inquired about in t1 to t2 time ranges The log data inquired about in database in t1 to t2 time ranges, are then presented the log data in the time range inquired Out;Log log servers therein are other internet (web) servers in addition to aol server.
The method of the log data of processing aol server provided in an embodiment of the present invention, by obtaining aol server Log data, obtain the real-time log data of aol server;Then the log data to the aol server are handled, and are obtained To the log data of set form, and it is defined as the log data after processing;Again by the log data Cun Chudao after the processing In database, to realize the mesh of the real-time log data syn-chronizations of aol server to the other equipment in addition to aol server 's;And to the log data setup time field index after the processing, to facilitate the follow-up lookup to log data;Finally by External interface is presented the log data in the range of query time from the database;So as to make developer obtain in real time The log data of aol server are taken, and then improve the reparation progress of aol server, operating efficiency is improved.
Embodiment two
Fig. 4 handles the flow chart of the embodiment of the method two of the log data of aol server for the present invention;As shown in figure 4, this The method of the log data for the processing aol server that inventive embodiments are provided may include steps of:
Step 401:The log data of aol server are obtained by activemq systems.
The server run on general line, i.e. aol server can produce a large amount of useful log data, for recording Access information;Inotify is a kind of change notification mechanism of file system, and the operation of file system can be monitored, than Such as, the events such as the increase, deletion, change of file can be monitored.
, can operation inotify clients be entered come the log data to aol server on server on line in practical application Row monitoring in real time;When the log data of aol server flow increase, inotify can monitor the change, and will be increased Log data are sent to activemq systems;Activemq systems will get the aol server that inotify client controls are arrived Real-time log data.
Step 402:Filtration treatment is carried out to the log data of aol server by storm systems, after being filtered Log data.
Storm systems are a stream data processing systems, and inotify client controls are got in activemq systems After the real-time log data of the aol server arrived, when the log data are flowed into storm systems from activemq systems, Storm systems will carry out filtration treatment to log data, that is, get rid of useless log data and wrong log data, obtain Log data after filtering.
Step 403:Log data after filtering are generated to the log data of set form by storm systems.
Storm systems after the log data of aol server to be carried out to filtration treatment, the log data after being filtered, Log data after this is filtered generate the log data of set form;Wherein, the fixation of the log data of the set form generated Form can be:
" the affiliated application module+log specifying informations of time+log rank+log ".
" time " in the set form is that aol server produces the time of the log data, or could also say that use The access time at family;" log ranks " can be " alarm (warn) ", " announcement information (info) ", " mistake (error) " grade Not, that is, being the type of the log data;" the affiliated application modules of log " represents which application is the log data be from , such as, the log data are produced when browsing news client, then " the affiliated application modules of log " will be " news client End ".
Step 404:Merge the log data of the set form of generation according to preparatory condition by storm systems, merged The log data of set form afterwards.
Storm systems, can be according to default bar after the log data of the log data generation set form after by filtering Part merges the log data of the set form, the log data of the set form after being merged;Preparatory condition therein Can be including at least one in application belonging to prefixed time interval, log data-levels, log data etc.;Such as, can be with every The log data of the set form are merged every prefixed time interval, can also be every prefixed time interval to same log numbers Such as it is all that the log data of the set form of " alarm (warn) " are merged according to rank.
For example, there is the log data of set form as shown in table 1:
Table 1
For the log data of the set form in table 1, it can be obtained to being merged every the data of 30 seconds after merging Set form log data, its result can be as shown in table 2:
Table 2
As can be seen from Table 2, to the log data of the set form in table 1, by 10:12:01-10:12:30 in this 30 seconds Log data merge together, by 10:12:31-10:13:The 00 log data in this 30 seconds merge together, by 10: 13:01-10:13:The 30 log data in this 30 seconds merge together, reduce the amount of storage of database.
For the log data of the set form in table 1, the data of the same log ranks every 30 seconds can also be carried out Merge, the log data of the set form after being merged, its result can be as shown in table 3:
Table 3
As can be seen from Table 3, to the log data of the set form in table 1, by 10:12:01-10:12:30 in this 30 seconds Belong to info ranks log data merge together, belong to warn ranks log data merge together;Will 10:12:31-10:13:The 00 log data that info ranks are belonged in this 30 seconds merge together;By 10:13:00-10: 13:The 30 log data that error ranks are belonged in this 30 seconds merge together;The amount of storage of database can be reduced.
Explanation is needed exist for, be these are only for example, being not intended to limit the present invention.
Step 405:By in the log data Cun Chudao databases of the set form after merging, and to the fixed grating after merging The log data setup time field index of formula.
Hbase databases are a PostgreSQL databases distributed, towards row, and bottom carries out data using HDFS systems Storage;The log data of the set form of generation are merged according to preparatory condition by storm systems, merged After the log data of set form afterwards, the log data of the set form after merging are arrived into Hbase numbers using the storage of HDFS systems According in the table_log forms in storehouse;In the log data of the set form after storing the merging, to the fixation after the merging The log data setup time field index of form, in order to lookup subsequently to the log data of the set form after the merging; Such as, it can directly invoke and set up index order to the time fields of the log data of the set form after merging in descending order just Index, is easiest to be queried to the log data for ensureing newest.
Step 406:By external interface by the log data of the set form after the merging in the range of query time from data Presented in storehouse.
In table_log forms in the log data Cun Chudao Hbase databases of the set form after by merging, and Time field to the log data of the set form after the merging is set up after index, when be able to will be inquired about by external interface Between in the range of log data showed from database;Such as, 23 days 10 December in 2016 will be inquired about:10 to 10:When 30 Between in the range of log data, then the log that can be inquired about by log log servers from Hbase databases in the time range Data, then show the log data in the time range inquired;Log log servers therein are except on line Other web servers beyond server.
In order to clearly embody the purpose of the present invention, make further to illustrate on the basis of above-described embodiment It is bright.
Fig. 5 handles the log of aol server in the embodiment of the method two for the log data of present invention processing aol server The Organization Chart of data;As shown in figure 5, running inotify clients on server 51 (web server on line) on line, work as line During the log data flowing increase of upper server 51, it is possible to use inotify clients are monitored, and increased log data are sent out Give activemq systems;Data are temporarily stored after being flowed into activemq systems, wait the processing of storm systems; Storm systems are filtered the log data that activemq systems flow out, and delete useless log data and wrong log numbers According to it can be the set form of " the affiliated application module+log specifying informations of time+log rank+log " then to generate form Log data, and by the table_log forms in the log data Cun Chudao Hbase databases of the set form;Should in storage During the log data of set form, the time field to the log data of the set form sets up index, such as, can directly invoke and build Lithol draw order to the time field of the log data according to descending with regard to line index, in order to inquiry subsequently to the log data, It ensure that newest log data are easiest to be queried to;Then just can be by log log servers 52 (except web takes on line Other web servers beyond business device) log data are obtained from Hbase databases;In this way, just realizing aol server Real-time log data syn-chronizations to other web servers in addition to web server on line on, so, developer just can be real When required log data are obtained by page URL (Uniform Resource Locator, URL).
In actual applications, after the log data of generation set form, the log data of the set form can also be entered Row merges, such as, the log data within 1 minute can be merged, then by the log numbers of the set form after merging According to storage into Hbase databases, to reduce the amount of storage of database.
Fig. 6 obtains log in the embodiment of the method two for the log data of present invention processing aol server by page URL The schematic diagram of data;As shown in fig. 6, at the real-time log data access systems interface of browser, user can be defeated at time range Enter the period of the log data to be inquired about, such as, can with input inquiry at the beginning of between be 2016-12-23 10:01, terminate Time is 2016-12-23 10:30, " inquiry " button is clicked on, at this moment, log log servers will be used as ginseng using the time of input Number, the log data inquired about from Hbase databases in the period, clicking on " export " button just can be by the log number inquired According to downloading in office software Excel;So, developer just can pass through page in real time using the time of log data as parameter Face URL downloads required log data, very convenient, thus avoid ask for log data to O&M engineer because needing and The appearance of the situation of the progress of server problem in repair line is extended, and then improves operating efficiency.
The method of the log data of processing aol server provided in an embodiment of the present invention, is obtained by activemq systems The log data of aol server;Filtration treatment is carried out to the log data of aol server by storm systems, obtained after filtering Log data;Log data after filtering are generated to the log data of set form by storm systems;Pass through storm system roots Merge the log data of the set form of generation, the log data of the set form after being merged according to preparatory condition;After merging Set form log data Cun Chudao databases in, and to the log data setup time field ropes of the set form after merging Draw;The log data of the set form after the merging in the range of query time are presented from database by external interface;Can By the real-time log data syn-chronizations of aol server to other web servers in addition to aol server, this causes developer Required log data can be downloaded by page URL in real time using the time of log data as parameter, without to fortune Dimension engineer asks for log data, very convenient, so as to improve the reparation progress of aol server, improves operating efficiency.
Embodiment three
Fig. 7 handles the structural representation of the device embodiment of the log data of aol server for the present invention;As shown in fig. 7, The device 07 of the log data for the processing aol server that the present embodiment is provided includes:Acquisition module 71, processing module 72, storage Module 73, presentation module 74;Wherein,
The acquisition module 71, the daily record log data for obtaining aol server;
The processing module 72, is handled, the log after being handled for the log data to the aol server Data;
The memory module 73, for by the log data Cun Chudao databases after the processing, and to the processing after Log data setup time field index;
The presentation module 74, for by external interface by the log data in the range of query time from the database It is middle to present.
Further, the acquisition module 71, specifically for obtaining described by alive messages queue activemq systems The log data of aol server, the log data of the aol server are the File System Watcher machines by the aol server What aol server described in inotify client controls processed was obtained.
Further, the processing module 72, specifically for inciting somebody to action described by distributed real time computation system storm systems The log data of aol server are filtered, and the log data of the aol server after filtering are generated into set form Log data, the log data of the set form are defined as the log data after the processing;
The set form is the form of the affiliated application module+log specifying informations of time+log rank+log.
Further, described device 07 also includes:Merging module 75;Wherein,
The merging module 75, the log for merging the set form according to preparatory condition by the storm systems Data, the log data of the set form after being merged;
The preparatory condition includes at least one following:Prefixed time interval, log data-levels, belonging to log data should With;
The processing module 72, the place is defined as also particularly useful for by the log data of the set form after the merging Log data after reason.
Further, the memory module 73, specifically for the log data after the processing are used into distributed field system Unite HDFS storage into the log data forms table_log of extra large dupp database Hbase databases, and to the processing after Log data setup time field index.
The device of the present embodiment, can be used for perform it is above-mentioned shown in embodiment of the method technical scheme, its realization principle and Technique effect is similar, and here is omitted.
In actual applications, the acquisition module 71, processing module 72, memory module 73, presentation module 74, merging module 75 can be by central processing unit (the Central Processing in the device 07 positioned at the log data of processing aol server Unit, CPU), microprocessor (Micro Processor Unit, MPU), digital signal processor (Digital Signal Processor, DSP) or field programmable gate array (Field Programmable Gate Array, FPGA) etc. realize.
It should be noted that herein, term " comprising ", "comprising" or its any other variant are intended to non-row His property is included, so that process, method, article or device including a series of key elements not only include those key elements, and And also including other key elements being not expressly set out, or also include for this process, method, article or device institute inherently Key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including this Also there is other identical element in process, method, article or the device of key element.
The embodiments of the present invention are for illustration only, and the quality of embodiment is not represented.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Understood based on such, technical scheme is substantially done to prior art in other words Going out the part of contribution can be embodied in the form of software product, and the computer software product is stored in a storage medium In (such as ROM/RAM, magnetic disc, CD), including some instructions are to cause a station terminal equipment (can be mobile phone, computer, clothes It is engaged in device, air conditioner, or network equipment etc.) perform method described by each embodiment of the invention.
The present invention is the flow with reference to method according to embodiments of the present invention, equipment (system) and computer program product Figure and/or block diagram are described.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided The processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce A raw machine so that produced by the instruction of computer or the computing device of other programmable data processing devices for real The device for the function of being specified in present one flow of flow chart or one square frame of multiple flows and/or block diagram or multiple square frames.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which is produced, to be included referring to Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that in meter Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, thus in computer or The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one The step of function of being specified in individual square frame or multiple square frames.
The preferred embodiments of the present invention are these are only, are not intended to limit the scope of the invention, it is every to utilize this hair Equivalent structure or equivalent flow conversion that bright specification and accompanying drawing content are made, or directly or indirectly it is used in other related skills Art field, is included within the scope of the present invention.

Claims (10)

1. a kind of device for the log data for handling aol server, it is characterised in that described device includes:
Acquisition module, the daily record log data for obtaining aol server;
Processing module, is handled, the log data after being handled for the log data to the aol server;
Memory module, for by the log data Cun Chudao databases after the processing, and to the log data after the processing Setup time field index;
Module is presented, for being presented from the database log data in the range of query time by external interface.
2. device according to claim 1, it is characterised in that the acquisition module, specifically for passing through alive messages team The log data that activemq systems obtain the aol server are arranged, the log data of the aol server are on the line What aol server described in the File System Watcher mechanism inotify client controls of server was obtained.
3. device according to claim 1, it is characterised in that the processing module, specifically for by distributed real-time Computing system storm systems are filtered the log data of the aol server, by the aol server after filtering Log data generate the log data of set form, and the log data of the set form are defined as to the log numbers after the processing According to;
The set form is the form of the affiliated application module+log specifying informations of time+log rank+log.
4. device according to claim 3, it is characterised in that described device also includes:
Merging module, the log data for merging the set form according to preparatory condition by the storm systems, is obtained The log data of set form after merging;
The preparatory condition includes at least one following:Prefixed time interval, log data-levels, the application belonging to log data;
The processing module, is defined as after the processing also particularly useful for by the log data of the set form after the merging Log data.
5. device according to claim 1, it is characterised in that the memory module, specifically for by after the processing Log data store the log data forms table_ to extra large dupp database Hbase databases using distributed file system HDFS In log, and to the log data setup time field index after the processing.
6. a kind of method for the log data for handling aol server, it is characterised in that methods described includes:
Obtain the daily record log data of aol server;
Log data to the aol server are handled, the log data after being handled;
By in the log data Cun Chudao databases after the processing, and to the log data setup time field ropes after the processing Draw;
The log data in the range of query time are presented from the database by external interface.
7. method according to claim 6, it is characterised in that the log data of the acquisition aol server, including:
The log data of the aol server are obtained by alive messages queue activemq systems, the aol server Log data are that the aol server as described in the File System Watcher mechanism inotify client controls of the aol server is obtained Arrive.
8. method according to claim 6, it is characterised in that at the log data to the aol server Reason, the log data after being handled, including:
The log data of the aol server are filtered by distributed real time computation system storm systems, after filtering The aol server log data generate set form log data, the log data of the set form are defined as Log data after the processing;
The set form is the form of the affiliated application module+log specifying informations of time+log rank+log.
9. method according to claim 8, it is characterised in that in the log of the aol server by after filtering After the log data of data generation set form, methods described also includes:
Merge the log data of the set form, the fixed grating after being merged according to preparatory condition by the storm systems The log data of formula;
The preparatory condition includes at least one following:Prefixed time interval, log data-levels, the application belonging to log data;
It is described that the log data of the set form are defined as the log data after the processing, including:
The log data of set form after the merging are defined as to the log data after the processing.
10. method according to claim 6, it is characterised in that the log data Cun Chudao data by after the processing In storehouse, and to the log data setup time field index after the processing, including:
Log data after the processing are used into distributed file system HDFS storages to extra large dupp database Hbase databases Log data forms table_log in, and to the log data setup time field index after the processing.
CN201710245653.XA 2017-04-14 2017-04-14 Handle the device and method of the log data of aol server Pending CN107153602A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710245653.XA CN107153602A (en) 2017-04-14 2017-04-14 Handle the device and method of the log data of aol server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710245653.XA CN107153602A (en) 2017-04-14 2017-04-14 Handle the device and method of the log data of aol server

Publications (1)

Publication Number Publication Date
CN107153602A true CN107153602A (en) 2017-09-12

Family

ID=59793327

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710245653.XA Pending CN107153602A (en) 2017-04-14 2017-04-14 Handle the device and method of the log data of aol server

Country Status (1)

Country Link
CN (1) CN107153602A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109189601A (en) * 2018-09-06 2019-01-11 郑州云海信息技术有限公司 The grasping means of RAID card log information under a kind of linux system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1265145A2 (en) * 2001-06-04 2002-12-11 Sony Computer Entertainment Inc. Log collecting/analyzing system with separated functions of collecting log information and analyzing the same
CN103914485A (en) * 2013-01-07 2014-07-09 上海宝信软件股份有限公司 System and method for remotely collecting, retrieving and displaying application system logs
CN105159964A (en) * 2015-08-24 2015-12-16 广东欧珀移动通信有限公司 Log monitoring method and system
CN105868075A (en) * 2016-03-31 2016-08-17 浪潮通信信息系统有限公司 System and method for monitoring and analyzing great deal of logs in real time
CN106340161A (en) * 2016-08-25 2017-01-18 山东联科云计算科技有限公司 Public security early warning system based on big data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1265145A2 (en) * 2001-06-04 2002-12-11 Sony Computer Entertainment Inc. Log collecting/analyzing system with separated functions of collecting log information and analyzing the same
CN103914485A (en) * 2013-01-07 2014-07-09 上海宝信软件股份有限公司 System and method for remotely collecting, retrieving and displaying application system logs
CN105159964A (en) * 2015-08-24 2015-12-16 广东欧珀移动通信有限公司 Log monitoring method and system
CN105868075A (en) * 2016-03-31 2016-08-17 浪潮通信信息系统有限公司 System and method for monitoring and analyzing great deal of logs in real time
CN106340161A (en) * 2016-08-25 2017-01-18 山东联科云计算科技有限公司 Public security early warning system based on big data

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109189601A (en) * 2018-09-06 2019-01-11 郑州云海信息技术有限公司 The grasping means of RAID card log information under a kind of linux system

Similar Documents

Publication Publication Date Title
CN106302385B (en) A kind of message distribution device and method
CN105228210B (en) Multichannel routing method and device
CN105718027B (en) The management method and mobile terminal of background application
CN104735255B (en) Multi-screen display method and system
CN105282245B (en) Cross-server message push system and method
CN104731512B (en) The method, apparatus and terminal that picture is shared
CN105718568B (en) A kind of device and method of the unified interaction of mostly application
CN106941443A (en) One population historical record checks terminal and method
CN106911806A (en) A kind of method of PUSH message, terminal, server and system
CN105897995B (en) A kind of method and apparatus for adding contact information
CN105430258B (en) A kind of method and apparatus of self-timer group photo
CN106789589A (en) One kind shares processing method, device and terminal
CN106909681A (en) A kind of information processing method and its device
CN107426282A (en) A kind of picture loading method, terminal and server
CN105095705B (en) A kind of information processing method and device
CN105049610B (en) A kind of input method and terminal
CN106911486B (en) A kind of message push processing method, apparatus and system
CN105049916A (en) Video recording method and device
CN104915103B (en) The method and mobile terminal of arranging desktop icons
CN107071161A (en) The aggregation display method and mobile terminal of icon in a kind of status bar
CN105118517B (en) A kind of device and method adjusting music rhythm
CN106777251A (en) A kind of file management method and device
CN106776845A (en) A kind of information flow adaptive management method and terminal
CN107153602A (en) Handle the device and method of the log data of aol server
CN105159644B (en) A kind of method of mobile terminal and realization split screen processing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170912