CN106817262A - A kind of log analysis device - Google Patents
A kind of log analysis device Download PDFInfo
- Publication number
- CN106817262A CN106817262A CN201710117244.1A CN201710117244A CN106817262A CN 106817262 A CN106817262 A CN 106817262A CN 201710117244 A CN201710117244 A CN 201710117244A CN 106817262 A CN106817262 A CN 106817262A
- Authority
- CN
- China
- Prior art keywords
- module
- data
- analysis
- log information
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/22—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention discloses a kind of log analysis device, including：Data collection module：Related log information is collected from each node of management platform, data filtering is carried out to log information, and being sent to data memory module and data analysis module after different types of log information consolidation form；Data memory module：The log information that data collection module sends is preserved in real time；Data analysis module：The log information that data collection module sends is analyzed by treatment by analyzer, and analysis processing result is sent to visualization model；Visualization model：The analysis processing result of data analysis module is presented by human-computer interaction interface.The present invention better adapts to multi-node environment, solves the problems, such as that all kinds data storage is difficult and reading efficiency is not high, moreover it is possible to effectively lift Consumer's Experience by display interface.
Description
Technical field
The present invention relates to cloud computing Log Analysis System.
Background technology
With the development of computer network, distributed computing technology etc., the use of cloud computing technology is more and more extensive.Cloud computing ring
Under border, the event and running status occurred come each node in record system by daily record are transported according to daily record to system
Dimension management and debugging.This mode is only applicable to the fewer system of interstitial content, for large-scale clustered node, this side
Method is not too much applicable.
The content of the invention
To solve the above problems, the present invention provides a kind of log analysis device that can better adapt to multi-node environment.
The technical scheme is that：A kind of log analysis device, including：
Data collection module：Related log information is collected from each node of management platform, data filtering is carried out to log information,
And being sent to data memory module and data analysis module after different types of log information consolidation form；
Data memory module：The log information that data collection module sends is preserved in real time；
Data analysis module：The log information that data collection module sends is analyzed by treatment by analyzer, and will be divided
Analysis result is sent to visualization model；
Visualization model：The analysis processing result of data analysis module is presented by human-computer interaction interface.
Further, data memory module includes the indexing units for searching.
Further, visualization model shows analysis processing result in graphical form.
Further, also including message queue module and data research and application module；
The data collection module sends to message queue module log information；
The processing pressure of the data analysis monitoring module monitors data analysis module, when data analysis monitoring module monitors to number
When being less than predetermined threshold value according to analysis module processing pressure, the log information hair that message queue module sends data collection module
Deliver to data analysis module；When data analysis monitoring module monitors to data analysis module processing pressure exceed predetermined threshold value,
The log information that data collection module sends is cached in message queue module.
Further, also including database；The data analysis module stores to database analysis processing result.
The log analysis device that the present invention is provided, better adapts to multi-node environment, solves all kinds data storage
Difficult and reading efficiency problem not high, moreover it is possible to which Consumer's Experience is effectively lifted by display interface.
Brief description of the drawings
Fig. 1 is specific embodiment of the invention structural representation.
Specific embodiment
The present invention will be described in detail below in conjunction with the accompanying drawings and by specific embodiment, and following examples are to the present invention
Explanation, and the invention is not limited in implementation below.
As shown in figure 1, the log analysis device that the present invention is provided, including：
Data collection module 1：Related log information is collected from each node of management platform, data mistake is carried out to log information
Filter, and being sent after different types of log information consolidation form to data memory module 2 and data analysis module 5.
Data memory module 2：The log information that data collection module 1 sends is preserved in real time.
Data analysis module 5：The log information that data collection module 1 sends is analyzed by treatment by analyzer,
And send to visualization model 6 analysis processing result；In order to improve the system expandability, implementation personnel can be according to connecing for providing
Cause for gossip shows the self-defined of analyzer, meets actual demand.
Visualization model 6：The analysis processing result of data analysis module 5 is presented by human-computer interaction interface.And visualization
Module 6 shows that Consumer's Experience is directly perceived in graphical form by structure is analyzed and processed.
In the present embodiment, data memory module 2 is additionally provided with indexing units, facilitates user to search information needed.
This analytical equipment also includes message queue module 3 and data research and application module 4.Data collection module 1 is by daily record
Information is sent to message queue module 3, the processing pressure of the Analysis on monitoring data module 5 of data analysis monitoring modular 4, when data point
When analysis monitoring modular 4 monitors that the processing pressure of data analysis module 5 is less than predetermined threshold value, message queue module 3 is by Data Collection
The log information that module 1 sends is sent to data analysis module 5；When data analysis monitoring modular 4 monitors data analysis mould
When the processing pressure of block 5 exceedes predetermined threshold value, the log information that data collection module 1 sends is cached in message queue module 3.
In this analytical equipment, data analysis module 5 stores to database 7 analysis processing result, is easy to persist.
Disclosed above is only the preferred embodiment of the present invention, but the present invention is not limited to this, any this area
What technical staff can think does not have creative change, and some improvement made without departing from the principles of the present invention and
Retouching, should all be within the scope of the present invention.
Claims (5)
1. a kind of log analysis device, it is characterised in that including：
Data collection module：Related log information is collected from each node of management platform, data filtering is carried out to log information,
And being sent to data memory module and data analysis module after different types of log information consolidation form；
Data memory module：The log information that data collection module sends is preserved in real time；
Data analysis module：The log information that data collection module sends is analyzed by treatment by analyzer, and will be divided
Analysis result is sent to visualization model；
Visualization model：The analysis processing result of data analysis module is presented by human-computer interaction interface.
2. log analysis device according to claim 1, it is characterised in that data memory module includes the rope for searching
Draw unit.
3. log analysis device according to claim 1, it is characterised in that visualization model is by analysis processing result scheming
Sheet form shows.
4. log analysis device according to claim 1, it is characterised in that also including message queue module and data analysis
Monitoring modular；
The data collection module sends to message queue module log information；
The processing pressure of the data analysis monitoring module monitors data analysis module, when data analysis monitoring module monitors to number
When being less than predetermined threshold value according to analysis module processing pressure, the log information hair that message queue module sends data collection module
Deliver to data analysis module；When data analysis monitoring module monitors to data analysis module processing pressure exceed predetermined threshold value,
The log information that data collection module sends is cached in message queue module.
5. the log analysis device according to claim 1 or 4, it is characterised in that also including database；The data analysis
Module stores to database analysis processing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710117244.1A CN106817262A (en) | 2017-03-01 | 2017-03-01 | A kind of log analysis device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710117244.1A CN106817262A (en) | 2017-03-01 | 2017-03-01 | A kind of log analysis device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106817262A true CN106817262A (en) | 2017-06-09 |
Family
ID=59114476
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710117244.1A Pending CN106817262A (en) | 2017-03-01 | 2017-03-01 | A kind of log analysis device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106817262A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948280A (en) * | 2017-11-24 | 2018-04-20 | 无锡南理工新能源电动车科技发展有限公司 | The monitoring system of point and mirror image spectral fluxes is visited in a kind of combination |
| CN109308329A (en) * | 2018-09-27 | 2019-02-05 | 深圳供电局有限公司 | Log collection method and device based on cloud platform |
| CN110780867A (en) * | 2019-10-09 | 2020-02-11 | 北京网聘咨询有限公司 | Development tool-oriented graphical log presentation method |
| CN111177360A (en) * | 2019-12-16 | 2020-05-19 | 中国电子科技网络信息安全有限公司 | Self-adaptive filtering method and device based on user logs on cloud |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103412893A (en) * | 2013-07-24 | 2013-11-27 | 广东电子工业研究院有限公司 | Collecting system and collecting method of logs |
| CN104104734A (en) * | 2014-08-04 | 2014-10-15 | 浪潮(北京)电子信息产业有限公司 | Log analysis method and device |
| CN105224445A (en) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN105471635A (en) * | 2015-11-18 | 2016-04-06 | 中国建设银行股份有限公司 | Method, device and system for processing system logs |
| CN105634834A (en) * | 2014-10-27 | 2016-06-01 | 青岛金讯网络工程有限公司 | Data monitoring system and method |
| CN105630869A (en) * | 2015-12-15 | 2016-06-01 | 北京奇虎科技有限公司 | Voice data storage method and device |
| CN106294091A (en) * | 2016-08-11 | 2017-01-04 | 福建富士通信息软件有限公司 | A kind of without intrusive mood daily record interception method for analyzing performance and system |
-
2017
- 2017-03-01 CN CN201710117244.1A patent/CN106817262A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103412893A (en) * | 2013-07-24 | 2013-11-27 | 广东电子工业研究院有限公司 | Collecting system and collecting method of logs |
| CN104104734A (en) * | 2014-08-04 | 2014-10-15 | 浪潮(北京)电子信息产业有限公司 | Log analysis method and device |
| CN105634834A (en) * | 2014-10-27 | 2016-06-01 | 青岛金讯网络工程有限公司 | Data monitoring system and method |
| CN105224445A (en) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN105471635A (en) * | 2015-11-18 | 2016-04-06 | 中国建设银行股份有限公司 | Method, device and system for processing system logs |
| CN105630869A (en) * | 2015-12-15 | 2016-06-01 | 北京奇虎科技有限公司 | Voice data storage method and device |
| CN106294091A (en) * | 2016-08-11 | 2017-01-04 | 福建富士通信息软件有限公司 | A kind of without intrusive mood daily record interception method for analyzing performance and system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948280A (en) * | 2017-11-24 | 2018-04-20 | 无锡南理工新能源电动车科技发展有限公司 | The monitoring system of point and mirror image spectral fluxes is visited in a kind of combination |
| CN109308329A (en) * | 2018-09-27 | 2019-02-05 | 深圳供电局有限公司 | Log collection method and device based on cloud platform |
| CN110780867A (en) * | 2019-10-09 | 2020-02-11 | 北京网聘咨询有限公司 | Development tool-oriented graphical log presentation method |
| CN111177360A (en) * | 2019-12-16 | 2020-05-19 | 中国电子科技网络信息安全有限公司 | Self-adaptive filtering method and device based on user logs on cloud |
| CN111177360B (en) * | 2019-12-16 | 2022-04-22 | 中国电子科技网络信息安全有限公司 | Self-adaptive filtering method and device based on user logs on cloud |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106371975B (en) | A kind of O&M automation method for early warning and system | |
| CN106817262A (en) | A kind of log analysis device | |
| CN109543067A (en) | Enterprise's production status based on artificial intelligence monitors analysis system in real time | |
| CN106817253A (en) | The monitor in real time of journal file and the method and system of alarm | |
| CN107480027A (en) | A kind of distributed deep learning operational system | |
| CN115237857A (en) | Log processing method and device, computer equipment and storage medium | |
| CN107391606A (en) | Log processing method and device based on Storm | |
| CN110414229A (en) | Operational order detection method, device, computer equipment and storage medium | |
| CN109800133A (en) | A kind of method, one-stop monitoring alarm platform and the system of unified monitoring alarm | |
| CN104391990A (en) | Multi-task type collecting and harvesting method based on vertical industry | |
| CN110851317A (en) | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment | |
| CN110677271A (en) | Big data alarm method, device, equipment and storage medium based on ELK | |
| CN110647762A (en) | Government affair supervising platform based on cloud desktop | |
| CN208046653U (en) | A kind of electric power monitoring system network security monitoring main website plateform system | |
| CN103780660A (en) | GPU cluster monitoring system and method for issuing monitoring alarm | |
| CN108334550A (en) | A kind of data processing system based on Human Resource Management System | |
| CN206149326U (en) | Data acquisition analysis monitoring system | |
| CN107766497A (en) | The method and terminal of Data Collection based on container | |
| CN107665260A (en) | A kind of log collection instrument based on Linux system | |
| CN107682201A (en) | Environmental sanitation platform O＆M monitoring system | |
| CN107729218A (en) | A kind of system and method for monitoring processing computing resource equipment | |
| CN114598731B (en) | Cluster log acquisition method, device, equipment and storage medium | |
| CN105072161A (en) | Application program management system based on cloud computing | |
| CN105446707A (en) | Data conversion method | |
| CN114003599A (en) | Processing system, system training method and image labeling method for massive medical images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170609 |