CN108737222A - A kind of server exception method of real-time based on data extraction - Google Patents
A kind of server exception method of real-time based on data extraction Download PDFInfo
- Publication number
- CN108737222A CN108737222A CN201810696873.9A CN201810696873A CN108737222A CN 108737222 A CN108737222 A CN 108737222A CN 201810696873 A CN201810696873 A CN 201810696873A CN 108737222 A CN108737222 A CN 108737222A
- Authority
- CN
- China
- Prior art keywords
- server
- real
- library
- exception
- data extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
Abstract
The present invention discloses a kind of server exception method of real-time based on data extraction, is related to server operation exception and security decision technology, initially sets up operation condition of server area monitoring network, the characterize data of real-time acquisition server operation；Server normal operation pattern base and abnormal patterns library are built using method for detecting abnormality；Then the operation condition of server in region is monitored in real time；Effective characteristic information is extracted by data extraction method, and utilizes method for detecting abnormality reconfiguration mode library；Respectively by characteristic information and normal mode library, abnormal patterns storehouse matching, corresponding modes library is updated according to matching result.The present invention can accomplish the timely discovery and quickly reply of server exception, the technical issues of method for detecting abnormality for solving in the related technology cannot take into account server operation exception, it can dish out the exception of class according to non-program is detected, the number for reducing hand inspection server, reduces the economic loss as caused by server exception.
Description
Technical field
The present invention relates to server operation exception and security decision technology, specifically a kind of clothes based on data extraction
Business device exception method of real-time.
Background technology
The composition of server includes that processor, hard disk, memory, system bus etc. are similar with general computer architecture, but
It is that in processing capacity, stability, reliability, safety, scalability, can be managed due to needing to provide highly reliable service
Rationality etc. is more demanding.For the abnormality detection of server operation, in terms of the stability of safeguard service device and reliability
It is of great significance.Abnormality detection (Anomaly-based detection) method defines one group of system and is in " normal " first
Data when situation, such as cpu busy percentage, memory usage, file verification and and then carry out analysis determine whether exception.
Data extraction be also known as data pick-up, data extraction, refer to collected complex data is started the cleaning processing, and
According to certain feature of data to its tissue and integration into row information, suitable expression-form is obtained to redescribe the number
According to realize effective extraction of data information.
Invention content
The present invention is directed to the demand and shortcoming of current technology development, and it is different to provide a kind of server based on data extraction
Normal method of real-time.
A kind of server exception method of real-time based on data extraction of the present invention solves above-mentioned technical problem and adopts
Technical solution is as follows：The server exception method of real-time based on data extraction, implementation process include：
Step 1 establishes operation condition of server area monitoring network, the characterize data of real-time acquisition server operation；
Normal mode library and the abnormal patterns library of operation condition of server are built using method for detecting abnormality；
Step 2 monitors the operation condition of server in region in real time；
Region server operation data is obtained in real time, and effective characteristic information, and profit are extracted by data extraction method
With method for detecting abnormality reconfiguration mode library；Respectively by characteristic information and normal mode library, abnormal patterns storehouse matching, tied according to matching
Fruit updates corresponding modes library.
Specifically, normal mode library and the abnormal patterns for building operation condition of server using method for detecting abnormality
Library：
The characterize data run to server using data extraction method is handled, and state feature database, root are therefrom extracted
According to state feature database, normal mode library and the abnormal patterns library of operation condition of server are built.
If specifically, characteristic information and normal mode storehouse matching, update normal mode library, and sending normal report to pipe
Reason person.
Specifically, if characteristic information and abnormal patterns storehouse matching, send exception reporting to administrator, list solution party
Case.
Specifically, if characteristic information is mismatched with normal mode library, abnormal patterns library, update abnormal pattern base, and
Forewarning management person.
Specifically, in the step 1, operation condition of server area monitoring network is established, network topology mode is from group
Mode is knitted, the frame of use includes kafka, hadoop, elasticsearch and kibana.
Specifically, in the step 1, the characterize data of the server operation includes the port of server, energy consumption, CPU
Load and data on flows.
A kind of server exception method of real-time based on data extraction of the present invention, has compared with prior art
Advantageous effect be：The present invention is in such a way that data extraction, abnormality detection and area monitoring network are combined, real-time monitoring section
Operation condition of server in domain is suitable for different server clusters；Tentatively judge area by detection service device operation data
Domain server operating status can accomplish the timely discovery and quickly reply of server exception, solve in the related technology
Method for detecting abnormality cannot take into account the technical issues of server operation exception, can dish out the different of class according to non-program is detected
Often, can identify Exception Type in monitoring client and provide solution, reduce hand inspection server number, reduce by
Economic loss caused by server exception.
Description of the drawings
Illustrate the embodiment of the present invention or technology contents in the prior art in order to clearer, below to the embodiment of the present invention
Or required attached drawing does simple introduction in the prior art.It will be apparent that attached drawing disclosed below is only the one of the present invention
Section Example to those skilled in the art without creative efforts, can also be attached according to these
Figure obtains other attached drawings, but within protection scope of the present invention.
Attached drawing 1 is the flow chart of the server exception method of real-time based on data extraction.
Specific implementation mode
The technical issues of to make technical scheme of the present invention, solving and technique effect are more clearly understood, below in conjunction with tool
Body embodiment is checked technical scheme of the present invention, is completely described, it is clear that described embodiment is only this hair
Bright a part of the embodiment, instead of all the embodiments.Based on the embodiment of the present invention, those skilled in the art are not doing
All embodiments obtained under the premise of going out creative work, all within protection scope of the present invention.
Embodiment:
The present embodiment proposes that a kind of server exception method of real-time based on data extraction, implementation process include：
Step 1 establishes operation condition of server area monitoring network, the characterize data of real-time acquisition server operation, packet
Port, energy consumption, cpu load and the data on flows for including real-time pickup area server, using data extraction method to characterizing number
According to being handled；
Under normal circumstances, with the help of staff, monitoring server is evenly distributed in the required region monitored,
Data in 720 hours uninterrupted sampling monitoring regions, are handled data using data extraction method.
Further include using method for detecting abnormality structure operation condition of server normal mode library and abnormal patterns library this
Step；Realize that this purpose operation is as follows：
The characterize data run to server using data extraction method is handled, and state feature database, root are therefrom extracted
According to state feature database, normal mode library and the abnormal patterns library of operation condition of server are built.
Step 2 monitors the operation condition of server in region in real time；Region server operation data is obtained in real time, is led to
It crosses data extraction method and extracts effective characteristic information；
The step for further including using method for detecting abnormality reconfiguration mode library, will realize that this purpose operation is as follows：Pass through
Data extraction method extracts effective characteristic information, and utilizes method for detecting abnormality reconfiguration mode library；Respectively by characteristic information with
Normal mode library, abnormal patterns storehouse matching update corresponding modes library according to matching result.
Server exception method of real-time of the present embodiment based on data extraction in the step 1, establishes server
Operating status area monitoring network, network topology mode be Ad hoc mode, the frame that may be used include kafka, hadoop,
Elasticsearch and kibana etc..Used data extraction method, realization process are：A) de-redundant is carried out to initial data
Remaining and denoising sound preconditioning；B) symbolism is carried out to data according to the variation tendency of data, realizes the compression of data and retouches again
It states；C) feature extraction algorithm is used to carry out characteristic information extraction to the data after redescribing.
In the step 2, method for detecting abnormality reconfiguration mode library, specific implementation process is：A) to obtained time series
Data carry out level label；B) by layering method (Level Representation, LR) to labeled time sequence
Row extract, and obtain hierarchical sequence, and utilize obtained hierarchical sequence to rebuild and be tested range server operating status
Ideograph；C) characterize data run according to server, builds normal mode library and the abnormal patterns library of operation condition of server；
D) sequence mark in the above method and extraction can press required autonomous definition by user.
In order to realize to server effective monitoring management in real time, finally, further include return monitoring information to administrator this
One step, to take corresponding measure.
Attached drawing 1 is the flow chart of server exception method of real-time of the present embodiment based on data extraction, such as 1 institute of attached drawing
Show, the present embodiment the specific implementation process is as follows：
(1) operation condition of server area monitoring network is established；
(2) operation condition of server normal mode library and abnormal patterns library are built；
(3) acquisition server runs characterize data；Effective characteristic information is extracted by data extraction technology；
(4) characteristic information is subjected to characteristic matching with normal mode library and abnormal patterns library respectively；
(5) if characteristic information and normal mode storehouse matching, normal mode library is updated, and sends normal report to management
Member；It is then back to step (3)；
(6) if characteristic information and abnormal patterns storehouse matching, send exception reporting to administrator, list solution；So
Return to step (3) afterwards；
(7) if characteristic information is mismatched with normal mode library, abnormal patterns library, update abnormal pattern base, and early warning
Administrator.
It is the specifically used process of the present embodiment below, by its specifically used operating process, to further appreciate that this reality
The novel design and operation ease for applying the technical solution that example is provided be entirely using operating process：
1) port, energy consumption, cpu load and the flow information of server are obtained；
2) relevant information on Analysis server, detection service device is with the presence or absence of abnormal；
3) when detecting server exception, according to the determining processing method of exception got；
4) when that can not determine whether server is abnormal, according to the server info got, to operation condition of server
It is analyzed；
5) data extraction method is utilized, operation condition of server is analyzed, determining server whether there is exception, if
Determining server is deposited when abnormal, according to server info, determines processing method.
Use above specific case elaborates the principle of the present invention and embodiment, these embodiments are
It is used to help understand core of the invention technology contents, the protection domain being not intended to restrict the invention, technical side of the invention
Case is not limited in above-mentioned specific implementation mode.Based on the above-mentioned specific embodiment of the present invention, those skilled in the art
Without departing from the principle of the present invention, any improvement and modification to made by the present invention should all fall into the special of the present invention
Sharp protection domain.
Claims (7)
1. a kind of server exception method of real-time based on data extraction, which is characterized in that implementation process includes：
Step 1 establishes operation condition of server area monitoring network, the characterize data of real-time acquisition server operation；
Normal mode library and the abnormal patterns library of operation condition of server are built using method for detecting abnormality；
Step 2 monitors the operation condition of server in region in real time；
Region server operation data is obtained in real time, effective characteristic information is extracted by data extraction method, and utilize different
Normal detection method reconfiguration mode library；Respectively by characteristic information and normal mode library, abnormal patterns storehouse matching, more according to matching result
New corresponding pattern base.
2. a kind of server exception method of real-time based on data extraction according to claim 1, described using abnormal
Detection method builds normal mode library and the abnormal patterns library of operation condition of server：
The characterize data run to server using data extraction method is handled, and state feature database is therefrom extracted, according to shape
State feature database builds normal mode library and the abnormal patterns library of operation condition of server.
3. a kind of server exception method of real-time based on data extraction according to claim 2, which is characterized in that if
Characteristic information and normal mode storehouse matching then update normal mode library, and send normal report to administrator.
4. a kind of server exception method of real-time based on data extraction according to claim 2, which is characterized in that if
Characteristic information and abnormal patterns storehouse matching, then send exception reporting to administrator, list solution.
5. a kind of server exception method of real-time based on data extraction according to claim 2, which is characterized in that if
Characteristic information is mismatched with normal mode library, abnormal patterns library, then update abnormal pattern base, and forewarning management person.
6. a kind of server exception method of real-time based on data extraction according to claim 1, which is characterized in that institute
It states in step 1, establishes operation condition of server area monitoring network, network topology mode is Ad hoc mode, the frame of use
Including kafka, hadoop, elasticsearch and kibana.
7. a kind of server exception method of real-time based on data extraction according to claim 1, which is characterized in that institute
It states in step 1, the characterize data of the server operation includes port, energy consumption, cpu load and the data on flows of server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810696873.9A CN108737222A (en) | 2018-06-29 | 2018-06-29 | A kind of server exception method of real-time based on data extraction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810696873.9A CN108737222A (en) | 2018-06-29 | 2018-06-29 | A kind of server exception method of real-time based on data extraction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108737222A true CN108737222A (en) | 2018-11-02 |
Family
ID=63931162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810696873.9A Pending CN108737222A (en) | 2018-06-29 | 2018-06-29 | A kind of server exception method of real-time based on data extraction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108737222A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274089A (en) * | 2020-01-15 | 2020-06-12 | 福建奇点时空数字科技有限公司 | Server abnormal behavior perception system based on bypass technology |
| CN112235273A (en) * | 2020-09-30 | 2021-01-15 | 福建奇点时空数字科技有限公司 | Server abnormal behavior monitoring system based on BS structure |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778259A (en) * | 2016-12-28 | 2017-05-31 | 北京明朝万达科技股份有限公司 | A kind of abnormal behaviour based on big data machine learning finds method and system |
| CN107241342A (en) * | 2017-06-30 | 2017-10-10 | 北京奇安信科技有限公司 | A kind of network attack crosstalk detecting method and device |
| CN107273273A (en) * | 2017-06-27 | 2017-10-20 | 郑州云海信息技术有限公司 | A kind of distributed type assemblies hardware fault method for early warning and system |
| CN108008332A (en) * | 2017-11-29 | 2018-05-08 | 国网山东省电力公司电力科学研究院 | A kind of new energy Remote testing device method for diagnosing faults based on data mining |
| US20180157249A1 (en) * | 2015-06-05 | 2018-06-07 | Hitachi, Ltd. | Abnormality Detecting Apparatus |
-
2018
- 2018-06-29 CN CN201810696873.9A patent/CN108737222A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180157249A1 (en) * | 2015-06-05 | 2018-06-07 | Hitachi, Ltd. | Abnormality Detecting Apparatus |
| CN106778259A (en) * | 2016-12-28 | 2017-05-31 | 北京明朝万达科技股份有限公司 | A kind of abnormal behaviour based on big data machine learning finds method and system |
| CN107273273A (en) * | 2017-06-27 | 2017-10-20 | 郑州云海信息技术有限公司 | A kind of distributed type assemblies hardware fault method for early warning and system |
| CN107241342A (en) * | 2017-06-30 | 2017-10-10 | 北京奇安信科技有限公司 | A kind of network attack crosstalk detecting method and device |
| CN108008332A (en) * | 2017-11-29 | 2018-05-08 | 国网山东省电力公司电力科学研究院 | A kind of new energy Remote testing device method for diagnosing faults based on data mining |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274089A (en) * | 2020-01-15 | 2020-06-12 | 福建奇点时空数字科技有限公司 | Server abnormal behavior perception system based on bypass technology |
| CN111274089B (en) * | 2020-01-15 | 2022-06-17 | 福建奇点时空数字科技有限公司 | Server abnormal behavior perception system based on bypass technology |
| CN112235273A (en) * | 2020-09-30 | 2021-01-15 | 福建奇点时空数字科技有限公司 | Server abnormal behavior monitoring system based on BS structure |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104796273B (en) | A kind of method and apparatus of network fault root diagnosis | |
| CN103200050B (en) | The hardware state monitoring method and system of server | |
| CN106371986A (en) | Log treatment operation and maintenance monitoring system | |
| JP6144408B2 (en) | Energy measurement information labeling system and energy measurement information labeling server | |
| Nováczki | An improved anomaly detection and diagnosis framework for mobile network operators | |
| CN105071954B (en) | Resource pool fault diagnosis and location processing method based on probe technique | |
| CN102123044A (en) | Detection device and method of network topology consistency based on topology discovery technology | |
| CN106656682A (en) | Method, system and device for detecting cluster heartbeat | |
| CN109034580B (en) | Information system overall health degree evaluation method based on big data analysis | |
| CN107704359A (en) | A kind of monitoring system of big data platform | |
| CN107390628B (en) | Distribution status monitoring and method for early warning and system | |
| CN102420700A (en) | Network fault diagnosis system | |
| CN104574219A (en) | System and method for monitoring and early warning of operation conditions of power grid service information system | |
| CN111179109A (en) | Electricity consumption data processing method for detecting elderly people living alone | |
| CN113282635A (en) | Micro-service system fault root cause positioning method and device | |
| CN108737222A (en) | A kind of server exception method of real-time based on data extraction | |
| CN103049365B (en) | Information and application resource running state monitoring and evaluation method | |
| CN110162445A (en) | The host health assessment method and device of Intrusion Detection based on host log and performance indicator | |
| CN110224889B (en) | Ammeter service monitoring method based on service logic consistency | |
| CN104092730A (en) | Cloud computing system | |
| CN107305487A (en) | One kind virtualizes assistant's management software system based on vmware | |
| CN109067000A (en) | One kind matching electricity consumption intelligence load monitoring system | |
| CN110968479A (en) | Business-level full-link monitoring method for application program and server | |
| CN102932170B (en) | Network element load inequality detection processing method, device and system thereof | |
| CN103713976B (en) | Signalling arrangement fault rootstock searching method for centralized signal supervision system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190312 Address after: Room 411, 238, Jiangchang Third Road, Zhabei District, Shanghai, 200436 Applicant after: Shanghai wave Cloud Computing Service Co., Ltd. Address before: 250100 S06 Floor, No. 1036 Tidal Road, Jinan High-tech Zone, Shandong Province Applicant before: Shandong Hui Trade Electronic Port Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200826 Address after: 214029 No. 999 Gaolang East Road, Binhu District, Wuxi City, Jiangsu Province (Software Development Building) 707 Applicant after: Chaozhou Zhuoshu Big Data Industry Development Co.,Ltd. Address before: 200436, room 238, No. three, 411 Shanghai Road, Zhabei District, China Applicant before: SHANGHAI INSPUR CLOUD CALCULATING SERVICE Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |