CN111209158A - Mining monitoring method and cluster monitoring system for server cluster - Google Patents
Mining monitoring method and cluster monitoring system for server cluster Download PDFInfo
- Publication number
- CN111209158A CN111209158A CN201911351810.0A CN201911351810A CN111209158A CN 111209158 A CN111209158 A CN 111209158A CN 201911351810 A CN201911351810 A CN 201911351810A CN 111209158 A CN111209158 A CN 111209158A
- Authority
- CN
- China
- Prior art keywords
- server
- mining
- attribute
- monitoring
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses a method for monitoring mine excavation of a server cluster, which comprises the following steps: generating a decision tree by using the collected server operation index data and the attribute influencing the ore excavation judgment; and subsequently, automatically judging whether the server is excavated according to the operation condition of the server monitored by the decision tree. The method and the device at least aim to automatically judge whether the server under the monitoring of the cluster monitoring system is excavated.
Description
Technical Field
The application relates to the technical field of excavation detection processing, in particular to an excavation monitoring method and a cluster monitoring system of a server cluster.
Background
With the explosion of Bingzi and blockchain technologies, more and more people and companies are joining the mining line, a large number of computers are used to mine, mining machines are also created that are dedicated to mine, and a batch of mining trojans also appear that illegally penetrate into the clusters of servers where personal computers and security measures are not in place. High performance computing clusters are easy targets for mining trojans due to their high computing power and the large benefits gained by infiltrated trojan distributors.
The cluster excavated influences the services which the cluster should normally provide, a large amount of calculation causes high electricity charge expenditure and causes economic damage to cluster operators, and therefore, it is necessary to find the excavated cluster in time and clear the excavated trojans.
The main means for detecting the excavation are as follows: manually discovering by experience, detecting plug-in based on the mining of a browser, matching whether a mining script exists through a preset rule, and the like.
The efficiency is too low and the workload is too large through manual discovery by experience.
The mining detection plug-in based on the browser is a personal user PC which is not suitable for clustering.
Although the method of matching whether the ore excavation script exists through the preset rule can detect the ore excavation trojan, each server needs to be started one by one for detection, the automation degree is not enough, once the ore excavation script is upgraded, the preset rule set needs to be upgraded at the same time, and otherwise, the ore excavation script becomes invalid quickly.
The block chain technology and the digital currency have not been long enough, and the research on mining trojans in the industry is not sufficient, so that the defects of multiple manual operations, low automation degree and untimely upgrading exist.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a server cluster mining monitoring method, which uses collected server operation index data and attributes influencing mining judgment to generate a decision tree; and subsequently, automatically judging whether the server is excavated according to the operation condition of the server monitored by the decision tree.
The technical scheme of the application is realized as follows:
the utility model provides a server cluster mine excavation monitoring method, which comprises the following steps:
generating a decision tree by using the collected server operation index data and the attribute influencing the ore excavation judgment;
and subsequently, automatically judging whether the server is excavated according to the operation condition of the server monitored by the decision tree.
According to the embodiment of the application, the server operation index data comprises at least one of server name, collection time, CPU index and process index.
According to an embodiment of the application, generating a decision tree comprises: processing the input of the process function through the process function and generating an output; the output is a decision tree; inputting a training set and an attribute set; the training set is a set formed based on the server operation index data, and is defined as D { (x1, y1), (x2, y2), … (xm, ym) }; the attribute set is a set formed based on the attributes that influence the mining determination, and is defined as a ═ { a1, a2, … ad }.
According to an embodiment of the application, the set of attributes includes: the process name with the highest CPU utilization rate, the user to which the process with the highest CPU utilization rate belongs, the time period of collection and the subsection interval in which the CPU utilization rate is located.
According to the embodiment of the application, the section interval in which the CPU utilization rate is located refers to the step of dividing the CPU utilization rate into different intervals according to the high, medium and low levels and converting a continuous value into a discrete value.
According to an embodiment of the application, an input of a process function is processed by a process function and an output is generated, the process function is a recursive function defined as treeGenerator (D, A), and the process function includes: generating a node; if all the samples in the D belong to the C class, marking the node as a C class leaf node, and returning in a recursive manner; if A is an empty set or the values of the samples in D in A are the same, marking the node as the class with the maximum number of samples in D to become a leaf node, and recursively returning the D samples as the current node.
According to an embodiment of the application, processing an input of a process function by the process function and generating an output further comprises: selecting an attribute value a from A, forming a sample subset Dv by the samples with the attribute value of the samples in D as the selected attribute value a, if the sample subset Dv is not null, using TreeGenerator (Dv, A { a }) as a branch node, if the sample subset Dv is null, marking the branch node as the leaf node with the maximum number of samples in D, and the samples in D as a parent node, and recursively returning; and repeating the selection of an attribute value from A until all values in A are selected, and outputting a decision tree taking the node as a root node by the method.
Decision trees, also called decision trees, are a common class of machine learning methods, and the basic algorithm is as follows:
according to the embodiment of the application, when the situation that the novel mining Trojan horse is mistakenly found or appears is judged, inaccurate results are marked and are brought into a training set, and the decision tree is regenerated.
According to the embodiment of the application, the judgment result of whether the server is excavated sends an alarm to operation and maintenance personnel.
The beneficial technical effect of this application lies in:
according to the method and the system, whether the server under the monitoring of the cluster monitoring system is dug can be automatically judged. Because the cluster monitoring system can continuously run, the detection timeliness is good.
When the situation that the novel mining trojan is mistaken or appears is judged, inaccurate results can be marked again and brought into training set, the decision tree model is regenerated, the detection accuracy is higher along with the increase of the marking times, and the model forms the automatic learning capacity.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a flowchart of a method for monitoring mining of a server cluster according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person having ordinary skill in the art, including the cluster monitoring system in combination with other classification algorithms in machine learning, belong to the scope of protection of the present application.
According to an embodiment of the application, a method for monitoring mining of a server cluster is provided. Fig. 1 shows a flowchart of a method for monitoring mining of a server cluster according to an embodiment of the present application.
And selecting a test cluster, wherein the mining trojans run on a plurality of servers, and the rest servers run normally.
The collection server operates on the index data as in step S10 of fig. 1. The cluster monitoring system is used for collecting the operation information of all the servers at regular time, and the operation information contains the servers which are normally operated and excavated, and the collection of the operation index data of the servers comprises the following steps: the method comprises the following steps of collecting at least one of time, server names, average CPU utilization rate, utilization rate of each CPU core, CPU utilization rate of a process and affiliated users of the process.
And (3) arranging all the collected server operation index data into a format required by a decision tree, marking whether each server is excavated, and adding the server into a training set.
The determination at step S10 of fig. 1 affects the attributes of the excavation determination. Determining attributes that affect mining decisions, including: adding the process name with the highest CPU utilization rate, the user to which the process with the highest CPU utilization rate belongs, the time period of collection, and the subsection interval (dividing the CPU utilization rate into different intervals according to the high, medium and low levels, and converting the continuous value into the discrete value) in which the CPU utilization rate is positioned into the attribute set.
And applying a decision tree algorithm to generate a decision tree by using the training set and the attribute set generated in the above steps, such as generating the decision tree at step S10 in fig. 1.
And (4) accessing the decision tree into the monitoring system, automatically judging whether the server is excavated according to the subsequently monitored server operation index, and sending an alarm through an alarm module of the monitoring system after the judgment as shown in step S20 in the figure 1.
And when the detection is judged to be wrong, marking the judgment result again, bringing the marked data into the training set again, and regenerating the decision tree, namely upgrading the detection model.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method for monitoring mining of a server cluster is characterized by comprising the following steps:
generating a decision tree by using the collected server operation index data and the attribute influencing the ore excavation judgment;
and subsequently, automatically judging whether the server is excavated according to the operation condition of the server monitored by the decision tree.
2. The method of monitoring mining of a server cluster according to claim 1, characterized by: the server operation index data comprises at least one of server name, acquisition time, CPU index and process index.
3. The method of monitoring mining of a server cluster of claim 1, wherein generating the decision tree comprises:
and processing the input of the process function through the process function and generating output, wherein the output is a decision tree, the input comprises a training set and an attribute set, the training set is a set formed on the basis of the server operation index data, and the attribute set is a set formed on the basis of the attributes influencing the mining judgment.
4. The method of monitoring mining of a server cluster of claim 3, wherein the set of attributes comprises: the process name with the highest CPU utilization rate, the user to which the process with the highest CPU utilization rate belongs, the time period of collection and the subsection interval in which the CPU utilization rate is located.
5. The method of monitoring mining of a server cluster according to claim 4, characterized by: the section interval where the CPU utilization rate is located refers to dividing the CPU utilization rate into different intervals according to the high, medium and low levels, and converting a continuous value into a discrete value.
6. The method of claim 3, wherein the processing function inputs are processed by a process function and generates an output, the process function being a recursive function comprising:
generating a node;
if all the samples in the training set belong to the first category, generating leaf nodes marked as the first category, and returning recursively;
and if the attribute set is an empty set or the values of the samples in the training set in the attribute set are the same, marking the samples in the training set as leaf nodes of a second category, and returning in a recursive manner, wherein the samples in the training set are current nodes, and the second category is the category with the largest number of samples in the training set.
7. The method of monitoring mining of a server cluster of claim 6, wherein an input of a process function is processed by the process function and an output is generated, the process function being a recursive function, further comprising:
selecting an attribute value from the attribute set, forming a sample subset by samples of which the attribute values of the samples in the training set are the selected attribute values, if the sample subset is not empty, generating a branch node for the node, if the sample subset is empty, marking the branch node as a leaf node of the second category, and returning in a recursive manner, wherein the node is a father node, and the second category is the category with the largest number of samples in the training set;
and repeatedly selecting an attribute value from the attribute set until all values of the attribute set are selected, and outputting a decision tree taking the node as a root node by the method.
8. A method of monitoring mining of a server cluster according to any of claims 3 to 7, characterized by: and when the situation that the novel mining Trojan horse is mistakenly found or appears is judged, marking inaccurate results, bringing the inaccurate results into a training set, and regenerating the decision tree.
9. The method of monitoring mining of a server cluster according to claim 1, characterized by: and sending an alarm to operation and maintenance personnel according to the judgment result of whether the server is excavated.
10. A cluster monitoring system for monitoring mining of a cluster of servers, comprising: a storage medium storing a program executed to implement the excavation monitoring method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911351810.0A CN111209158B (en) | 2019-12-25 | 2019-12-25 | Mining monitoring method and cluster monitoring system for server cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911351810.0A CN111209158B (en) | 2019-12-25 | 2019-12-25 | Mining monitoring method and cluster monitoring system for server cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111209158A true CN111209158A (en) | 2020-05-29 |
| CN111209158B CN111209158B (en) | 2023-06-23 |
Family
ID=70784282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911351810.0A Active CN111209158B (en) | 2019-12-25 | 2019-12-25 | Mining monitoring method and cluster monitoring system for server cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209158B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052053A (en) * | 2020-10-10 | 2020-12-08 | 国科晋云技术有限公司 | Method and system for cleaning mining program in high-performance computing cluster |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8352409B1 (en) * | 2009-06-30 | 2013-01-08 | Symantec Corporation | Systems and methods for improving the effectiveness of decision trees |
| CN105577796A (en) * | 2015-12-25 | 2016-05-11 | 曙光信息产业(北京)有限公司 | Cluster power consumption control method and device |
| US9762593B1 (en) * | 2014-09-09 | 2017-09-12 | Symantec Corporation | Automatic generation of generic file signatures |
-
2019
- 2019-12-25 CN CN201911351810.0A patent/CN111209158B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8352409B1 (en) * | 2009-06-30 | 2013-01-08 | Symantec Corporation | Systems and methods for improving the effectiveness of decision trees |
| US9762593B1 (en) * | 2014-09-09 | 2017-09-12 | Symantec Corporation | Automatic generation of generic file signatures |
| CN105577796A (en) * | 2015-12-25 | 2016-05-11 | 曙光信息产业(北京)有限公司 | Cluster power consumption control method and device |
Non-Patent Citations (3)
| Title |
|---|
| 易军凯等: "手机流量非侵入式监测的决策树算法", 《计算机科学》 * |
| 沙超群等: "高密度存储服务器高速链路设计与仿真", 《国防科技大学学报》 * |
| 黄维维等: "基于ID3 决策树的木马动态检测技术研究", 《智能计算机与应用》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052053A (en) * | 2020-10-10 | 2020-12-08 | 国科晋云技术有限公司 | Method and system for cleaning mining program in high-performance computing cluster |
| CN112052053B (en) * | 2020-10-10 | 2023-12-19 | 国科晋云技术有限公司 | Method and system for cleaning ore mining program in high-performance computing cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111209158B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110826648B (en) | Method for realizing fault detection by utilizing time sequence clustering algorithm | |
| CN102722709B (en) | Method and device for identifying garbage pictures | |
| CN105471882A (en) | Behavior characteristics-based network attack detection method and device | |
| CN111352971A (en) | Bank system monitoring data anomaly detection method and system | |
| CN108965340B (en) | Industrial control system intrusion detection method and system | |
| CN103679012A (en) | Clustering method and device of portable execute (PE) files | |
| CN113409555B (en) | Real-time alarm linkage method and system based on Internet of things | |
| CN111598179A (en) | Power monitoring system user abnormal behavior analysis method, storage medium and equipment | |
| CN109002810A (en) | Model evaluation method, Radar Signal Recognition method and corresponding intrument | |
| CN115794803B (en) | Engineering audit problem monitoring method and system based on big data AI technology | |
| CN115277113A (en) | Power grid network intrusion event detection and identification method based on ensemble learning | |
| CN102567405A (en) | Hotspot discovery method based on improved text space vector representation | |
| CN111726351A (en) | Bagging-improved GRU parallel network flow abnormity detection method | |
| CN111209158A (en) | Mining monitoring method and cluster monitoring system for server cluster | |
| CN113886832A (en) | Intelligent contract vulnerability detection method, system, computer equipment and storage medium | |
| CN112039907A (en) | Automatic testing method and system based on Internet of things terminal evaluation platform | |
| CN113282920A (en) | Log abnormity detection method and device, computer equipment and storage medium | |
| CN117034149A (en) | Fault processing strategy determining method and device, electronic equipment and storage medium | |
| CN111026940A (en) | Network public opinion and risk information monitoring system and electronic equipment for power grid electromagnetic environment | |
| KR101621959B1 (en) | Apparatus for extracting and analyzing log pattern and method thereof | |
| CN115842645A (en) | UMAP-RF-based network attack traffic detection method and device and readable storage medium | |
| CN111163053B (en) | Malicious URL detection method and system | |
| CN114553468A (en) | Three-level network intrusion detection method based on feature intersection and ensemble learning | |
| CN113407495A (en) | SIMHASH-based file similarity determination method and system | |
| CN112052453A (en) | Webshell detection method and device based on Relief algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211011 Address after: 100193 building 36, yard 8, Dongbeiwang West Road, Haidian District, Beijing Applicant after: Dawning Information Industry (Beijing) Co.,Ltd. Applicant after: ZHONGKE SUGON INFORMATION INDUSTRY CHENGDU Co.,Ltd. Address before: 100193 building 36, yard 8, Dongbeiwang West Road, Haidian District, Beijing Applicant before: Dawning Information Industry (Beijing) Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |