CN111209158B - Mining monitoring method and cluster monitoring system for server cluster - Google Patents
Mining monitoring method and cluster monitoring system for server cluster Download PDFInfo
- Publication number
- CN111209158B CN111209158B CN201911351810.0A CN201911351810A CN111209158B CN 111209158 B CN111209158 B CN 111209158B CN 201911351810 A CN201911351810 A CN 201911351810A CN 111209158 B CN111209158 B CN 111209158B
- Authority
- CN
- China
- Prior art keywords
- mining
- server
- attribute
- training set
- monitoring method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a mining monitoring method of a server cluster, which comprises the following steps: generating a decision tree by using the collected server operation index data and the attribute affecting the mining judgment; and then automatically judging whether the server is dug or not according to the running condition of the server monitored by the decision tree. The method and the system have the advantage that whether the mine is dug or not can be automatically judged by a server under the monitoring of the cluster monitoring system.
Description
Technical Field
The application relates to the technical field of mining detection processing, in particular to a mining monitoring method and a cluster monitoring system of a server cluster.
Background
Along with the fire explosion of bit coin and blockchain technology, more and more people and companies add mining lines, a large number of computers are used for mining, a mining machine special for mining is also generated, and a batch of mining trojans appear at the same time, so that illegal penetration into a personal computer and a server cluster with insufficient safety measures occurs. The high-performance computing clusters are easy to be the penetration targets of the Trojan horse for mining due to the high computing power and the great benefits of the Trojan horse spreaders after penetration.
The cluster is dug to influence the service that the cluster should normally provide, a large amount of calculation leads to very high electricity expenditure, and economic damage is caused to a cluster operator, so that the cluster dug is found in time and the dug Trojan horse is cleared.
The main means for mining detection are as follows: by means of experience and manual discovery, a browser-based mining detection plug-in, whether a mining script exists or not through preset rules, and the like.
Experience has found that the efficiency is too low and the workload is too great.
The mining detection plug-in based on the browser protects the PC of which the object is a personal user and is not applicable to the cluster.
Though the method of whether the mine digging script exists or not through preset rule matching can detect the mine digging Trojan, each server needs to be started one by one for detection, the automation degree is insufficient, and once the mine digging script is upgraded, the preset rule set also needs to be upgraded at the same time, otherwise, the mine digging Trojan is quickly failed.
Because the rising time of the blockchain technology and the digital currency is not long, the research of the industry on the mining Trojan horse is insufficient, and the defects of more manual operations, low automation degree and untimely upgrading exist.
Disclosure of Invention
Aiming at the problems in the related art, the application provides an mining monitoring method of a server cluster, which uses collected server operation index data and attributes affecting mining judgment to generate a decision tree; and then automatically judging whether the server is dug or not according to the running condition of the server monitored by the decision tree.
The technical scheme of the application is realized as follows:
the mining monitoring method of the server cluster comprises the following steps:
generating a decision tree by using the collected server operation index data and the attribute affecting the mining judgment;
and then automatically judging whether the server is dug or not according to the running condition of the server monitored by the decision tree.
According to an embodiment of the present application, the server operation index data includes at least one of a server name, a collection time, a CPU index, and a process index.
According to an embodiment of the present application, generating a decision tree includes: processing the input of the process function by the process function and generating an output; the output is a decision tree; the input includes a training set and a property set; a training set is a set formed based on the server operation index data, defined as d= { (x 1, y 1), (x 2, y 2), … (xm, ym) }; the attribute set is a set formed based on the attribute affecting the mining determination, and is defined as a= { a1, a2, … ad }.
According to an embodiment of the present application, the set of attributes includes: at least one of a process name with highest CPU utilization rate, a user to whom the process with highest CPU utilization rate belongs, a time period during acquisition and a segmentation interval in which the CPU utilization rate is located.
According to the embodiment of the application, the segmentation section where the CPU utilization rate is located refers to dividing the CPU utilization rate into different sections according to the high, medium and low, and converting the continuous value into the discrete value.
According to an embodiment of the present application, the input of a process function is processed by a process function, which is a recursive function, defined as treebenerate (D, a), and the output is generated, the process function comprising: generating node; if all samples in the D belong to the class C, marking the node as a class C leaf node, and recursively returning; if A is an empty set or the values of the samples in D in A are the same, the node is marked as the class with the largest number of samples in D, the class is a leaf node, the samples in D are the current node, and the recursion returns.
According to an embodiment of the present application, the processing of the inputs of the process functions by the process functions and the generation of outputs further comprises: selecting an attribute value a from A, forming a sample subset Dv by samples with the attribute value of the samples in D being the selected attribute value a, if the sample subset Dv is not empty, taking a treenogenesis (Dv, A { a }) as a branch node, if the sample subset Dv is empty, marking the branch node as a leaf node with the highest sample number class in D, and recursively returning the sample in D as a father node; and repeatedly selecting an attribute value from A until all values in A have been selected, and outputting a decision tree taking node as a root node by using the method.
Decision trees are also called decision trees, are a common machine learning method, and basic algorithms of the decision trees are as follows:
according to the embodiment of the application, when the situation that the mine Trojan is wrong or a novel mine Trojan is found is judged, inaccurate results are marked and are included in a training set, and a decision tree is regenerated.
According to the embodiment of the application, the judgment result of whether the server is dug or not gives an alarm to operation and maintenance personnel.
The beneficial technical effects of this application lie in:
the server under the monitoring of the cluster monitoring system can automatically judge whether the mine is dug or not. Because the cluster monitoring system can continuously run, the timeliness of detection is good.
When the situation that the mining Trojan is wrong or a novel mining Trojan appears is judged, inaccurate results can be marked again, the inaccurate results are brought into a training set, a decision tree model is regenerated, the more the marking times are, the higher the detection accuracy is, and the model forms the automatic learning capability.
Drawings
For a clearer description of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
Fig. 1 is a flow chart of a mining monitoring method of a server cluster according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, including cluster monitoring systems in combination with other classification algorithms in machine learning, obtained by one of ordinary skill in the art based on the embodiments herein are within the scope of the present application.
According to the embodiment of the application, an ore mining monitoring method of a server cluster is provided. Fig. 1 shows a flowchart of a mining monitoring method of a server cluster according to an embodiment of the present application.
Selecting one test cluster, running mining Trojan horse on several servers, and normally running the rest servers.
The acquisition server operates the index data as at step S10 of fig. 1. The cluster monitoring system is used for regularly acquiring the operation information of all servers, the servers are used for normal operation and mined, and the acquisition of the operation index data of the servers comprises the following steps: the method comprises the steps of collecting at least one of time, server name, average CPU utilization rate, utilization rate of each CPU core, CPU utilization rate of a process and user to which the process belongs.
And (3) arranging all collected server operation index data into a format required by a decision tree, marking whether each server is dug or not, and adding a training set.
The determination at step S10 of fig. 1 affects the attribute of the mining decision. Determining attributes that affect a mining decision, comprising: the method comprises the steps of adding a process name with highest CPU utilization rate, a user to which the process with highest CPU utilization rate belongs, a time period during acquisition, and a segmented interval in which the CPU utilization rate is located (dividing the CPU utilization rate into different intervals according to the height, the middle and the low, converting a continuous value into a discrete value), into an attribute set.
A decision tree algorithm is applied to generate a decision tree using the training set and the attribute set generated in the above steps, such as the decision tree generated at step S10 of fig. 1.
And accessing the decision tree into a monitoring system, automatically judging whether the server is dug according to the running index of the server monitored later, and sending an alarm through an alarm module of the monitoring system after judging in step S20 of the figure 1.
And when the detection is judged to be in error, marking the judgment result again, re-incorporating the marked data into the training set, and regenerating a decision tree, namely upgrading the detection model.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (8)
1. The mining monitoring method for the server cluster is characterized by comprising the following steps of:
generating a decision tree by using the collected server operation index data and the attribute affecting the mining judgment;
then automatically judging whether the server is dug according to the running condition of the server monitored by the decision tree,
wherein generating the decision tree comprises:
processing an input of a process function by the process function and generating an output, the output being a decision tree, the input comprising a training set and an attribute set, the training set being a set formed based on the server operation index data, the attribute set being a set formed based on the attributes affecting the mining decision,
wherein the process function is a recursive function comprising:
generating nodes;
if all the samples in the training set belong to the first category, generating leaf nodes marked as the first category, and recursively returning;
if the attribute set is an empty set or the values of the samples in the training set in the attribute set are the same, the samples in the training set are marked as leaf nodes of a second class, the samples in the training set are returned in a recursion mode, the samples in the training set are current nodes, and the second class is the class with the largest sample number in the training set.
2. The mining monitoring method of a server cluster according to claim 1, wherein: the server operation index data comprises at least one of server name, acquisition time, CPU index and process index.
3. The mining monitoring method of a server cluster according to claim 1, wherein the attribute set includes: at least one of a process name with highest CPU utilization rate, a user to whom the process with highest CPU utilization rate belongs, a time period during acquisition and a segmentation interval in which the CPU utilization rate is located.
4. A mining monitoring method of a server cluster according to claim 3, wherein: the segmentation section where the CPU utilization rate is located refers to dividing the CPU utilization rate into different sections according to the high, medium and low, and converting the continuous value into a discrete value.
5. The mining monitoring method of a server cluster according to claim 1, wherein an input of a process function is processed by a process function and an output is generated, the process function being a recursive function, further comprising:
selecting an attribute value from the attribute set, forming a sample subset by samples with the attribute value of the samples in the training set being the selected attribute value, generating a branch node for the node if the sample subset is not empty, marking the branch node as a leaf node of the second category if the sample subset is empty, recursively returning, wherein the node is a father node, and the second category is the category with the largest number of samples in the training set;
and repeatedly selecting an attribute value from the attribute set until all values of the attribute set are selected, and outputting a decision tree taking the node as a root node by using the method.
6. The mining monitoring method of a server cluster according to claim 1,3,4 or 5, wherein: when the situation that the mining Trojan is wrong or a novel mining Trojan appears is judged, inaccurate results are marked and are included in a training set, and a decision tree is regenerated.
7. The mining monitoring method of a server cluster according to claim 1, wherein: and sending an alarm to operation and maintenance personnel according to the judging result of whether the server is dug or not.
8. A cluster monitoring system for mine excavation monitoring of a server cluster, comprising: a storage medium storing a program that is executed to implement the mining monitoring method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911351810.0A CN111209158B (en) | 2019-12-25 | 2019-12-25 | Mining monitoring method and cluster monitoring system for server cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911351810.0A CN111209158B (en) | 2019-12-25 | 2019-12-25 | Mining monitoring method and cluster monitoring system for server cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111209158A CN111209158A (en) | 2020-05-29 |
| CN111209158B true CN111209158B (en) | 2023-06-23 |
Family
ID=70784282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911351810.0A Active CN111209158B (en) | 2019-12-25 | 2019-12-25 | Mining monitoring method and cluster monitoring system for server cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209158B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052053B (en) * | 2020-10-10 | 2023-12-19 | 国科晋云技术有限公司 | Method and system for cleaning ore mining program in high-performance computing cluster |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8352409B1 (en) * | 2009-06-30 | 2013-01-08 | Symantec Corporation | Systems and methods for improving the effectiveness of decision trees |
| CN105577796A (en) * | 2015-12-25 | 2016-05-11 | 曙光信息产业(北京)有限公司 | Cluster power consumption control method and device |
| US9762593B1 (en) * | 2014-09-09 | 2017-09-12 | Symantec Corporation | Automatic generation of generic file signatures |
-
2019
- 2019-12-25 CN CN201911351810.0A patent/CN111209158B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8352409B1 (en) * | 2009-06-30 | 2013-01-08 | Symantec Corporation | Systems and methods for improving the effectiveness of decision trees |
| US9762593B1 (en) * | 2014-09-09 | 2017-09-12 | Symantec Corporation | Automatic generation of generic file signatures |
| CN105577796A (en) * | 2015-12-25 | 2016-05-11 | 曙光信息产业(北京)有限公司 | Cluster power consumption control method and device |
Non-Patent Citations (3)
| Title |
|---|
| 基于ID3 决策树的木马动态检测技术研究;黄维维等;《智能计算机与应用》;20110630;第1卷(第1期);摘要，第1-3节 * |
| 手机流量非侵入式监测的决策树算法;易军凯等;《计算机科学》;20160615;372-375 * |
| 高密度存储服务器高速链路设计与仿真;沙超群等;《国防科技大学学报》;20150228(第01期);42-49 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111209158A (en) | 2020-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111798312B (en) | Financial transaction system anomaly identification method based on isolated forest algorithm | |
| CN112148772A (en) | Alarm root cause identification method, device, equipment and storage medium | |
| CN109981625B (en) | Log template extraction method based on online hierarchical clustering | |
| CN103679012A (en) | Clustering method and device of portable execute (PE) files | |
| CN114090402A (en) | User abnormal access behavior detection method based on isolated forest | |
| CN114281864A (en) | Correlation analysis method for power network alarm information | |
| CN113409555A (en) | Real-time alarm linkage method and system based on Internet of things | |
| CN111209158B (en) | Mining monitoring method and cluster monitoring system for server cluster | |
| CN114978877A (en) | Exception handling method and device, electronic equipment and computer readable medium | |
| CN113282920B (en) | Log abnormality detection method, device, computer equipment and storage medium | |
| CN113313304A (en) | Power grid accident abnormity analysis method and system based on big data decision tree | |
| CN112039907A (en) | Automatic testing method and system based on Internet of things terminal evaluation platform | |
| CN117034149A (en) | Fault processing strategy determining method and device, electronic equipment and storage medium | |
| CN116228429A (en) | Method and device for detecting transaction data | |
| CN113535458B (en) | Abnormal false alarm processing method and device, storage medium and terminal | |
| CN116126807A (en) | Log analysis method and related device | |
| CN115344538A (en) | Log processing method, device and equipment and readable storage medium | |
| CN111680572B (en) | Dynamic judgment method and system for power grid operation scene | |
| CN113887932A (en) | Operation and maintenance management and control method and device based on artificial intelligence and computer equipment | |
| CN112418449A (en) | Generation method, positioning method and device of power supply line fault positioning model | |
| CN113407495A (en) | SIMHASH-based file similarity determination method and system | |
| CN111027296A (en) | Report generation method and system based on knowledge base | |
| CN113746780A (en) | Abnormal host detection method, device, medium and equipment based on host image | |
| CN114721861B (en) | Log differentiation comparison-based fault positioning method and system | |
| CN113612765B (en) | Website detection method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211011 Address after: 100193 building 36, yard 8, Dongbeiwang West Road, Haidian District, Beijing Applicant after: Dawning Information Industry (Beijing) Co.,Ltd. Applicant after: ZHONGKE SUGON INFORMATION INDUSTRY CHENGDU Co.,Ltd. Address before: 100193 building 36, yard 8, Dongbeiwang West Road, Haidian District, Beijing Applicant before: Dawning Information Industry (Beijing) Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |