CN104346255A - Method for automatically monitoring service conditions of process memories in cloud computation - Google Patents
Method for automatically monitoring service conditions of process memories in cloud computation Download PDFInfo
- Publication number
- CN104346255A CN104346255A CN201410561098.8A CN201410561098A CN104346255A CN 104346255 A CN104346255 A CN 104346255A CN 201410561098 A CN201410561098 A CN 201410561098A CN 104346255 A CN104346255 A CN 104346255A
- Authority
- CN
- China
- Prior art keywords
- jvmstat
- threshold value
- internal memory
- script
- district
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a method for automatically monitoring service conditions of process memories in cloud computation, and belongs to the field of cluster performance optimization and monitoring in cloud computation. The method is realized by virtue of a timed execution script jvmstat.sh, and specifically comprises the following steps: 1, copying the script jvmstat.sh and a configuration file, transmitting the copied script jvmstat.sh and configuration file to each node of a Hadoop cluster, adding the script jvmstat.sh to crontab, and starting crondservice; 2, acquiring the service conditions of all the java process memories by virtue of the script jvmstat.sh, and giving an alarm about the condition that a threshold value of each area of a process exceeds a set alarming threshold value in real time. The method is used for monitoring each java process memory in the Hadoop cluster and providing bases for the timely discovery of potential threats of the memories and solutions to OOM (out of memory) errors to fulfill the aims of optimizing the usage of the memories and improving the system performance of the cluster.
Description
Technical field
The present invention relates to a kind of method of monitoring process internal memory service condition, belong to cluster Performance tuning and supervision field in cloud computing, in a kind of cloud computing, automatically monitor the method for proceeding internal memory service condition specifically.
Background technology
Hadoop is a software frame that can carry out distributed treatment to mass data.Hadoop is reliable, because its hypothesis calculates element and the meeting of storage failure, therefore it safeguards multiple operational data copy, guarantees to process for the node redistribution of failure.Hadoop is efficient, because it works in a parallel fashion, by parallel processing speed up processing.Wherein distributed hadoop cluster is the gordian technique of cloud computing, and it takes full advantage of the facility that cloud computing brings, and also injects endless vigor to cloud computing.The finger daemon of cluster and the application process of operation, usual close relation, and it is huge to process data volume, meeting at substantial internal memory, Real-Time Monitoring grasps the situation of change of internal memory, and accordingly by Memory adjustments to best proportion, at utmost play system performance, improve the throughput of application program, avoid internal memory overflow error, become the vital problem needing to solve, automatically the method for proceeding internal memory service condition is monitored in a kind of cloud computing provided by the invention, the service condition of all jvm proceeding internal memories of the method self-timing record, and be saved in different daily records, and carry out Realtime Alerts according to the threshold value set in configuration file, internal memory for java process each in hadoop cluster is monitored, for the internal memory tuning of distributed task scheduling, Timeliness coverage internal memory potential threat provides foundation, reach optimization internal memory to use, improve the object of group system performance.
Summary of the invention
The present invention is directed to deficiency and the problem of prior art existence, provide the method for monitoring proceeding internal memory service condition automatically in a kind of cloud computing, the concrete scheme of proposition is:
Automatically monitor a method for proceeding internal memory service condition in cloud computing, utilize timing to perform script jvmstat.sh and realize, concrete steps are:
1. jvmstat.sh script and configuration file are copied, and be sent to each node of Hadoop cluster, jvmstat.sh is added into crontab, start crond service;
2. jvmstat.sh script obtains all java proceeding internal memory service conditions, and to process each district threshold value exceed warning setting threshold value situation carry out Realtime Alerts;
Described step 2. in jvmstat.sh script obtain the jvm process name that running of server, use jstat order acquisition internal memory service condition also to write daily record.
Described step 2. middle process each district threshold value comprises the internal memory use threshold value in survivor district, Eden district, Old district, Perm district, and Yong GC and Full GC produces the threshold value of frequency and utilizes the threshold value of time.
Described step is the middle warning setting threshold value using configuration file jvmstat.conf to preserve process each district threshold value 2..
Usefulness of the present invention is: the method for automatically monitoring proceeding internal memory service condition in a kind of cloud computing provided by the invention, the service condition of all jvm proceeding internal memories of the method self-timing record, and be saved in different daily records, and carry out Realtime Alerts according to the threshold value set in configuration file, internal memory for java process each in hadoop cluster is monitored, for the internal memory tuning of distributed task scheduling, Timeliness coverage internal memory potential threat provide foundation, improve the object of group system performance.
Accompanying drawing explanation
Fig. 1 performs the schematic flow sheet of jvmstat.sh script.
Embodiment
In conjunction with example, the present invention is specifically described.
(1) jvmstat.sh script false code is as follows:
Read all jvm processes numbering and title stored in output
do
Proname=takes out current process name from output
Prono=takes out current process number from output
filename=proc_$proname_$i.log
Current time is write daily record
Jstat – gc $ prono >> $ filename // internal memory service condition write daily record
jstat –gcutil $pron >> $ filename
for((i=$1;i<$8;i++))
do
Respective threshold is read from configuration file jvmstat.conf
Read journal file and obtain memory parameters analog value
If(memory parameters value > (or<) threshold value)
Send alarm email to operation maintenance personnel
fi
done
done
（2）poll.cron
*/1 * * * * /usr/hadoop/script/poll.sh
Embodiment 1
Automatically monitor a method for proceeding internal memory service condition in cloud computing, utilize timing to perform script jvmstat.sh and realize, concrete steps are:
1. jvmstat.sh script and configuration file are copied, and be sent to each node of Hadoop cluster, jvmstat.sh is added into crontab, start crond service;
2. jvmstat.sh script obtains all java proceeding internal memory service conditions, jstat order is used to obtain internal memory service condition and write daily record, and to process each district threshold value exceed warning setting threshold value situation carry out Realtime Alerts, configuration file jvmstat.conf is used to preserve the warning setting threshold value of process each district threshold value
Composition graphs 1, jvmstat.sh script specific works flow process is: (1) brings into operation; (2) local server process information is read; (3) if also have process: be, find survivor district process, then Proname=takes out process name xxx/Prono=and takes out process number xxx, and jstat obtains the internal memory service condition of current process stored in corresponding journal file; (4) if also have non-compare threshold: be read threshold value xxx from configuration file, corresponding parametric values is obtained from journal file; (5) threshold value of reading in (4) whether over range is judged: be return (4) after sending alarm email.
Described step 2. middle process each district threshold value comprises the internal memory use threshold value in survivor district, Eden district, Old district, Perm district, and Yong GC and Full GC produces the threshold value of frequency and utilizes the threshold value of time.
Embodiment 2
Automatically monitor a method for proceeding internal memory service condition in cloud computing, utilize timing to perform script jvmstat.sh and realize, concrete steps are:
1. jvmstat.sh script and configuration file are copied, and be sent to each node of Hadoop cluster, jvmstat.sh is added into crontab, start crond service;
2. jvmstat.sh script obtains all java proceeding internal memory service conditions, jstat order is used to obtain internal memory service condition and write daily record, and to process each district threshold value exceed warning setting threshold value situation carry out Realtime Alerts, configuration file jvmstat.conf is used to preserve the warning setting threshold value of process each district threshold value
Composition graphs 1, jvmstat.sh script specific works flow process is: (1) brings into operation; (2) local server process information is read; (3) if also have process: no, then terminate; Be that Proname=takes out process name/Prono=and takes out process number, jstat obtains the internal memory service condition of current process stored in corresponding journal file; (4) if also have non-compare threshold: be, find that Yong GC and Full GC produces the threshold value of frequency, from configuration file, read threshold value, obtain corresponding parametric values from journal file; (5) judge threshold value of reading in (4) whether over range: no, then return (4).
Embodiment 3
Automatically monitor a method for proceeding internal memory service condition in cloud computing, utilize timing to perform script jvmstat.sh and realize, concrete steps are:
1. jvmstat.sh script and configuration file are copied, and be sent to each node of Hadoop cluster, jvmstat.sh is added into crontab, start crond service;
2. jvmstat.sh script obtains all java proceeding internal memory service conditions, jstat order is used to obtain internal memory service condition and write daily record, and to process each district threshold value exceed warning setting threshold value situation carry out Realtime Alerts, configuration file jvmstat.conf is used to preserve the warning setting threshold value of process each district threshold value
Composition graphs 1, jvmstat.sh script specific works flow process is: (1) brings into operation; (2) local server process information is read; (3) if also have process: no, then terminate.
Claims (3)
1. automatically monitor a method for proceeding internal memory service condition in cloud computing, it is characterized in that utilizing timing to perform script jvmstat.sh realizes, concrete steps are:
1. jvmstat.sh script and configuration file are copied, and be sent to each node of Hadoop cluster, jvmstat.sh is added into crontab, start crond service;
2. jvmstat.sh script obtains all java proceeding internal memory service conditions, and to process each district threshold value exceed warning setting threshold value situation carry out Realtime Alerts;
Automatically the method for proceeding internal memory service condition is monitored in a kind of cloud computing according to claim 1, it is characterized in that the jvm process name that during described step 2., jvmstat.sh script acquisition server is running, use jstat order to obtain internal memory service condition and also write daily record.
2. in a kind of cloud computing according to claim 1 and 2, automatically monitor the method for proceeding internal memory service condition, it is characterized in that during described step 2., process each district threshold value comprises the internal memory use threshold value in survivor district, Eden district, Old district, Perm district, and Yong GC and Full GC produces the threshold value of frequency and utilizes the threshold value of time.
3. automatically monitor the method for proceeding internal memory service condition in a kind of cloud computing according to claim 3, it is characterized in that during described step 2., using configuration file jvmstat.conf to preserve the warning setting threshold value of process each district threshold value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410561098.8A CN104346255A (en) | 2014-10-21 | 2014-10-21 | Method for automatically monitoring service conditions of process memories in cloud computation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410561098.8A CN104346255A (en) | 2014-10-21 | 2014-10-21 | Method for automatically monitoring service conditions of process memories in cloud computation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104346255A true CN104346255A (en) | 2015-02-11 |
Family
ID=52501924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410561098.8A Pending CN104346255A (en) | 2014-10-21 | 2014-10-21 | Method for automatically monitoring service conditions of process memories in cloud computation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104346255A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105049933A (en) * | 2015-07-14 | 2015-11-11 | 四川长虹电器股份有限公司 | Automatic recording monitoring method of memory of smart TV |
| CN106250292A (en) * | 2016-08-11 | 2016-12-21 | 上海泛微网络科技股份有限公司 | A kind of office management system performance monitoring platform |
| CN107360317A (en) * | 2017-06-29 | 2017-11-17 | 努比亚技术有限公司 | The detection method and mobile terminal, storage medium of application resource leakage |
| CN107766123A (en) * | 2017-10-11 | 2018-03-06 | 郑州云海信息技术有限公司 | A kind of JVM tunings method |
| CN107797901A (en) * | 2017-10-25 | 2018-03-13 | 四川长虹电器股份有限公司 | A kind of storehouse analysis and the implementation method of mail Realtime Alerts |
| CN108874602A (en) * | 2018-06-15 | 2018-11-23 | 郑州云海信息技术有限公司 | It a kind of automatic setting and configures IMDT and collects the method and system of information |
| CN109324882A (en) * | 2018-08-22 | 2019-02-12 | 深圳点猫科技有限公司 | Method and electronic equipment based on linux system control application program |
| CN109558297A (en) * | 2018-11-27 | 2019-04-02 | 苏州思必驰信息科技有限公司 | EMS memory management process and device |
| CN109597676A (en) * | 2018-10-25 | 2019-04-09 | 平安科技（深圳）有限公司 | Monitoring and early warning method, apparatus and storage medium based on JVM |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101515247A (en) * | 2009-03-30 | 2009-08-26 | 福建星网锐捷网络有限公司 | Method and device for monitoring memory |
| CN101707632A (en) * | 2009-10-28 | 2010-05-12 | 浪潮电子信息产业股份有限公司 | Method for dynamically monitoring performance of server cluster and alarming real-timely |
| CN102929667A (en) * | 2012-10-24 | 2013-02-13 | 曙光信息产业(北京)有限公司 | Method for optimizing hadoop cluster performance |
| US20130332778A1 (en) * | 2012-06-07 | 2013-12-12 | Vmware, Inc. | Performance-imbalance-monitoring processor features |
-
2014
- 2014-10-21 CN CN201410561098.8A patent/CN104346255A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101515247A (en) * | 2009-03-30 | 2009-08-26 | 福建星网锐捷网络有限公司 | Method and device for monitoring memory |
| CN101707632A (en) * | 2009-10-28 | 2010-05-12 | 浪潮电子信息产业股份有限公司 | Method for dynamically monitoring performance of server cluster and alarming real-timely |
| US20130332778A1 (en) * | 2012-06-07 | 2013-12-12 | Vmware, Inc. | Performance-imbalance-monitoring processor features |
| CN102929667A (en) * | 2012-10-24 | 2013-02-13 | 曙光信息产业(北京)有限公司 | Method for optimizing hadoop cluster performance |
Non-Patent Citations (2)
| Title |
|---|
| KEVIN: "《https://www.forzw.com/archives/342》", 17 October 2009 * |
| 周志明著: "《深入理解Java虚拟机 JVM高级特性与最佳实践》", 30 June 2013 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105049933A (en) * | 2015-07-14 | 2015-11-11 | 四川长虹电器股份有限公司 | Automatic recording monitoring method of memory of smart TV |
| CN105049933B (en) * | 2015-07-14 | 2018-12-07 | 四川长虹电器股份有限公司 | The automation of smart television memory records monitoring method |
| CN106250292A (en) * | 2016-08-11 | 2016-12-21 | 上海泛微网络科技股份有限公司 | A kind of office management system performance monitoring platform |
| CN106250292B (en) * | 2016-08-11 | 2018-06-08 | 上海泛微网络科技股份有限公司 | A kind of office management system performance monitoring platform |
| CN107360317A (en) * | 2017-06-29 | 2017-11-17 | 努比亚技术有限公司 | The detection method and mobile terminal, storage medium of application resource leakage |
| CN107360317B (en) * | 2017-06-29 | 2020-12-01 | 湖南胜云光电科技有限公司 | Application program resource leakage detection method, mobile terminal and storage medium |
| CN107766123B (en) * | 2017-10-11 | 2020-10-27 | 苏州浪潮智能科技有限公司 | JVM (JVM) tuning method |
| CN107766123A (en) * | 2017-10-11 | 2018-03-06 | 郑州云海信息技术有限公司 | A kind of JVM tunings method |
| CN107797901A (en) * | 2017-10-25 | 2018-03-13 | 四川长虹电器股份有限公司 | A kind of storehouse analysis and the implementation method of mail Realtime Alerts |
| CN108874602A (en) * | 2018-06-15 | 2018-11-23 | 郑州云海信息技术有限公司 | It a kind of automatic setting and configures IMDT and collects the method and system of information |
| CN108874602B (en) * | 2018-06-15 | 2021-10-22 | 郑州云海信息技术有限公司 | Method and system for automatically setting and configuring IMDT and collecting information |
| CN109324882A (en) * | 2018-08-22 | 2019-02-12 | 深圳点猫科技有限公司 | Method and electronic equipment based on linux system control application program |
| CN109597676A (en) * | 2018-10-25 | 2019-04-09 | 平安科技（深圳）有限公司 | Monitoring and early warning method, apparatus and storage medium based on JVM |
| CN109558297A (en) * | 2018-11-27 | 2019-04-02 | 苏州思必驰信息科技有限公司 | EMS memory management process and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104346255A (en) | Method for automatically monitoring service conditions of process memories in cloud computation | |
| US11005730B2 (en) | System, method, and apparatus for high throughput ingestion for streaming telemetry data for network performance management | |
| CN109918349B (en) | Log processing method, log processing device, storage medium and electronic device | |
| US9917735B2 (en) | System and method for big data aggregation in sensor network | |
| US20190222603A1 (en) | Method and apparatus for network forensics compression and storage | |
| CN103761309A (en) | Operation data processing method and system | |
| WO2018032936A1 (en) | Method and device for checking domain name generated by domain generation algorithm | |
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| CN105357061A (en) | Operation and maintenance monitoring analysis system based on large-data-flow processing technology | |
| CN112615905B (en) | Method, device and equipment for scheduling block chain fragments and storage medium | |
| CN112434039A (en) | Data storage method, device, storage medium and electronic device | |
| CN109005162B (en) | Industrial control system security audit method and device | |
| Gulisano et al. | Online and scalable data validation in advanced metering infrastructures | |
| WO2017219873A1 (en) | Method and apparatus for locating domain names attacked by syn | |
| Clemm et al. | DNA: An SDN framework for distributed network analytics | |
| CN107580052A (en) | From the network self-adapting reptile method and system of evolution | |
| CN112148504A (en) | Target message processing method and device, storage medium and electronic device | |
| CN105069029A (en) | Real-time ETL (extraction-transformation-loading) system and method | |
| US11336536B1 (en) | Dynamic processing distribution for utility communication networks | |
| CN112820091B (en) | Power quality data transmission method, device, terminal equipment and medium | |
| CN115952884A (en) | Textile industry carbon emission and carbon footprint monitoring method and system based on industrial Internet | |
| CN116910144A (en) | Computing power network resource center, computing power service system and data processing method | |
| CN113051323B (en) | Water environment big data exchange method | |
| CN114398348A (en) | Smart grid abnormal data processing method, gateway, electronic device and storage medium | |
| CN112612811A (en) | Data import method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150211 |
|
| RJ01 | Rejection of invention patent application after publication |