CN111949490A - JVM process monitoring method and device - Google Patents

JVM process monitoring method and device Download PDF

Info

Publication number
CN111949490A
CN111949490A CN202010821228.2A CN202010821228A CN111949490A CN 111949490 A CN111949490 A CN 111949490A CN 202010821228 A CN202010821228 A CN 202010821228A CN 111949490 A CN111949490 A CN 111949490A
Authority
CN
China
Prior art keywords
difference
jvm
jvm process
heap memory
memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010821228.2A
Other languages
Chinese (zh)
Other versions
CN111949490B (en
Inventor
姜华
宋哲
张志海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202010821228.2A priority Critical patent/CN111949490B/en
Publication of CN111949490A publication Critical patent/CN111949490A/en
Application granted granted Critical
Publication of CN111949490B publication Critical patent/CN111949490B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/301Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is a virtual computing platform, e.g. logically partitioned systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3037Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a memory, e.g. virtual memory, cache
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3055Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/32Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/324Display of status information
    • G06F11/327Alarm or error message display
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45583Memory management, e.g. access or allocation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45591Monitoring or debugging support
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/815Virtual
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/865Monitoring of software
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application provides a JVM process monitoring method and a device, wherein the method comprises the following steps: acquiring a difference of the occupied quantity of the heap memory of a target JVM process; acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table; and judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased. The method and the device can improve the efficiency and the accuracy of monitoring the JVM process, and further can ensure the normal operation of a service system.

Description

JVM process monitoring method and device
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for monitoring a JVM process.
Background
The Java programming language is widely applied to a plurality of industries such as the internet industry, the financial industry, the telecommunication industry, the power industry and the like due to the characteristics of concise code syntax, cross-platform portability, safe automatic memory management, high development efficiency and the like. A large number of Java Virtual Machine (JVM) processes are typically deployed on many types of servers in a data center, and these different JVM processes each carry an enterprise-critical business system and serve different users within the enterprise.
When a business system has a large number of SQL statements without binding variables, a foreground web page export data without limiting export line number, a web query page without paging and other conditions, memory overflow may be caused, and the normal operation of the business system is affected. The existing solution generally checks the JVM processes one by one in a manual mode, determines the JVM process with a fault, and checks the JVM process in a manual mode, which has the problems of low efficiency, poor reliability and the like.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a JVM process monitoring method and device, which can improve the efficiency and accuracy of JVM process monitoring and further can ensure the normal operation of a service system.
In order to solve the technical problem, the present application provides the following technical solutions:
in a first aspect, the present application provides a method for monitoring a JVM process, including:
acquiring a difference of the occupied quantity of the heap memory of a target JVM process;
acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table;
and judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
Further, before the obtaining the heap memory occupation amount difference range corresponding to the target JVM process, the method further includes:
acquiring the difference of the number of object instances of a target JVM process;
correspondingly, after the determination of the heap memory growth exception of the target JVM process, the method further includes:
acquiring a difference range of the number of object instances corresponding to the target JVM process from a preset JVM monitoring second threshold table;
and judging whether the difference of the number of the object instances is within the range of the difference of the number of the object instances, and if not, determining that the increase of the number of the object instances of the target JVM process is abnormal.
Further, before the obtaining the difference range of the number of the object instances corresponding to the target JVM process, the method further includes:
acquiring the total difference of the object instances of the target JVM process in the memory;
correspondingly, after the determining that the number of the object instances of the target JVM process is abnormally increased, the method further includes:
and judging whether the total difference of the memory occupied by the object instance exceeds a threshold of the amount of the occupied memory, if so, determining that the memory occupied by the object instance of the target JVM process is abnormally increased.
Further, the JVM process monitoring method further includes:
acquiring historical heap memory occupancy difference records of a plurality of JVM processes;
and obtaining a plurality of difference ranges of the occupied quantity of the heap memory corresponding to each JVM process based on the history difference record of the occupied quantity of the heap memory and a three-sigma algorithm, and storing the difference ranges of the occupied quantity of the heap memory corresponding to each JVM process in the JVM monitoring first threshold value table, wherein each difference range of the occupied quantity of the heap memory of the same JVM process corresponds to different preset time periods respectively.
Further, the JVM process monitoring method further includes:
acquiring historical heap memory occupancy records of a plurality of JVM processes;
and generating a historical heap memory occupancy difference record of each JVM process based on the historical heap memory occupancy records.
Further, the JVM process monitoring method further includes:
acquiring historical object instance difference records of a plurality of JVM processes;
and obtaining a plurality of object instance number difference ranges corresponding to the JVM processes respectively based on the historical object instance difference record and the three-sigma algorithm, storing the object instance number difference ranges in the JVM monitoring second threshold table, wherein the object instance number difference ranges of the same target JVM process respectively correspond to different preset time periods.
Further, the JVM process monitoring method further includes:
acquiring historical object instance records of a plurality of JVM processes;
and generating a historical object instance difference record of each JVM process based on the historical object instance record.
Further, after the determining that the object instance of the target JVM process occupies a memory growth exception, the method further includes:
generating early warning information of abnormal memory growth occupied by the object instance corresponding to the target JVM process;
and judging whether the number of times of continuously generating the early warning information corresponding to the target JVM process in a preset first early warning time range exceeds a generation number threshold, if so, judging whether historical early warning information output corresponding to the target JVM process exists in a preset second early warning time range, and if not, outputting the generated early warning information.
In a second aspect, the present application provides a JVM process monitoring apparatus, including:
the acquisition module is used for acquiring the difference of the occupied quantity of the heap memory of the target JVM process;
the determining module is used for acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table;
and the judging module is used for judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
In a third aspect, the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the JVM process monitoring method when executing the computer program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon computer instructions that, when executed, implement the JVM process monitoring method.
According to the technical scheme, the JVM process monitoring method and device are provided. Wherein, the method comprises the following steps: acquiring a difference of the occupied quantity of the heap memory of a target JVM process; acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table; judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, if not, determining that the heap memory of the target JVM process is abnormally increased, so that the efficiency and the accuracy of monitoring the JVM process can be improved, and the normal operation of a service system can be further ensured; specifically, the JVM heap memory occupied by the instance object is abnormal, and abnormal alarm is timely carried out, so that the object instance occupying the memory is determined, the function module and the program code which have problems can be further determined, the problem program code can be timely repaired, the phenomenon that the OutOfMemoryError memory overflows is avoided, the effect of repairing the production environment program code can be played, and stable operation of the production system is powerfully guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a JVM process monitoring method in an embodiment of the present application;
FIG. 2 is a flow chart of a JVM process monitoring method in another embodiment of the present application;
FIG. 3 is a logic diagram of a JVM process monitoring method in an application example of the present application;
FIG. 4 is a graphical representation of an exemplary normal distribution;
FIG. 5 is a flow chart of a JVM monitoring information collection process in an application example of the present application;
FIG. 6 is a flow chart illustrating an alarm notification process in an exemplary application of the present application;
FIG. 7 is a flow chart illustrating a flexible query analysis process in an exemplary application of the present application;
FIG. 8 is a flow chart illustrating a data cleansing process in an application example of the present application;
FIG. 9 is a schematic structural diagram of a JVM process monitoring apparatus according to an embodiment of the present application;
fig. 10 is a schematic block diagram of a system configuration of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to solve the problems of low efficiency, poor reliability and the like in the prior art of manually checking the JVM process, the application provides a JVM process monitoring method and a device; the monitoring information of the monitored JVM process can be acquired based on a point-buried component mode, the acquired monitoring information is uploaded to a kafka collection cluster in the form of heartbeat data packets, then heartbeat data packets are acquired from the kafka cluster, the heartbeat data packets are analyzed and stored in different data tables of a distributed database in a classified mode, then the occupation situation of the JVM process in a heap memory is analyzed and calculated by utilizing SQL sentences and combining with the Sanxi check algorithm, and the JVM process is found out abnormal and early warned. Whether the JVM process is in a healthy state is judged based on the regularly uploaded monitored JVM process survival heartbeat information (including JVM process heap memory occupation condition, JVM process in-heap object instance number and object instance heap memory occupation amount), and collection, analysis, abnormal state judgment and early warning of running state information of a plurality of JVM processes deployed on one or more servers can be realized.
Based on this, in order to improve efficiency and accuracy of JVM process monitoring and further ensure normal operation of a service system, an embodiment of the present application provides a JVM process monitoring apparatus, which may be a server or a client device, where the client device may include a smart phone, a tablet electronic device, a network set top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), a vehicle-mounted device, an intelligent wearable device, and the like. Wherein, intelligence wearing equipment can include intelligent glasses, intelligent wrist-watch and intelligent bracelet etc..
In practical applications, the monitoring part of the JVM process may be executed on the server side as described above, or all operations may be completed in the client device. The selection may be specifically performed according to the processing capability of the client device, the limitation of the user usage scenario, and the like. This is not a limitation of the present application. The client device may further include a processor if all operations are performed in the client device.
The client device may have a communication module (i.e., a communication unit), and may be communicatively connected to a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.
The server and the client device may communicate using any suitable network protocol, including network protocols not yet developed at the filing date of this application. The network protocol may include, for example, a TCP/IP protocol, a UDP/IP protocol, an HTTP protocol, an HTTPS protocol, or the like. Of course, the network Protocol may also include, for example, an RPC Protocol (Remote Procedure Call Protocol), a REST Protocol (Representational State Transfer Protocol), and the like used above the above Protocol.
The following examples are intended to illustrate the details.
In order to improve efficiency and accuracy of monitoring a JVM process and further ensure normal operation of a service system, this embodiment provides a JVM process monitoring method in which an execution main body is a JVM process monitoring device, as shown in fig. 1, the method specifically includes the following contents:
s11: and acquiring the difference of the occupied quantity of the heap memory of the target JVM process.
Specifically, the target JVM process may be a JVM process corresponding to a heap memory occupancy difference record newly added in real time or at regular time in a heap memory occupancy difference table, and the heap memory occupancy difference of the target JVM process may be obtained according to the newly added heap memory occupancy difference record, and the heap memory occupancy difference table may be stored in the target distributed database in advance. The target JVM processes may be one or more, and each of the target JVM processes may be in the same JVM server or in different JVM servers.
S12: and acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold value table.
Specifically, the range of the difference in the amount of the heap memory occupied may include an upper difference in the amount of the heap memory occupied and a lower difference in the amount of the heap memory occupied.
S13: and judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
As can be seen from the above description, the JVM process monitoring method provided in this embodiment can determine whether the heap memory growth of the target JVM process is abnormal by determining whether the heap memory occupancy difference is within the heap memory occupancy difference range, so as to improve the accuracy and efficiency of monitoring the JVM process; each of the difference ranges of the occupied amount of the heap memory can be obtained by performing three-sigma calculation on a plurality of historical occupied amount of the heap memory in the same preset time period according to the same target JVM process, and the accuracy of monitoring of the next JVM process can be improved by improving the reliability of obtaining the difference range of the occupied amount of the heap memory.
In order to efficiently and accurately monitor the abnormal heap memory growth condition and further improve the accuracy of monitoring the JVM, in an embodiment of the present application, referring to fig. 2, before step S12, the method further includes:
s21: acquiring the difference of the number of object instances of a target JVM process;
correspondingly, after step S13, the method further includes:
s22: acquiring a difference range of the number of object instances corresponding to the target JVM process from a preset JVM monitoring second threshold table;
specifically, the range of the number difference of the object instances may include an upper limit difference of the heap memory occupancy amount and a lower limit difference of the heap memory occupancy amount.
S23: and judging whether the difference of the number of the object instances is within the range of the difference of the number of the object instances, and if not, determining that the increase of the number of the object instances of the target JVM process is abnormal.
As can be seen from the above description, the JVM process monitoring method provided in this embodiment can efficiently and accurately monitor the increment of the number of the object instances on the basis of efficiently and accurately monitoring the increment of the heap memory in the abnormal increment condition of the object instances by determining whether the difference of the number of the object instances is within the range of the difference of the number of the object instances, so as to further improve the accuracy of JVM monitoring, and in addition, after determining that the heap memory of the target JVM process is abnormally increased, execute steps S22 and S23, so as to reduce the data volume of the process of determining the difference of the number of the newly added object instances, thereby improving the efficiency of JVM monitoring.
In order to efficiently and accurately monitor the memory occupied by the object instance and further improve the accuracy of monitoring the JVM on the basis of efficiently and accurately monitoring the heap memory growth condition and the object instance number growth condition, in an embodiment of the present application, before the obtaining of the difference range of the object instance number corresponding to the target JVM process, the method further includes:
s031: acquiring the total difference of the object instances of the target JVM process in the memory;
correspondingly, after the determining that the number of the object instances of the target JVM process is abnormally increased, the method further includes:
s31: and judging whether the total difference of the memory occupied by the object instance exceeds a threshold of the amount of the occupied memory, if so, determining that the memory occupied by the object instance of the target JVM process is abnormally increased.
Specifically, the threshold of the amount of occupied memory may be set according to actual needs, and is preferably 16 MB.
As can be seen from the above description, the JVM monitoring method provided in this embodiment of the present application can efficiently and accurately monitor the memory occupied by the object instance on the basis of efficiently and accurately monitoring the memory growth condition and the number growth condition of the object instances, determine the object instances that occupy abnormal memory, and further determine abnormal program codes, so as to repair the abnormal program agents in time, and ensure normal operation of the service system, and at the same time, determine whether the memory occupied by the object instances is abnormally increased after determining that the number growth of the object instances of the target JVM process corresponding to the newly-added memory occupancy difference record is abnormal, can reduce the data volume in the process of determining the total memory occupancy difference occupied by the object instances, and improve the efficiency.
In order to improve the reliability of monitoring the first threshold table by the JVM and further improve the accuracy of monitoring the JVM by monitoring the first threshold table by the JVM, in an embodiment of the present application, the method for monitoring a JVM process further includes:
s41: acquiring historical heap memory occupancy difference records of a plurality of JVM processes;
s42: and obtaining a plurality of difference ranges of the occupied quantity of the heap memory corresponding to each JVM process based on the history difference record of the occupied quantity of the heap memory and a three-sigma algorithm, and storing the difference ranges of the occupied quantity of the heap memory corresponding to each JVM process in the JVM monitoring first threshold value table, wherein each difference range of the occupied quantity of the heap memory of the same JVM process corresponds to different preset time periods respectively.
Specifically, the preset time period may be set according to actual needs, for example, the preset time period may be a common working day, month 1, month end, 11/month/11/12/month/rest day.
As can be seen from FIG. 4 and the following expressions, the probability of the numerical distribution in (μ -3 σ, μ +3 σ) is about 0.9973, the probability in (μ -2 σ, μ +2 σ) is about 0.9545, and the probability in (μ -2 σ, μ +2 σ) is about 0.6827 in the normal distribution.
Pr(μ-σ≤X≤μ+σ)≈0.6827;
Pr(μ-2σ≤X≤μ+2σ)≈0.9545;
Pr(μ-3σ≤X≤μ+3σ)≈0.9973;
It can be considered that the values of X are almost entirely concentrated in the (μ -3 σ, μ +3 σ) interval, and the probability of exceeding this range is only less than 0.3%, where σ represents the standard deviation and μ represents the mean value; in this embodiment, a three-sigma algorithm may be applied to process the multiple historical heap memory occupancy differences of the same target JVM process in the same preset time period to obtain each heap memory occupancy difference range, that is, to calculate a standard deviation σ and a mean μ corresponding to the multiple historical heap memory occupancy differences of the same target JVM process in the same preset time period, so as to obtain μ -3 σ and μ +3 σ, where the heap memory occupancy difference range is a range in which the heap memory occupancy difference is greater than or equal to μ -3 σ and less than or equal to μ +3 σ, and the three-sigma algorithm is based on a monitoring index value of a time series, and can check most normal values and identify abnormal values.
In order to improve the reliability of obtaining the history heap memory occupancy difference record and further improve the accuracy of determining the heap memory occupancy difference range by applying the history heap memory occupancy difference, in an embodiment of the present application, the JVM process monitoring method further includes:
s51: acquiring historical heap memory occupancy records of a plurality of JVM processes;
s52: and generating a historical heap memory occupancy difference record of each JVM process based on the historical heap memory occupancy records.
In order to improve the reliability of monitoring the second threshold table by the JVM and further improve the accuracy of monitoring the JVM by applying the JVM to monitor the second threshold table, in an embodiment of the present application, the method for monitoring the JVM process further includes:
s61: acquiring historical object instance difference records of a plurality of JVM processes;
s62: and obtaining a plurality of object instance number difference ranges corresponding to the JVM processes respectively based on the historical object instance difference record and the three-sigma algorithm, storing the object instance number difference ranges in the JVM monitoring second threshold table, wherein the object instance number difference ranges of the same target JVM process respectively correspond to different preset time periods.
In order to improve the reliability of obtaining the history object instance difference record and further improve the accuracy of determining the object instance number difference range by applying the history object instance difference, in an embodiment of the present application, the JVM process monitoring method further includes:
s71: acquiring historical object instance records of a plurality of JVM processes;
s72: and generating a historical object instance difference record of each JVM process based on the historical object instance record.
In order to improve the reliability of the JVM process exception early warning on the basis of ensuring the instantaneity and efficiency of the JVM process exception early warning, in an embodiment of the present application, after step S31, the method further includes:
s81: generating early warning information of abnormal memory growth occupied by the object instance corresponding to the target JVM process;
s82: and judging whether the number of times of continuously generating the early warning information corresponding to the target JVM process in a preset first early warning time range exceeds a generation number threshold, if so, judging whether historical early warning information output corresponding to the target JVM process exists in a preset second early warning time range, and if not, outputting the generated early warning information.
It can be understood that, the determining whether there is historical early warning information output corresponding to the target JVM process within a preset second early warning time range may include: judging whether a preset alarm history table has a history early warning information output record corresponding to the target JVM process within a second early warning time range, wherein the alarm history table can be stored in a target distributed database in advance; correspondingly, after the generated early warning information is output, the generated early warning information and the output time can be stored in the preset alarm history table.
Specifically, the first early warning time range and the second early warning time range can be set according to actual conditions, influences of factors such as GC garbage collection and JVM process restart on JVM monitoring results can be avoided by judging whether the number of times of generating the early warning information exceeds a generation number threshold value, historical early warning information is output by judging whether the second early warning time range, whether the generated early warning information is output or not is determined, early warning efficiency can be improved, and repeated early warning is avoided.
In order to improve efficiency and accuracy of monitoring the JVM process and further ensure normal operation of a service system, the present application provides an application example of a JVM process monitoring method, which specifically includes the following contents:
s1101: acquiring newly added difference records of the occupancy rates of the heap memories in a preset time range in a preset difference table of the occupancy rates of the heap memories, wherein each difference record of the occupancy rates of the heap memories comprises: and the process identification and the heap memory occupation amount difference corresponding to the unique JVM process.
Specifically, the preset time range may be set according to actual needs, for example, a time range from 5S before the current system time to the current system time; the process identification may include: the IP address of the server, the process name of the JVM and the application system code can be the same, and the process identifiers recorded by the occupancy difference of different heap memories can be the same.
S1102: acquiring each heap memory occupancy difference range corresponding to the process identifier of the newly-added heap memory occupancy difference record from a preset JVM monitoring first threshold table, wherein the preset JVM monitoring first threshold table comprises: and the process identifier corresponding to each JVM process and the multiple heap memory occupation amount difference ranges corresponding to each process identifier.
S1103: and judging whether the difference of the occupied quantity of the heap memory recorded by the newly added difference of the occupied quantity of the heap memory is in the range of the corresponding at least one difference of the occupied quantity of the heap memory, if not, determining that the increase of the heap memory of the JVM process corresponding to the newly added difference of the occupied quantity of the heap memory is abnormal.
Further, before step S1102, the method may further include:
s1201: acquiring newly added object instance difference records of a preset object instance difference table in a preset time range, wherein each object instance difference record comprises: and the process identification corresponding to the unique JVM process and the difference of the number of the object instances. Correspondingly, after step S1103, the method further includes:
s1202: acquiring the difference range of the number of each object instance corresponding to the process identifier of the newly added object instance difference record from a preset JVM monitoring second threshold table, wherein the preset JVM monitoring second threshold table comprises: and the process identifier corresponding to each JVM process and the difference range of the number of the multiple object instances corresponding to each process identifier.
S1203: and judging whether the difference of the number of the object instances of the newly added object instance difference record is in the range of the corresponding difference of the number of at least one object instance, if not, determining that the increase of the number of the object instances of the JVM process corresponding to the newly added object instance difference record is abnormal.
Further, the object instance deficit record may further include: the object instance occupies the total difference of the memory; correspondingly, after step S1203, the method further includes:
s1301: and judging whether the total difference of the memory occupied by the object instance of the newly added object instance difference record exceeds the threshold of the memory occupied amount, if so, determining that the memory occupied by the object instance of the JVM process corresponding to the newly added object instance difference record is abnormally increased.
Further, the heap memory occupancy difference record may further include: a first acquisition time of the heap memory occupancy difference; correspondingly, before step S1102, the method further includes:
s1401: and acquiring a plurality of heap memory occupation amount difference records from the preset heap memory occupation amount difference table.
Each of the heap memory footprint difference records comprises: and the process identification and the heap memory occupation amount difference corresponding to the unique JVM process.
S1402: and dividing the heap memory occupancy difference recorded by each heap memory occupancy difference into a plurality of groups of heap memory occupancy difference groups according to the first acquisition time, a preset time period and a process identifier, wherein each heap memory occupancy difference group of the same JVM process corresponds to different preset time periods respectively.
Specifically, the same JVM process means that the process identifications of the JVM processes are the same.
S1403: and obtaining the difference range of the occupied quantity of the heap memories by applying the average value and the variance of the occupied quantity difference of the heap memories of each difference group of the occupied quantity of the heap memories.
Specifically, each of the heap memory occupancy difference groups corresponds to a unique heap memory occupancy difference range, and each JVM process has one heap memory occupancy difference group for different preset time periods, so that the same process identifies multiple heap memory occupancy difference ranges that can correspond to each other.
S1404: and storing the difference range of the occupied quantity of the heap memory, the preset time period and the process identifier which respectively correspond to each difference group of the occupied quantity of the heap memory in the JVM monitoring first threshold value table.
Specifically, the preset JVM monitoring first threshold value table includes a plurality of occupancy difference range groups, each occupancy difference range group includes a unique process identifier, a preset time period, and heap memory occupancy difference range information, and the preset time periods of the occupancy difference range groups corresponding to the same process identifier are different. Because the load degrees of the application system have certain differences on different dates and different time periods, the occupied capacity of the heap memory, the number of the object instances and the occupied capacity of the heap memory have different characteristics, different time periods are determined to distinguish different loads, and each preset time period can be set according to actual needs, which is not limited in the application.
Further, before step S1401, the method may further include:
s1501: acquiring a plurality of history heap memory occupation volume records corresponding to the JVM processes, wherein each history heap memory occupation volume record comprises: the unique process identification, the historical heap memory occupancy and a second acquisition time for acquiring the historical heap memory occupancy.
Specifically, second acquisition times of all historical heap memory occupancy records corresponding to the same JVM process are different, and the second acquisition time may be system time for determining the historical heap memory occupancy in a target JVM server; the heap memory occupation amount, the current system time and the process identification corresponding to the JVM process in the target JVM server can be sent to the target kafka cluster in a json character string mode by adopting a wire-buried component mode at fixed time intervals, and the distributed database receives the heap memory occupation amount, the current system time and the process identification in the target kafka cluster and stores the heap memory occupation amount, the current system time and the process identification as the heap memory occupation amount record in a preset JVM heap memory occupation amount table.
S1502: and sequencing the historical heap memory occupancy corresponding to each process identifier respectively in sequence according to the front and back sequence of the second acquisition time, and acquiring the difference between every two adjacent sequenced historical heap memory occupancy as the difference of the heap memory occupancy. Specifically, the memory occupation amounts of the historical heaps corresponding to the same process identifier are sequentially sorted according to the front-back sequence of the second acquisition time; the difference between the two sorted historical memory occupancy amounts can be a difference value obtained by subtracting the former from the latter.
S1503: and taking the second acquisition time of the latter one of the two adjacent sorted historical internal storage occupancy rates as the first acquisition time of the corresponding internal storage occupancy rate difference, generating an internal storage occupancy rate difference record based on each internal storage occupancy rate difference, the first acquisition time corresponding to each internal storage occupancy rate difference and the process identification, and storing the internal storage occupancy rate difference record in the internal storage occupancy rate difference table. Specifically, the second obtaining time of the last historical memory occupancy of the two adjacent sorted historical memory occupancies is used as the first obtaining time of the difference of the memory occupancies corresponding to the two adjacent sorted historical memory occupancies.
Further, the object instance deficit record may further include: a third acquisition time of the difference in the number of the object instances; correspondingly, before step S1202, the method further includes:
s1601: and acquiring a plurality of object instance difference records from the preset object instance difference table.
Wherein each of the object instance deficit records comprises: and the process identification corresponding to the unique JVM process and the difference of the number of the object instances.
S1602: and dividing the object instance number difference of each object instance difference record into a plurality of groups of object instance number difference groups according to the third acquisition time, a preset time period and a process identifier, wherein each object instance number difference group of the same JVM process corresponds to different preset time periods respectively.
S1603: and obtaining the difference range of the number of the object examples by applying the mean and the variance of the difference of the number of the object examples in the difference group of the number of the object examples.
Specifically, the object instance number difference group and the object instance number difference range are in a one-to-one correspondence relationship; the difference of the occupation amounts of a plurality of historical heap memories in the same JVM process in the same preset time period can be processed by applying a three-sigma algorithm to obtain the difference range of the number of the object instances.
S1604: and storing the object instance number difference range, the preset time period and the process identification which respectively correspond to each object instance number difference group in a JVM monitoring second threshold table.
Further, before step S1601, the method may further include:
s1701: obtaining a plurality of history object instance records corresponding to each JVM process, wherein each history object instance record comprises: the number of the historical object instances and the fourth acquisition time for acquiring the number of the historical object instances. Specifically, the fourth obtaining time of each history object instance record corresponding to the same JVM process is different, and the fourth obtaining time may be system time for determining the number of history object instances in the target JVM server; the number of object instances corresponding to the JVM process in the target JVM server, the current system time, that is, the fourth acquisition time and the process identifier may be sent to the target kafka cluster at fixed time intervals by using a wire-burying component, and the distributed database receives the number of object instances, the current system time and the process identifier in the target kafka cluster, and stores the object instances, the current system time and the process identifier as object instance records in a preset object instance statistical table; in addition, the heap memory occupancy amount, the number of object instances, the current system time and the process identifier can be collected at fixed time intervals, the heap memory occupancy amount, the current system time and the process identifier are stored in a preset JVM heap memory occupancy table, and the number of the object instances, the current system time and the process identifier are stored in a preset object instance statistical table.
S1702: and sorting the number of the historical object instances corresponding to each process identifier respectively according to the front and back sequence of the fourth acquisition time, and obtaining the difference value between the numbers of the two adjacent sorted historical object instances as the difference of the numbers of the object instances. Specifically, the number of the historical object instances corresponding to the same process identifier is sequentially sorted according to the front-back sequence of the fourth acquisition time; the difference between the numbers of the two sorted history object instances adjacent to each other may be a difference value obtained by subtracting the former from the latter.
S1703: and taking the fourth acquisition time of the latter one of the two adjacent sorted historical object instance numbers as the third acquisition time of the corresponding object instance number difference, generating an object instance difference record based on the third acquisition time and the process identifier corresponding to each object instance number difference and each heap memory occupation amount difference, and storing the object instance difference record in the object instance difference table. Specifically, the fourth obtaining time of the next history object instance number in the two adjacent sorted history object instance numbers is used as the third obtaining time of the difference of the object instance numbers corresponding to the two adjacent sorted history object instance numbers.
Further, before step S1201, the method may further include:
s1801: obtaining a plurality of history object instance records corresponding to each JVM process, wherein each history object instance record comprises: the historical object instance number, the memory sum occupied by the object instances, and the fifth acquisition time for acquiring the historical object instance number and the memory sum occupied by the object instances. Specifically, fifth obtaining time of each history object instance record corresponding to the same JVM process is different, where the fifth obtaining time may be system time for determining the total memory usage amount of the history object instances and the object instances in the target JVM server; the number of the historical object instances is the number of the object instances when the JVM process runs, and the sum of the memory occupied by the object instances is the sum of the memory occupied by each historical object instance corresponding to the number of the historical object instances in the same historical object instance record.
S1802: and sorting the history object instance records corresponding to each process identifier respectively according to the front and back sequence of the fifth acquisition time, obtaining the difference value of the number of the history object instances between every two adjacent sorted history object instance records as the difference of the number of the object instances, and obtaining the difference value of the sum of the memory occupied by the object instances between every two adjacent sorted history object instance records as the total difference of the memory occupied by the object instances. It can be understood that the history object instance records corresponding to the same process identifier are sorted according to the front-back order of the fifth acquisition time.
S1803: and generating an object instance difference record based on the number difference of the object instances corresponding to the two adjacent sorted historical object instance records, the total difference of the memory occupied by the object instances and the process identification, and storing the object instance difference record in the object instance difference table.
In order to further improve the accuracy of the monitoring result of the JVM, in an application example of the present application, a fifth obtaining time of a subsequent one of the two adjacent sorted historical object instance records is used as a third obtaining time of a corresponding object instance number difference, and an object instance difference record is generated and stored in the object instance difference table based on each object instance number difference, a total difference of memory occupied by the object instance, a third obtaining time corresponding to each object instance number difference, and a process identifier. And generating a JVM monitoring second threshold table according to an object instance difference table containing the total difference of the memory occupied by the object instances, further judging whether the total difference of the memory occupied by the object instances of the newly added object instance difference record is within the range of the total difference of the memory occupied by the corresponding object instances, if not, determining that the memory occupied by the object instances of the JVM process corresponding to the object instance difference record is abnormal in growth, and further improving the accuracy of monitoring the abnormal memory occupied by the object instances.
To further illustrate the present solution, the present application further provides a specific application example of a JVM monitoring method, as shown in fig. 3, which is described in detail as follows:
s1: collecting JVM monitoring information. A general embedded point component tool is adopted, a background thread special for collecting JVM monitoring information is started in the background when a JVM is started, and the JVM monitoring information is uploaded to a monitoring information collection kafka cluster at fixed time intervals in a json string format according to an agreed interface. In order to trace the source of the uploaded JVM monitoring information, a source identifier of a monitored JVM process is uploaded together, the source identifier of the JVM process comprises a server IP address, a JVM process name, an application system code and system time, the source identifier is equivalent to the process identifier and the acquisition time, and the monitoring information uploaded to the kafka cluster comprises two types of information: 1. the JVM heap memory occupancy, namely the runtime heap memory usage information of the monitored JVM; 2. the statistical information of the JVM heap memory object instances, namely the memory amount occupied by the runtime heap memory object instances of the monitored JVM, the number of the object instances and the full path class name of the object.
S2: the JVM monitoring information is stored in a classified manner. And (3) extracting the collected JVM monitoring information from the kafka cluster, extracting the monitoring information according to the type (heap memory occupancy and in-heap object instance statistical information) of the monitoring information, and storing the monitoring information after classification into a JVM heap memory occupancy table and an object instance statistical table of a distributed database in sequence, wherein the IP address of the server, the process name of the JVM and the application system code are logic primary keys.
S3: and (5) analyzing and calculating. And executing SQL statements in the distributed database to complete the creation of the three sigma threshold of the monitored JVM process and the analysis of abnormal heap memory occupation allocation of the JVM process.
Checking and calculating the quantity of occupied JVM heap memory and the quantity of object instances, calculating a difference value of the occupied quantity of the heap memory collected from the same JVM process, namely subtracting the information of the occupied quantity of the heap memory collected at the 1 st time from the occupied quantity of the heap memory collected at the 2 nd time, subtracting the information of the occupied quantity of the heap memory collected at the 2 nd time from the occupied quantity of the heap memory collected at the 3 rd time, and analogizing to subtracting the information of the occupied quantity of the heap memory collected at the (N-1) th time from the occupied quantity of the heap memory collected at the Nth time to generate a difference table of the occupied quantity of the heap memory. Similarly, the difference values are respectively calculated for the number of the object instances collected from the same JVM process and the total memory occupied by the object instances, and an object instance number difference table, namely the object instance difference table, is generated.
Because the load degrees of the application system have certain differences on different dates and different time periods, the occupied quantity of the heap memory, the number of the object instances and the occupied quantity of the memory have different characteristics, and the workload calendar is established to distinguish different loads. For example, the workload calendar is shown in Table 1 below:
TABLE 1
Serial number Time period Time horizon
1.0 Common workday 8:30~11:30、13:30~18:00、18:00~21:00、21:00~8:30
2.0 1 at the beginning of the month 8:30~11:30、13:30~18:00、18:00~21:00、21:00~8:30
3.0 At the end of the month 8:30~11:30、13:30~18:00、18:00~21:00、21:00~8:30
4.0 Double 11, double 12 8:30~11:30、13:30~18:00、18:00~23:00、23:00~8:30
5.0 Rest day 8:30~11:30、13:30~18:00、18:00~21:00、21:00~8:30
As can be seen from table 1, the workload calendar is divided into 5 time periods, the heap memory occupancy margin table is subjected to time dimension screening and filtering according to table 1, the identifier traceability information of the JVM with three dimensions of server IP address, JVM process name and application system code is used for grouping, the mean value μ and standard deviation σ of the heap memory margin in different time periods on different dates are calculated, μ -3 σ and μ +3 σ are calculated, and the calculated mean value μ and standard deviation σ are stored in the JVM monitoring first threshold value table (the server IP address, the JVM process name, the application system code and the time period identifier field are logical primary keys).
And (3) carrying out time dimension screening and filtering on the object instance number difference table according to the working calendar table, grouping by using identification traceability information of JVMs with three dimensions of server IP addresses, JVM process names and application system codes, calculating the mean value mu and standard difference sigma of the object instance number difference and the object occupied memory difference in different time periods on different dates, calculating mu-3 sigma and mu +3 sigma, and storing the mean value mu and the standard difference sigma in the JVM monitoring second threshold table (the server IP addresses, the JVM process names, the application system codes and the time period identification fields are logical primary keys).
Grouping all newly collected records stored in a difference table of the internal memory occupancy of the JVM by using the IP address of a server, the process name of the JVM and identification traceability information of the JVM with three dimensions encoded by an application system, and respectively executing three sigma algorithm verification, wherein if the newly collected records fall into the range of a first threshold table monitored by the JVM, the increase difference of the internal memory occupancy of the JVM is considered to be normal; if the number of the object instances does not fall within the range of the JVM monitoring second threshold value table, the number of the object instances is considered to be normal, and if the number of the object instances does not fall within the range of the JVM monitoring second threshold value table, the number of the object instances is considered to be abnormally increased. And summarizing and calculating the difference memories of all the object instances with the abnormal increase of the number of the object instances, if the difference memories exceed a third threshold value of 16MB, considering that the heap memory is abnormally increased, and generating early warning information to an early warning information table.
Specifically, the method comprises the steps that the heap memory occupancy information of the same JVM process is collected, the difference value of the heap memory occupancy obtained by subtracting the (N-1) th heap memory occupancy from the nth heap memory occupancy is in accordance with normal distribution, and therefore the heap memory occupancy condition of the same JVM process is verified and analyzed based on a three-sigma algorithm; for the number of the object instances and the memory occupancy information of the object instances acquired by the same JVM process, the difference value of the number of the object instances acquired for the Nth time minus the number of the object instances acquired for the (N-1) th time and the memory occupancy of the object instances acquired for the Nth time minus the memory occupancy of the object instances acquired for the (N-1) th time accords with normal distribution, and therefore verification analysis is conducted on the number of the object instances and the memory occupancy of the object instances of the same JVM process based on a three-sigma algorithm.
S4: and (6) alarm notification. And reading the early warning information from the early warning information table, acquiring the mobile phone number and the mail contact address of the person in charge, and sending a short message and a mail to inform the person in charge.
S5: and (6) flexible query. After the related responsible person receives the alarm notification, the flexible query module can be used for sending SQL query analysis statements to the distributed database to analyze and view the reasons related to the abnormal growth of the memory of the JVM heap and determine the number of the object instances with abnormal growth.
S6: and (6) clearing data. In order to control the increase of the capacity of the distributed database for storing various types of monitoring data of the JVM process, the outdated data cleaning is required to be performed on the data tables in the distributed database to release the storage space.
Referring to fig. 5, the JVM monitoring information collection process in this application example is described in detail as follows:
step 300: acquiring source identification information of a monitored JVM process; the source identification information includes: the IP address of the server, the JVM process name and the identification code of the application system are used for marking source identification information for the monitoring information collected from the JVM process, and the collected JVM monitoring information can be traced.
Step 301: acquiring a global snapshot running sequence number of a monitored JVM process; in each acquisition cycle, incremental serial number information needs to be acquired based on the source identification information of the JVM process, and is used as the tag ID information of the JVM monitoring information acquired in the current acquisition cycle, and the incremental serial number is generated based on the serial number of the last acquisition cycle plus 1.
Step 302: acquiring current system time; specifically, the system time in the monitored JVM process is obtained in the format YYYMMDD hh24: mi: ss.
Step 303: acquiring the memory occupancy information of the JVM heap, and writing the memory occupancy information into a kafka cluster; specifically, heap memory occupancy information of the monitored JVM process is acquired, and together with the information acquired in steps 300 to 302, uploaded to the kafka cluster in json format.
Step 304: acquiring statistical information of memory objects of a spliced JVM heap, and writing the statistical information into a kafka cluster; specifically, statistical information of the monitored JVM heap memory objects is obtained, and together with the information obtained in steps 300 to 302, uploaded to the kafka cluster in a json format.
In order to prevent the alarm storm caused by excessive alarms, it is necessary to control the number of alarms of the same monitored JVM and set the time interval of continuous alarms, and referring to fig. 6, the alarm notification process in this application example is described as follows:
step 400: reading early warning information and a source identifier from an early warning information table; the source identifier includes: the server IP address of the JVM process, the JVM process name and the application system identification coding information.
Step 401: calculating the continuous alarming times of the same JVM process; and calculating the times of continuously generating the alarm message within a certain time interval according to the IP address of the server, the process name of the JVM and the identification coding information of the application system, and alarming when the times reach or exceed the threshold value for 2 times. The reason why the alarm is given when the number of times of continuously generating the alarm information is 2 or more is that it is necessary to eliminate the influence of factors such as the restart of the GC and the JVM process.
Step 402: JVM source identification information, namely a server IP address, a JVM process name and application system identification coding information of a JVM process, is obtained.
Step 403: acquiring JVM last-time alarm information; and acquiring last alarm time information of the JVM process according to the JVM source identification information, and searching an alarm history table for alarm notification if the alarm history table does not have the alarm record of the JVM process notifying a responsible person within a time interval specified by a threshold value.
Step 404: obtaining phone and mail of person in charge according to the source identification information of the monitoring JVM; and determining a person responsible for the JVM process according to the IP address of the server, the name of the JVM process and the identification code of the application system, and acquiring a contact telephone and a mailbox of the person responsible for the JVM process.
Step 405: and calling a short message sending interface to inform the responsible person of the alarm message.
Step 406: and calling a mail sending interface to inform the responsible person of the alarm message.
Step 407: inserting the alarm information into an alarm history table; and archiving and managing the historical early warning information.
Referring to fig. 7, the flexible query analysis process in the present application example is described as follows:
step 500: receiving SQL statements to inquire and analyze the reasons of abnormal heap memory growth; and the related responsible person inputs an SQL statement after receiving the alarm notification to inquire and analyze the reason causing the abnormal growth of the JVM heap memory and locate the abnormal heap memory allocation caused by the abnormal growth of the number of the object instances. A commonly used SQL query statement is provided as a query analysis template.
Step 501: checking the input SQL statement; and carrying out security check on the input SQL statement, and prohibiting accessing to the unauthorized table object, wherein the SQL statement is a must-SELECT query statement.
Step 502: submitting SQL sentences to a distributed database; and submitting SQL statements passing the security check to a distributed database for execution.
Step 503: and obtaining a query result of the SQL statement.
Step 504: returning the query result; and returning the query result to the related responsible person for analysis.
Since the application example relates to real-time acquisition and storage of monitoring information of a monitored JVM process and generation and storage of derivative data, it is necessary to perform an outdated cleaning operation on various types of data on the premise of satisfying data lifecycle management to prevent an increase in storage capacity of a distributed database, and referring to fig. 8, a data cleaning process in the application example is described in detail as follows:
step 601: and generating a data cleaning strategy according to the data life cycle retention strategy.
Step 602: and deleting the expired data of the object instance statistical table according to the data cleaning strategy.
Step 603: and deleting the overdue data of the JVM heap memory occupation table according to the data cleaning strategy.
Step 604: and deleting the overdue data of the object instance number difference table according to the data cleaning strategy.
Step 605: and deleting the overdue data of the heap memory occupancy amount difference table according to a data cleaning strategy.
In terms of software, in order to improve efficiency and accuracy of monitoring a JVM process and further ensure normal operation of a service system, the present application provides an embodiment of a JVM process monitoring apparatus for implementing all or part of contents in the JVM process monitoring method, where, referring to fig. 9, the JVM process monitoring apparatus specifically includes the following contents:
an obtaining module 10, configured to obtain a heap memory occupancy difference of a target JVM process.
And the determining module 20 is configured to acquire a heap memory occupation amount difference range corresponding to the target JVM process from a preset JVM monitoring first threshold table.
A determining module 30, configured to determine whether the heap memory occupancy difference is within the heap memory occupancy difference range, and if not, determine that the heap memory growth of the target JVM process is abnormal.
The embodiment of the JVM process monitoring apparatus provided in this specification may be specifically configured to execute the processing procedure of the embodiment of the JVM process monitoring method, and its functions are not described herein again, and refer to the detailed description of the embodiment of the JVM process monitoring method.
From the above description, it can be known that the JVM process monitoring method and apparatus provided by the present application can improve efficiency and accuracy of JVM process monitoring, and further can ensure normal operation of a service system. Specifically, the effects and advantages of the present application are mainly reflected in the following aspects:
1. the acquisition of various monitoring information is easy to expand. Monitoring information of the monitored JVM is acquired by adopting a point embedding tool through a preset interface, the range of acquiring the monitoring information is easy to expand subsequently, for example, number information of running threads, thread blocking information, GC information and the like of the JVM are acquired, key service indexes of an application layer can be acquired, and monitoring of full coverage of a JVM process technology layer and a service layer is realized;
2. and early warning is timely. When the abnormal memory allocation of the monitored JVM process heap is found, a notification alarm mechanism can be triggered in time to notify a responsible person, and emergency treatment measures are executed to stop loss in time;
3. and (5) treating production environment codes. For example, when a large number of SQL statements without binding variables are executed in a business system, data is exported from a foreground web page, the number of export lines is not limited, or a web query page is not paged, and other problems occur, a large number of fine-grained Java objects are generated in the JVM heap memory, and the increase of the heap memory usage is abnormal. The object instance occupying the memory can be positioned by analyzing the exception of the Java heap memory occupied by the instance object, so that the function module and the program code which have problems can be positioned, the problem program code can be repaired in a targeted manner, the occurrence of the OutOfMemoryError memory overflow in the production environment can be greatly reduced, the effect of managing the production environment program code can be achieved, and the stable operation of a service system can be powerfully guaranteed;
4. the SQL-based analysis method is simple, efficient, convenient and flexible. Various monitoring information collected from the JVM is stored in a distributed database, the mean value mu and the standard deviation sigma can be calculated by means of an SQL function of the database, and further analysis on various kinds of stored JVM monitoring data can be efficiently and quickly achieved by compiling SQL sentences. After receiving the JVM heap memory occupation quantity abnormal alarm notification, SQL sentences can be input to determine the object instances causing the memory occupation abnormity.
In terms of hardware, in order to improve efficiency and accuracy of monitoring a JVM process and further ensure normal operation of a service system, an embodiment of an electronic device for implementing all or part of contents in the method for monitoring a JVM process provided by the present application includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete mutual communication through the bus; the communication interface is used for realizing information transmission between the JVM process monitoring device and related equipment such as a user terminal; the electronic device may be a desktop computer, a tablet computer, a mobile terminal, and the like, but the embodiment is not limited thereto. In this embodiment, the electronic device may be implemented with reference to the embodiment for implementing the JVM process monitoring method and the embodiment for implementing the JVM process monitoring apparatus in the embodiments, and the contents thereof are incorporated herein, and repeated details are not described herein.
Fig. 10 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 10, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 10 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one or more embodiments of the present application, the JVM process monitoring function may be integrated into the central processor 9100. The central processor 9100 may be configured to control as follows:
s11: acquiring a difference of the occupied quantity of the heap memory of a target JVM process;
s12: acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table;
s13: and judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
As can be seen from the above description, the electronic device provided in the embodiments of the present application can improve efficiency and accuracy of monitoring the JVM process, and further can ensure normal operation of a service system.
In another embodiment, the JVM process monitoring apparatus may be configured separately from the central processor 9100, for example, the JVM process monitoring apparatus may be configured as a chip connected to the central processor 9100, and the JVM process monitoring function is implemented by the control of the central processor.
As shown in fig. 10, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 also does not necessarily include all of the components shown in fig. 10; in addition, the electronic device 9600 may further include components not shown in fig. 10, which can be referred to in the prior art.
As shown in fig. 10, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, an LCD display, but is not limited thereto.
The memory 9140 can be a solid state memory, e.g., Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
As can be seen from the above description, the electronic device provided in the embodiment of the present application can improve efficiency and accuracy of monitoring the JVM process, and further can ensure normal operation of a service system.
An embodiment of the present application further provides a computer-readable storage medium capable of implementing all the steps in the JVM process monitoring method in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and the computer program implements all the steps of the JVM process monitoring method in the foregoing embodiment when being executed by a processor, for example, the processor implements the following steps when executing the computer program:
s11: acquiring a difference of the occupied quantity of the heap memory of a target JVM process;
s12: acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table;
s13: and judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
As can be seen from the foregoing description, the computer-readable storage medium provided in the embodiment of the present application can improve efficiency and accuracy of monitoring a JVM process, so as to ensure normal operation of a service system.
In the present application, each embodiment of the method is described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. Reference is made to the description of the method embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the present application are explained by applying specific embodiments in the present application, and the description of the above embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (11)

1. A JVM process monitoring method is characterized by comprising the following steps:
acquiring a difference of the occupied quantity of the heap memory of a target JVM process;
acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table;
and judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
2. The JVM process monitoring method of claim 1, further comprising, before the obtaining the heap memory footprint difference range corresponding to the target JVM process:
acquiring the difference of the number of object instances of a target JVM process;
correspondingly, after the determination of the heap memory growth exception of the target JVM process, the method further includes:
acquiring a difference range of the number of object instances corresponding to the target JVM process from a preset JVM monitoring second threshold table;
and judging whether the difference of the number of the object instances is within the range of the difference of the number of the object instances, and if not, determining that the increase of the number of the object instances of the target JVM process is abnormal.
3. The JVM process monitoring method of claim 2, before the obtaining the difference range of the number of object instances corresponding to the target JVM process, further comprising:
acquiring the total difference of the object instances of the target JVM process in the memory;
correspondingly, after the determining that the number of the object instances of the target JVM process is abnormally increased, the method further includes:
and judging whether the total difference of the memory occupied by the object instance exceeds a threshold of the amount of the occupied memory, if so, determining that the memory occupied by the object instance of the target JVM process is abnormally increased.
4. The JVM process monitoring method of claim 1, further comprising:
acquiring historical heap memory occupancy difference records of a plurality of JVM processes;
and obtaining a plurality of difference ranges of the occupied quantity of the heap memory corresponding to each JVM process based on the history difference record of the occupied quantity of the heap memory and a three-sigma algorithm, and storing the difference ranges of the occupied quantity of the heap memory corresponding to each JVM process in the JVM monitoring first threshold value table, wherein each difference range of the occupied quantity of the heap memory of the same JVM process corresponds to different preset time periods respectively.
5. The JVM process monitoring method of claim 1, further comprising:
acquiring historical heap memory occupancy records of a plurality of JVM processes;
and generating a historical heap memory occupancy difference record of each JVM process based on the historical heap memory occupancy records.
6. The JVM process monitoring method of claim 2, further comprising:
acquiring historical object instance difference records of a plurality of JVM processes;
and obtaining a plurality of object instance number difference ranges corresponding to the JVM processes respectively based on the historical object instance difference record and the three-sigma algorithm, storing the object instance number difference ranges in the JVM monitoring second threshold table, wherein the object instance number difference ranges of the same target JVM process respectively correspond to different preset time periods.
7. The JVM process monitoring method of claim 2, further comprising:
acquiring historical object instance records of a plurality of JVM processes;
and generating a historical object instance difference record of each JVM process based on the historical object instance record.
8. The JVM process monitoring method of claim 3, further comprising, after the determining that the object instance of the target JVM process occupies a memory growth exception:
generating early warning information of abnormal memory growth occupied by the object instance corresponding to the target JVM process;
and judging whether the number of times of continuously generating the early warning information corresponding to the target JVM process in a preset first early warning time range exceeds a generation number threshold, if so, judging whether historical early warning information output corresponding to the target JVM process exists in a preset second early warning time range, and if not, outputting the generated early warning information.
9. A JVM process monitoring apparatus, comprising:
the acquisition module is used for acquiring the difference of the occupied quantity of the heap memory of the target JVM process;
the determining module is used for acquiring a difference range of the occupied quantity of the heap memory corresponding to the target JVM process from a preset JVM monitoring first threshold table;
and the judging module is used for judging whether the difference of the occupied quantity of the heap memory is within the range of the difference of the occupied quantity of the heap memory, and if not, determining that the heap memory of the target JVM process is abnormally increased.
10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the JVM process monitoring method of any of claims 1 to 8 when executing the program.
11. A computer readable storage medium having stored thereon computer instructions, wherein said instructions when executed implement the JVM process monitoring method of any of claims 1 to 8.
CN202010821228.2A 2020-08-14 2020-08-14 JVM process monitoring method and device Active CN111949490B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010821228.2A CN111949490B (en) 2020-08-14 2020-08-14 JVM process monitoring method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010821228.2A CN111949490B (en) 2020-08-14 2020-08-14 JVM process monitoring method and device

Publications (2)

Publication Number Publication Date
CN111949490A true CN111949490A (en) 2020-11-17
CN111949490B CN111949490B (en) 2024-04-19

Family

ID=73342501

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010821228.2A Active CN111949490B (en) 2020-08-14 2020-08-14 JVM process monitoring method and device

Country Status (1)

Country Link
CN (1) CN111949490B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112764985A (en) * 2020-12-30 2021-05-07 中国人寿保险股份有限公司上海数据中心 Intelligent monitoring method for data center system
CN114815641A (en) * 2022-03-29 2022-07-29 青岛海尔科技有限公司 Monitoring method and device for intelligent equipment, storage medium and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050204341A1 (en) * 2004-03-11 2005-09-15 International Business Machines Corp. Method, system and article for detecting critical memory leaks causing out-of-memory errors in Java software
CN104932970A (en) * 2015-05-29 2015-09-23 小米科技有限责任公司 Monitoring method and device of memory leakage
CN109597676A (en) * 2018-10-25 2019-04-09 平安科技(深圳)有限公司 Monitoring and early warning method, apparatus and storage medium based on JVM

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050204341A1 (en) * 2004-03-11 2005-09-15 International Business Machines Corp. Method, system and article for detecting critical memory leaks causing out-of-memory errors in Java software
CN104932970A (en) * 2015-05-29 2015-09-23 小米科技有限责任公司 Monitoring method and device of memory leakage
CN109597676A (en) * 2018-10-25 2019-04-09 平安科技(深圳)有限公司 Monitoring and early warning method, apparatus and storage medium based on JVM

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112764985A (en) * 2020-12-30 2021-05-07 中国人寿保险股份有限公司上海数据中心 Intelligent monitoring method for data center system
CN112764985B (en) * 2020-12-30 2024-05-17 中国人寿保险股份有限公司上海数据中心 Intelligent monitoring method for data center system
CN114815641A (en) * 2022-03-29 2022-07-29 青岛海尔科技有限公司 Monitoring method and device for intelligent equipment, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN111949490B (en) 2024-04-19

Similar Documents

Publication Publication Date Title
CN110428127B (en) Automatic analysis method, user equipment, storage medium and device
CN110489313B (en) Operation log recording method and device based on block chain and storage medium
CN110069572A (en) HIVE method for scheduling task, device, equipment and storage medium based on big data platform
CN105468492A (en) SE(search engine)-based data monitoring method and system
CN102902752A (en) Method and system for monitoring log
CN111949490B (en) JVM process monitoring method and device
CN112506743A (en) Log monitoring method and device and server
CN112559475B (en) Data real-time capturing and transmitting method and system
CN110009347B (en) Block chain transaction information auditing method and device
CN111324480B (en) Large-scale host transaction fault positioning system and method
CN115237857A (en) Log processing method and device, computer equipment and storage medium
CN111383104A (en) Method and device for acquiring fund investment supervision indexes
CN112241362A (en) Test method, test device, server and storage medium
CN113570379A (en) Abnormal transaction group partner identification method and device
CN112463799A (en) Data extraction method, device, equipment and storage medium
CN117036062A (en) Accounting resource calculation method and device
CN110532533B (en) Form precision collocation method, device, computer equipment and storage medium
CN112860527A (en) Fault monitoring method and device of application server
CN113344333A (en) Configurable enterprise credit investigation early warning method, device, equipment and storage medium
CN102521717A (en) Method and device for carrying out statistics on configuration management resource library information
CN112085497A (en) User account data processing method and device
CN112101810A (en) Risk event control method, device and system
CN111951011A (en) Monitoring system threshold determination method and device
CN112631850B (en) Fault scene simulation method and device
CN111930620B (en) Application running environment data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant