CN110865919B - Monitoring method and device based on java process and computer equipment - Google Patents

Monitoring method and device based on java process and computer equipment Download PDF

Info

Publication number
CN110865919B
CN110865919B CN201911067544.9A CN201911067544A CN110865919B CN 110865919 B CN110865919 B CN 110865919B CN 201911067544 A CN201911067544 A CN 201911067544A CN 110865919 B CN110865919 B CN 110865919B
Authority
CN
China
Prior art keywords
java
constructing
monitoring
local
machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911067544.9A
Other languages
Chinese (zh)
Other versions
CN110865919A (en
Inventor
王志
孟博
徐伟
孙迁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suning Cloud Computing Co Ltd
Original Assignee
Suning Cloud Computing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suning Cloud Computing Co Ltd filed Critical Suning Cloud Computing Co Ltd
Priority to CN201911067544.9A priority Critical patent/CN110865919B/en
Publication of CN110865919A publication Critical patent/CN110865919A/en
Application granted granted Critical
Publication of CN110865919B publication Critical patent/CN110865919B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3055Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3089Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
    • G06F11/3093Configuration details thereof, e.g. installation, enabling, spatial arrangement of the probes
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The application discloses a monitoring method and device based on a java process and computer equipment, wherein the monitoring method comprises the following steps: dynamically scanning a java process on a machine to obtain a process identifier; constructing a local link of a java process on a machine; constructing a manager agent of a java process in a monitoring process, and constructing a mapping from process identification to a multi-type resource management instance; and collecting monitoring data indexes corresponding to the java process, and converging the monitoring data indexes to the dimension of the cluster. The application solves the process monitoring requirement of the java program, and simultaneously monitors other existing systems in a non-invasive way; the realization of a single language environment based on pure java greatly improves the stability and expandability of the system; aiming at various acquisition indexes of different dimensions, the problem of pain points of users with multiple application scenes and multiple use requirements is solved based on the use of a multi-level architecture level.

Description

Monitoring method and device based on java process and computer equipment
Technical Field
The application belongs to the technical field of process monitoring, and particularly relates to a java process-based monitoring method, device and computer equipment.
Background
As big data technology matures gradually, the usage of heterogeneous data source data exchange tasks also reaches a huge magnitude. The system builds a large number of service process containers in a large number of servers in a cluster environment, obtains the service states and performances of all processes except the state monitoring of machines, and the state analysis and monitoring of the process containers become a focus problem to be solved.
There are many implementations of java process monitoring today. Firstly, directly calling linux commands such as jmap and top to realize the monitoring and collection of related indexes, and the method is simple and convenient to use, but has a larger performance problem. Each time a command is called is equivalent to starting a new process, when second-level high-frequency monitoring is needed, the influence on the performance of the system is large, the phenomena of slow response, more occupied CPU resources and the like are easy to generate, and even when the machine configuration is low, the machine is directly caused to be down. The other method is to directly use jdk self-contained tools, but the use is more complex, dynamic parameters are required to be set, meanwhile, the monitoring indexes are few, and the calculation of the related indexes of the CPU has deviation from the actual values.
Disclosure of Invention
The application aims to provide a monitoring method, a device and computer equipment based on a java process, which realize jvm monitoring service aiming at an actual business scene, wherein the monitoring method, the device and the computer equipment are high in performance, simple and flexible to use, easy to customize, easy to transplant and easy to maintain.
The technical scheme for realizing the purpose of the application is as follows: a monitoring method based on java process includes:
dynamically scanning a java process on a machine to obtain a process identifier;
constructing a local link of a java process on a machine;
constructing a manager agent of a java process in a monitoring process, and constructing a mapping from process identification to a multi-type resource management instance;
and collecting monitoring data indexes corresponding to the java process, and converging the monitoring data indexes to the dimension of the cluster.
A java process-based monitoring device, comprising:
the process identification acquisition module is used for dynamically scanning java processes on the machine to acquire process identifications;
the local link construction module is used for constructing a local link of a java process on the machine;
the mapping relation establishing module is used for constructing a manager agent of the java process in the monitoring process and mapping the process identification to the multi-type resource management instance;
the aggregation module is used for collecting monitoring data indexes corresponding to the java process and aggregating the monitoring data indexes to the dimension of the cluster.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above-mentioned monitoring method when executing the computer program.
A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described monitoring method.
Compared with the prior art, the application has the remarkable advantages that: (1) The application is based on the single-thread implementation mode of pure java, abandons the acquisition of resource use data from an operating system, and reduces the system interaction to the minimum; (2) The monitoring process is not invasive, and modification and reconstruction of other process systems are not needed, so that the invasiveness is reduced to the minimum; (3) The method is different from the mode of realizing process performance index collection in the traditional process, and index data collection is completed outside the process; (4) high availability, easy expansion, easy deployment characteristics; the acquisition architecture has no any external resource dependence, and the characteristic of easy expansion and deployment is greatly improved; (5) The monitoring technology of the application can complete the data acquisition of the fullGC and the youngGC.
Drawings
FIG. 1 is a flowchart of a java process-based monitoring method of the present application.
FIG. 2 is a schematic diagram of a process for obtaining process identification.
FIG. 3 is a schematic diagram of a local link of a process on a build machine.
FIG. 4 is a schematic diagram of a build process identifying a mapping to multiple types of resource management instances.
FIG. 5 is a schematic diagram of aggregating process index data of java into a cluster dimension.
FIG. 6 is a schematic diagram of an unhealthy detection standard for a JVM service process constructed for different metrics in accordance with embodiment 1.
Fig. 7 is a flowchart of task allocation scheduling in embodiment 2.
Detailed Description
With big data, cloud computing and the development of service architecture, a large number of java-based software systems implemented by virtual machines need to be built on a computer server. Most existing monitoring systems stay in monitoring of hardware resource consumption, and resource monitoring based on java processes and technical tools for analyzing the running state of a software system are needed.
The application provides a monitoring method based on a java process, which is shown in fig. 1 and comprises the following steps:
dynamically scanning a java process on a machine to obtain a process identifier;
constructing a local link of a java process on a machine;
constructing a manager agent of a java process in a monitoring process, and constructing a mapping from process identification to a multi-type resource management instance;
and collecting monitoring data indexes corresponding to the java process, and converging the monitoring data indexes to the dimension of the cluster.
Further, as shown in fig. 2, a specific method for obtaining the process identifier is as follows:
constructing a timing scheduling task, and setting a task scheduling interval according to requirements;
constructing a local monitored host;
and acquiring the java process identifier running on the monitored host.
Further, as shown in fig. 3, the specific method for constructing the local link of the Java process on the machine is as follows:
obtaining an operating virtual machine instance of a monitored host with consistent java process identification;
acquiring a file package of a local management agent;
the virtual machine instance loads a file package of the management agent to realize remote management of the virtual machine instance;
the URL of the inter-process link of the local machine based on jmx is obtained.
Further, as shown in fig. 4, the mapping from the process identification to the multi-type resource management instance is constructed by the following specific method:
constructing a link based on jmx based on the address link URL of the last step;
constructing a service linker of a management instance class platform;
obtaining resource management examples of various types in a management service platform;
and converging the resource management examples acquired on the basis of the link to form a mapping relation from the process identifier to the multi-kind management examples.
Further, the monitoring data indexes corresponding to the java process are collected and converged to the dimension of the cluster, as shown in fig. 5, specifically:
using various management examples under the process identification to obtain monitoring data indexes of corresponding dimensions and granularity;
the monitoring data index of the current process is added with the process identifier to be converged to a local cache on an upper machine level;
and asynchronously extracting data from the local cache and adding a machine identifier to perform aggregation to an external storage medium storage server on an upper layer cluster level.
Wherein the monitor data indicator comprises GC-related data, stack-related data, threads, class loading and/or environmental information data indicators.
Preferably, the local cache is a ring buffer and the external storage medium storage server is a mysql or es storage server.
The application also provides a monitoring device based on the java process, which comprises:
the process identification acquisition module is used for dynamically scanning java processes on the machine to acquire process identifications;
the local link construction module is used for constructing a local link of a java process on the machine;
the mapping relation establishing module is used for constructing a manager agent of the java process in the monitoring process and mapping the process identification to the multi-type resource management instance;
the aggregation module is used for collecting monitoring data indexes corresponding to the java process and aggregating the monitoring data indexes to the dimension of the cluster.
The computer equipment comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the steps of the monitoring method when executing the computer program.
The application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the above-described monitoring method.
The monitoring method and the device support dynamic scanning perception of the java process running on the machine; supporting the free java process on the machine to be brought into a monitoring range; supporting analysis of the system running state of the java process; supporting dynamic statistics of use resources of the java process; supporting unified monitoring of multiple processes on multiple machines in a cluster environment; the application can improve the effective utilization rate of machine resources, predict the unhealthy state of system service as early as possible, and greatly improve the stability of the system by adopting a pure java implementation mode.
The application of the java process index is further described below with reference to the embodiments and the accompanying drawings.
Example 1
Unhealthy detection criteria for java processes are constructed for different metrics, as shown in fig. 6.
(1) To fully detect the unhealthy degree of the current process service, unhealthy degree detection standards for different dimension directions are constructed.
(2) And (3) referring to the quantitative threshold values such as excessively frequent FULLGC, blocked threads and excessively large heap memory usage, constructing an index filter chain, and filtering the acquired progress index.
(3) The process of triggering the measurement threshold value in the filter chain is used for indicating the unhealthy of the current service process, possibly generating service failure or unsatisfactory service effect, and the current service process needs to be intervened and processed.
(4) And sending a short message and mail alarm to prompt a system administrator in time.
Example 2
Aiming at the current Java service process information feedback, task allocation scheduling is carried out:
(1) And evaluating the service capability of the current service process according to the collected process monitoring index information.
(2) If the current service process has good performance, the subsequent service tasks can be distributed, and the distribution method is shown in fig. 7.
Example 3
Monitoring of service processes of multiple types and multiple architectures is orchestrated:
the existing monitoring means are mostly built in the process, and the index monitoring is realized. Different architectures and different service types are caused, and processes in different languages occupy machine system resources respectively, but cannot realize monitoring comprehensively. If a micro service item is deployed and a bottom platform system process such as spark, resume, hive and the like is operated on a certain machine at the same time, the monitoring of each technical architecture cannot sense the service process operation condition of other frames.
The system service processes with various architectures can realize noninvasive overall monitoring based on the realization of jvm. Technical support is provided for the application of the multi-technical framework in the same project system.
The application solves the performance problems of slow response, more occupied resources and the like possibly existing in the prior art, and overcomes the defects of complex use and high maintenance cost. In the process of monitoring the related indexes obtained by high frequency, objects and memory spaces are not required to be frequently created, only objects and a ring buffer area are required to be created when the service is started, the monitoring service is used as a producer to write index data into the ring buffer area, and a consumer continuously takes out data from the ring buffer area and writes the data into a database. Due to the adoption of the annular buffer area, the problem that the buffer area is overrun due to the fact that continuous production data possibly occurs due to abnormal consumption can be effectively avoided. In addition, the system is deployed on each server, and only monitors the local machine, so that the connection and data problems possibly generated due to multi-machine communication are avoided. The performance comparison of the present application with the prior art process is shown in the following table.
TABLE 1 comparison of the Performance of the application with the prior art
By using the technical scheme of the application, when the method is applied to process monitoring, the requirement of java process monitoring constructed on jvm is solved, and the method also has the advantage of non-invasive monitoring on other existing systems. The realization of a single language environment based on pure java greatly improves the stability and expandability of the system. Aiming at various acquisition indexes of different dimensions, the problem of pain points of users with multiple application scenes and multiple use requirements is perfectly solved based on the use of multi-level architecture levels.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims (6)

1. The monitoring method based on the java process is characterized by comprising the following steps of:
dynamically scanning a java process on a machine to obtain a process identifier, wherein the process identifier specifically comprises the following steps of:
constructing a timing scheduling task, and setting a task scheduling interval according to requirements;
constructing a proxy object of a local monitored host;
acquiring a java process identifier running on a monitored host;
the method comprises the steps of constructing a local link of a java process on a machine, wherein the local link is specifically as follows:
obtaining an operating virtual machine instance of a monitored host with consistent java process identification;
acquiring a file package of a local management agent;
the virtual machine instance loads a file package of the management agent to realize remote management of the virtual machine instance;
acquiring a URL of an inter-process link of the jmx-based local machine;
constructing a manager agent of a java process in a monitoring process, and constructing a mapping from process identification to a multi-type resource management instance, wherein the mapping is specifically as follows:
constructing a link based on jmx based on the address link URL of the last step;
constructing a service linker of a management instance class platform;
obtaining resource management examples of various types in a management service platform;
aggregating the resource management examples obtained on the basis of the link to form a mapping relation from the process identifier to the multi-kind management examples;
collecting monitoring data indexes corresponding to java processes, and converging the monitoring data indexes to a cluster dimension, wherein the monitoring data indexes are specifically as follows:
using various management examples under the process identification to obtain monitoring data indexes of corresponding dimensions and granularity;
the monitoring data index of the current process is added with the process identifier to be converged to a local cache on an upper machine level;
and asynchronously extracting data from the local cache and adding a machine identifier to perform aggregation to an external storage medium storage server on an upper layer cluster level.
2. The method of claim 1, wherein the monitor data indicators comprise stack related data, threads, class loading, and environmental information data indicators.
3. The method for monitoring a java based process according to claim 1, wherein the local buffer is a ring buffer, and the external storage medium storage server is a mysql or es storage server.
4. The utility model provides a monitoring device based on java process which characterized in that includes:
the process identifier acquisition module is used for dynamically scanning java processes on the machine to acquire process identifiers, and specifically comprises the following steps: constructing a timing scheduling task, and setting a task scheduling interval according to requirements; constructing a proxy object of a local monitored host; acquiring a java process identifier running on a monitored host;
the local link construction module is used for constructing a local link of a java process on a machine, and specifically comprises the following steps: obtaining an operating virtual machine instance of a monitored host with consistent java process identification; acquiring a file package of a local management agent; the virtual machine instance loads a file package of the management agent to realize remote management of the virtual machine instance; acquiring a URL of an inter-process link of the jmx-based local machine;
the mapping relation establishing module is used for constructing a manager agent of a java process in a monitoring process, and the construction process identifies mapping to a multi-type resource management instance, and specifically comprises the following steps: constructing a link based on jmx based on the address link URL of the last step; constructing a service linker of a management instance class platform; obtaining resource management examples of various types in a management service platform; aggregating the resource management examples obtained on the basis of the link to form a mapping relation from the process identifier to the multi-kind management examples;
the aggregation module is used for collecting monitoring data indexes corresponding to java processes, aggregating the monitoring data indexes to the dimension of the cluster, and specifically comprises the following steps: using various management examples under the process identification to obtain monitoring data indexes of corresponding dimensions and granularity; the monitoring data index of the current process is added with the process identifier to be converged to a local cache on an upper machine level; and asynchronously extracting data from the local cache and adding a machine identifier to perform aggregation to an external storage medium storage server on an upper layer cluster level.
5. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 3 when the computer program is executed by the processor.
6. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 3.
CN201911067544.9A 2019-11-04 2019-11-04 Monitoring method and device based on java process and computer equipment Active CN110865919B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911067544.9A CN110865919B (en) 2019-11-04 2019-11-04 Monitoring method and device based on java process and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911067544.9A CN110865919B (en) 2019-11-04 2019-11-04 Monitoring method and device based on java process and computer equipment

Publications (2)

Publication Number Publication Date
CN110865919A CN110865919A (en) 2020-03-06
CN110865919B true CN110865919B (en) 2023-11-28

Family

ID=69654579

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911067544.9A Active CN110865919B (en) 2019-11-04 2019-11-04 Monitoring method and device based on java process and computer equipment

Country Status (1)

Country Link
CN (1) CN110865919B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115051979B (en) * 2022-04-28 2023-09-19 福思(杭州)智能科技有限公司 Monitoring data debugging system, method, vehicle and computer readable storage medium
CN115617451A (en) * 2022-09-23 2023-01-17 长春吉大正元信息技术股份有限公司 Data processing method and data processing device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101799751A (en) * 2009-12-02 2010-08-11 山东浪潮齐鲁软件产业股份有限公司 Method for building monitoring agent software of host machine
CN102081548A (en) * 2009-11-30 2011-06-01 华为技术有限公司 Method and system for processing Java application and SAF (service availability forum) platform
CN102222037A (en) * 2010-04-15 2011-10-19 国际商业机器公司 Method and equipment for positioning bottleneck of JAVA program
CN106909498A (en) * 2015-12-22 2017-06-30 阿里巴巴集团控股有限公司 A kind of java applet injects the method and system of failure

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102081548A (en) * 2009-11-30 2011-06-01 华为技术有限公司 Method and system for processing Java application and SAF (service availability forum) platform
CN101799751A (en) * 2009-12-02 2010-08-11 山东浪潮齐鲁软件产业股份有限公司 Method for building monitoring agent software of host machine
CN102222037A (en) * 2010-04-15 2011-10-19 国际商业机器公司 Method and equipment for positioning bottleneck of JAVA program
CN106909498A (en) * 2015-12-22 2017-06-30 阿里巴巴集团控股有限公司 A kind of java applet injects the method and system of failure

Also Published As

Publication number Publication date
CN110865919A (en) 2020-03-06

Similar Documents

Publication Publication Date Title
JP4980792B2 (en) Virtual machine performance monitoring method and apparatus using the method
EP3507692B1 (en) Resource oversubscription based on utilization patterns in computing systems
JP5608222B2 (en) Application efficiency engine
CN107040578B (en) Data synchronization method, device and system
US8954971B2 (en) Data collecting method, data collecting apparatus and network management device
CN107451147B (en) Method and device for dynamically switching kafka clusters
CN112910945A (en) Request link tracking method and service request processing method
CN108920153B (en) Docker container dynamic scheduling method based on load prediction
CN110865919B (en) Monitoring method and device based on java process and computer equipment
US10055134B2 (en) Data compression method and storage system
CN112698952A (en) Unified management method and device for computing resources, computer equipment and storage medium
CN112152874A (en) System and method for acquiring and analyzing application monitoring data in distributed system
CN111221700B (en) Cluster node state monitoring method, device, equipment and readable storage medium
CN115883407A (en) Data acquisition method, system, equipment and storage medium
JP5597293B2 (en) Computer system and program
US20140096127A1 (en) Systems and methods for installing, managing, and provisioning applications
CN106330595B (en) Heartbeat detection method and device for distributed platform
CN115525392A (en) Container monitoring method and device, electronic equipment and storage medium
CN114860432A (en) Method and device for determining information of memory fault
CN109902067B (en) File processing method and device, storage medium and computer equipment
CN112559565A (en) Abnormity detection method, system and device
JP5390651B2 (en) Computer system and program
JPWO2012172641A1 (en) Computer, resource usage calculation method, and resource usage calculation program
CN108449343B (en) SSH protocol text data acquisition method, acquisition device and computer equipment
CN108845884B (en) Physical resource allocation method, device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant