CN111211938A - Biological information software monitoring system and method - Google Patents
Biological information software monitoring system and method Download PDFInfo
- Publication number
- CN111211938A CN111211938A CN201911398949.0A CN201911398949A CN111211938A CN 111211938 A CN111211938 A CN 111211938A CN 201911398949 A CN201911398949 A CN 201911398949A CN 111211938 A CN111211938 A CN 111211938A
- Authority
- CN
- China
- Prior art keywords
- information
- software
- biological information
- monitoring
- biological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
Abstract
The embodiment of the invention provides a biological information software monitoring system and a method, comprising the following steps: the system comprises a Docker cluster, a kafka message system, a monitoring module, biological information software and an analysis module; each piece of biological information software and the corresponding monitoring module are packaged in a corresponding Docker container; the monitoring module is used for collecting monitoring information of biological information software in the operating Docker container in real time when the Docker container where the monitoring module is located operates on the computing node, classifying the monitoring information and pushing the classified monitoring information to the kafka message system for storage; the analysis module is used for analyzing the monitoring information of each type in the kafka message system and obtaining an analysis result corresponding to the monitoring information of each type. The embodiment of the invention realizes real-time monitoring and analysis of the biological information software, and the biological information software is operated according to the deployment mode of the Docker cluster, so that the utilization rate and the usability of the cluster are improved, and the cost is saved.
Description
Technical Field
The invention belongs to the technical field of distributed systems, and particularly relates to a biological information software monitoring system and method.
Background
Under rapid development in the field of biology, a large amount of biological data is generally analyzed using various biological information software to acquire core data from the biological data.
In the prior art, biological information software is directly operated on a cluster, system resources are used in a preemptive mode, and free distribution of the system resources cannot be realized. In addition, the service condition of system resources and the running condition of biological information software need to be monitored manually at regular time, the information cannot be monitored in time, the monitoring time is long, and the efficiency is low.
In summary, the existing biological information software runs directly on the cluster, and on one hand, free allocation of system resources cannot be realized; on the other hand, manual monitoring of the operation environment of biological information and the like lacks real-time performance and is time-consuming and labor-consuming.
Disclosure of Invention
In order to overcome the problems that the existing biological information software is directly operated on a cluster, resource allocation is not free, and manual monitoring is time-consuming and labor-consuming, or at least partially solve the problems, embodiments of the present invention provide a biological information software monitoring system and method.
According to a first aspect of embodiments of the present invention, there is provided a biological information software monitoring system, including a Docker cluster, a kafka message system, a monitoring module, biological information software, and an analysis module;
the Docker cluster comprises a plurality of Docker containers and a computing node where each Docker container is located, and the biological information software corresponds to the monitoring modules one to one;
each piece of biological information software and the corresponding monitoring module are packaged in a corresponding Docker container;
the monitoring module is used for collecting the running environment information and the log information of the biological information software in the running Docker container in real time when the Docker container where the monitoring module is located runs on the computing node, classifying the running environment information and the log information and then pushing the classified running environment information and the log information to the kafka message system for storage;
the analysis module is used for analyzing the operating environment information and the log information of each type in the kafka message system and obtaining an analysis result corresponding to the operating environment information and the log information of each type.
Specifically, the analysis module is specifically configured to:
analyzing the operating environment information of each type in the kafka message system to acquire the CPU use condition, the IO use condition and the MEM use condition of the biological information software;
judging whether the CPU use condition, the IO use condition and the MEM use condition of the biological information software exceed preset resource limits or not;
and if so, informing the Docker container of a message that the biological information software exceeds the preset resource limit so as to be used for the Docker container to perform corresponding processing, returning an error code so as to allow a manager to obtain an error reason corresponding to the error code according to the error code, and adjusting the preset resource limit according to the error reason.
Specifically, the analysis module is specifically configured to:
analyzing the log information of each category in the kafka message system, updating the running state of the biological information software according to the result of the log information analysis, and marking the running step of the biological information software; wherein the log information includes step information and exception information.
Specifically, a mysql database is also included;
the mysql database is used for storing running environment information and log information of the biological information software running for multiple times;
correspondingly, the system further comprises an offline analysis module, wherein the offline analysis module is used for:
and performing off-line analysis on the running environment information and the log information in the mysql database.
Specifically, the offline analysis module is specifically configured to:
performing off-line analysis on the running environment information and the log information of the biological information software which runs for many times to obtain the file size and the used resource size of the biological information software which runs each time;
and performing function fitting according to the file size of each operation of the biological information software and the size of the used resource, so as to allocate the resource for the biological information software by using the fitted function.
According to a second aspect of the embodiments of the present invention, there is provided a biological information software monitoring method, including:
when a Docker container in a Docker cluster runs on a computing node where the Docker container is located, collecting running environment information and log information of biological information software packaged in the Docker container in real time based on a monitoring module packaged in the Docker container, classifying the running environment information and the log information, and pushing the classified running environment information and the log information to a kafka message system for storage;
analyzing the operating environment information and the log information of each type in the kafka message system based on an analysis module, and acquiring an analysis result corresponding to the operating environment information and the log information of each type.
Specifically, the step of analyzing the operating environment information of each category in the kafka message system based on the analysis module and acquiring the analysis result corresponding to the operating environment information of each category includes:
analyzing the operating environment information of each type in the kafka message system to acquire the CPU use condition, the IO use condition and the MEM use condition of the biological information software;
judging whether the CPU use condition, the IO use condition and the MEM use condition of the biological information software exceed preset resource limits or not;
and if so, informing the Docker container of a message that the biological information software exceeds the preset resource limit so as to be used for the Docker container to perform corresponding processing, returning an error code so as to allow a manager to obtain an error reason corresponding to the error code according to the error code, and adjusting the preset resource limit according to the error reason.
Specifically, the step of analyzing the log information of each category in the kafka message system based on the analysis module and acquiring the analysis result corresponding to the log information of each category includes:
analyzing the log information of each category in the kafka message system, updating the running state of the biological information software according to the result of the log information analysis, and marking the running step of the biological information software; wherein the log information includes step information and exception information.
Specifically, the method further comprises the following steps:
performing off-line analysis on the running environment information and the log information of the biological information software which runs for many times to obtain the file size and the used resource size of the biological information software which runs each time;
and performing function fitting according to the file size of each operation of the biological information software and the size of the used resource, so as to allocate the resource for the biological information software by using the fitted function.
According to a third aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor calls the program instructions to execute the method for monitoring the biological information software provided in any one of the various possible implementations of the second aspect.
The embodiment of the invention provides biological information software monitoring, which integrates biological information software and a monitoring module through a Docker container at the same time, uses the monitoring module to acquire running environment information and log information of the biological information software in running in real time, realizes real-time monitoring, and stores the monitoring information in high availability through a kafka message system, and finally uses an analysis module to analyze the acquired monitoring information, and all system modules are mutually depended and cooperated to achieve the real-time monitoring and analysis of the biological information software; meanwhile, the biological information software is operated according to the deployment mode of the Docker cluster, so that the utilization rate and the usability of the cluster are improved, and the cost is saved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic diagram of an overall architecture of a biological information software monitoring system according to an embodiment of the present invention;
fig. 2 is a block diagram of a biological information software monitoring system according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method for monitoring biological information software according to an embodiment of the present invention;
fig. 4 is a schematic view of an overall structure of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In an embodiment of the present invention, a biological information software monitoring system is provided, and fig. 1 is a schematic structural diagram of the biological information software monitoring system provided in the embodiment of the present invention, where the system includes: the system comprises a Docker cluster, a kafka message system, a monitoring module, biological information software and an analysis module; the system comprises a Docker cluster, a biological information software module and a monitoring module, wherein the Docker cluster comprises a plurality of Docker containers and computing nodes where each Docker container is located, and the biological information software corresponds to the monitoring module one to one; each piece of biological information software and the corresponding monitoring module are packaged in a corresponding Docker container;
as shown in fig. 2, the biological information software monitoring system in this embodiment is a cluster system composed of a plurality of single machines. Each individual includes a Docker container and a compute node. Each Docker container is packaged with a biological information software and a corresponding monitoring module. And the user submits the monitoring task to the computing node through the cluster upstream system. The number of single machines included in the Docker cluster may be dynamically expanded and contracted according to actual situations, and the number of single machines shown in fig. 2 is only an example.
The Docker cluster is mainly used for encapsulating biological information software and a monitoring module, and collecting running environment information of the biological information software in real time in a Docker container, such as IO (Input/Output), CPU (central processing Unit), memory and the like, and log information of the biological information software, such as step information, abnormal information and the like, so as to achieve the purpose of running simultaneously without influencing each other, and improve the utilization rate of Docker cluster resources and the availability of the Docker cluster through the characteristics of the Docker container.
The monitoring module is used for collecting the running environment information and the log information of the biological information software in the running Docker container in real time when the Docker container where the monitoring module is located runs on the computing node, classifying the running environment information and the log information and then pushing the classified running environment information and the log information to the kafka message system for storage;
when any Docker container runs on a computing node, the monitoring module promoter process in the Docker container collects running environment information and log information of biological information software in the Docker container in real time, classifies the collected information according to different topics, and then pushes the classified information to a kafka message system for storage. The kafka messaging system is mainly used to guarantee high availability of information and to decouple dependencies between parts.
The analysis module is used for analyzing the operating environment information and the log information of each type in the kafka message system and acquiring an analysis result corresponding to the operating environment information and the log information of each type.
The analysis module acquires the running environment information and the log information of the corresponding category of the biological information software from the kafka message system through the categories of different topic classifications, analyzes the acquired information and acquires an analysis result.
In the embodiment, the Docker container is used for simultaneously integrating the biological information software and the monitoring module, the monitoring module is used for acquiring the running environment information and the log information of the biological information software during running in real time to realize real-time monitoring, the kafka message system is used for storing the monitoring information in high availability, and finally the analysis module is used for analyzing the acquired monitoring information, and all system modules are mutually depended and cooperated to achieve the real-time monitoring and analysis of the biological information software; meanwhile, the biological information software is operated according to the deployment mode of the Docker cluster, so that the utilization rate and the usability of the cluster are improved, and the cost is saved.
On the basis of the foregoing embodiment, the analysis module in this embodiment is specifically configured to: analyzing the operating environment information of each type in the kafka message system to acquire the CPU use condition, the IO use condition and the MEM use condition of the biological information software; judging whether the CPU use condition, the IO use condition and the MEM use condition of the biological information software exceed the preset resource limit or not; and if so, informing the Docker container of a message that the biological information software exceeds the preset resource limit so as to be used for the Docker container to perform corresponding processing, returning an error code so as to allow a manager to obtain an error reason corresponding to the error code according to the error code, and adjusting the preset resource limit according to the error reason.
Specifically, the analysis module analyzes various kinds of operating environment information, and determines the current CPU usage, IO usage, and MEM usage of the bioinformation software. And judging whether the current CPU (Central processing Unit) use condition, IO (input output) use condition and MEM (minimum memory access) use condition of the biological information software exceed the resource limit according to the preset resource limit, if so, giving an early warning, and informing related responsible persons through a mail system or other modes, so that the early warning and the response are timely carried out according to the monitoring information, the environment during the operation of the biological information software which is early warned in real time is timely processed, and the operation efficiency of the biological information software is improved.
On the basis of the foregoing embodiment, the analysis module in this embodiment is specifically configured to: analyzing the log information of each category in the kafka message system, updating the running state of the biological information software according to the result of the log information analysis, and marking the running step of the biological information software; wherein the log information includes step information and exception information.
Specifically, the analyst knows the running condition of the biological information software in real time according to the analysis result of the log information and grasps the running progress of the biological information software.
On the basis of the above embodiments, the present embodiment further includes a mysql database; the mysql database is used for storing running environment information and log information of multiple running of the biological information software; correspondingly, the system further comprises an offline analysis module, wherein the offline analysis module is used for: and performing offline analysis on the running environment information and the log information in the mysql database.
Specifically, the monitoring information is stored in the mysql database, and the mysql database is mainly used for persisting the monitoring information for subsequent offline analysis. Since the file size of the bio-information software is generally changed every time it runs, the environment information and the log information are also changed every time it runs. And performing offline analysis on the monitoring information of each running of the biological information software by using an offline analysis module.
On the basis of the foregoing embodiment, the offline analysis module in this embodiment is specifically configured to: performing off-line analysis on the running environment information and the log information of the biological information software which runs for many times to obtain the file size and the used resource size of the biological information software which runs each time; and performing function fitting according to the file size of each operation of the biological information software and the size of the used resource, so as to use the fitted function to distribute the resource for the biological information software.
Specifically, in the present embodiment, through offline analysis of a large amount of running environment information and log information, the function to be fitted is an association relationship between the file size of the biological information software and the resource size required by the running of the biological information software. The incidence relation provides a reference basis for the optimization of the subsequent bioinformation software resource allocation, and the resource utilization rate of the Docker cluster is improved.
In another embodiment of the present invention, a biological information software monitoring method is provided, which is implemented based on the systems in the foregoing embodiments. Therefore, the descriptions and definitions in the embodiments of the aforementioned biological information software monitoring system can be used for understanding the respective execution steps in the embodiments of the present invention. Fig. 3 is a schematic flow chart of a biological information software monitoring method according to an embodiment of the present invention, where the method includes: s301, when a Docker container in a Docker cluster runs on a computing node where the Docker container is located, collecting running environment information and log information of biological information software packaged in the Docker container in real time based on a monitoring module packaged in the Docker container, classifying the running environment information and the log information, and pushing the classified running environment information and the log information to a kafka message system for storage;
the system comprises a Docker cluster, a biological information software module and a monitoring module, wherein the Docker cluster comprises a plurality of Docker containers and computing nodes where each Docker container is located, and the biological information software corresponds to the monitoring module one to one; each piece of biological information software and the corresponding monitoring module are packaged in a corresponding Docker container. The Docker cluster is mainly used for packaging biological information software and a monitoring module, collecting running environment information of the biological information software such as IO, CPU and memory and log information of the biological information software such as step information and abnormal information in a Docker container in real time, achieving the purpose of running simultaneously without influencing each other, and improving the utilization rate of Docker cluster resources and the availability of the Docker cluster through the characteristics of the Docker container.
When any Docker container runs on a computing node, the monitoring module promoter process in the Docker container collects running environment information and log information of biological information software in the Docker container in real time, classifies the collected information according to different topics, and then pushes the classified information to a kafka message system for storage. The kafka messaging system is mainly used to guarantee high availability of information and to decouple dependencies between parts.
S302, analyzing the operating environment information and the log information of each type in the kafka message system based on an analysis module, and acquiring an analysis result corresponding to the operating environment information and the log information of each type.
And acquiring the running environment information and the log information of the corresponding category of the biological information software from the kafka message system through an analysis module according to the categories of different topic classifications, and analyzing the acquired information to acquire an analysis result.
In the embodiment, the Docker container is used for simultaneously integrating the biological information software and the monitoring module, the monitoring module is used for acquiring the running environment information and the log information of the biological information software during running in real time to realize real-time monitoring, the kafka message system is used for storing the monitoring information in high availability, and finally the analysis module is used for analyzing the acquired monitoring information, and all system modules are mutually depended and cooperated to achieve the real-time monitoring and analysis of the biological information software; meanwhile, the biological information software is operated according to the deployment mode of the Docker cluster, so that the utilization rate and the usability of the cluster are improved, and the cost is saved.
On the basis of the foregoing embodiment, in this embodiment, the step of analyzing, by an analysis module, the operating environment information of each type in the kafka message system, and acquiring an analysis result corresponding to the operating environment information of each type includes: analyzing the operating environment information of each type in the kafka message system to acquire the CPU use condition, the IO use condition and the MEM use condition of the biological information software; judging whether the CPU use condition, the IO use condition and the MEM use condition of the biological information software exceed preset resource limits or not; and if so, informing the Docker container of a message that the biological information software exceeds the preset resource limit so as to be used for the Docker container to perform corresponding processing, returning an error code so as to allow a manager to obtain an error reason corresponding to the error code according to the error code, and adjusting the preset resource limit according to the error reason.
Specifically, various kinds of running environment information are analyzed through the analysis module, and the current CPU use condition, IO use condition and MEM use condition of the biological information software are determined. And judging whether the current CPU (Central processing Unit) use condition, IO (input output) use condition and MEM (minimum memory access) use condition of the biological information software exceed the resource limit according to the preset resource limit, if so, giving an early warning, and informing related responsible persons through a mail system or other modes, so that the early warning and the response are timely carried out according to the monitoring information, the environment during the operation of the biological information software which is early warned in real time is timely processed, and the operation efficiency of the biological information software is improved.
On the basis of the foregoing embodiment, in this embodiment, the step of analyzing the log information of each category in the kafka message system based on the analysis module, and acquiring an analysis result corresponding to the log information of each category includes: analyzing the log information of each category in the kafka message system, updating the running state of the biological information software according to the result of the log information analysis, and marking the running step of the biological information software; wherein the log information includes step information and exception information.
Specifically, the analyst knows the running condition of the biological information software in real time according to the analysis result of the log information and grasps the running progress of the biological information software.
On the basis of the above embodiments, the present embodiment further includes: performing off-line analysis on the running environment information and the log information of the biological information software which runs for many times to obtain the file size and the used resource size of the biological information software which runs each time; and performing function fitting according to the file size of each operation of the biological information software and the size of the used resource, so as to allocate the resource for the biological information software by using the fitted function.
Specifically, the monitoring information is stored in the mysql database, and the mysql database is mainly used for persisting the monitoring information for subsequent offline analysis. Since the file size of the bio-information software is generally changed every time it runs, the environment information and the log information are also changed every time it runs. And performing offline analysis on the monitoring information of each running of the biological information software by using an offline analysis module.
Through off-line analysis of a large amount of running environment information and log information, the fitted function is the incidence relation between the file size of the biological information software and the size of resources required by the running of the biological information software. The incidence relation provides a reference basis for the optimization of the subsequent bioinformation software resource allocation, and the resource utilization rate of the Docker cluster is improved.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)401, a communication Interface (communication Interface)402, a memory (memory)403 and a communication bus 404, wherein the processor 401, the communication Interface 402 and the memory 403 complete communication with each other through the communication bus 404. Processor 401 may call logic instructions in memory 403 to perform the following method: when a Docker container in a Docker cluster runs on a computing node where the Docker container is located, collecting running environment information and log information of biological information software packaged in the Docker container in real time based on a monitoring module packaged in the Docker container, classifying the running environment information and the log information, and pushing the classified running environment information and the log information to a kafka message system for storage; analyzing the operating environment information and the log information of each type in the kafka message system based on an analysis module, and acquiring an analysis result corresponding to the operating environment information and the log information of each type.
In addition, the logic instructions in the memory 403 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A biological information software monitoring system is characterized by comprising a Docker cluster, a kafka message system, a monitoring module, biological information software and an analysis module;
the Docker cluster comprises a plurality of Docker containers and a computing node where each Docker container is located, and the biological information software corresponds to the monitoring modules one to one;
each piece of biological information software and the corresponding monitoring module are packaged in a corresponding Docker container;
the monitoring module is used for collecting the running environment information and the log information of the biological information software in the running Docker container in real time when the Docker container where the monitoring module is located runs on the computing node, classifying the running environment information and the log information and then pushing the classified running environment information and the log information to the kafka message system for storage;
the analysis module is used for analyzing the operating environment information and the log information of each type in the kafka message system and obtaining an analysis result corresponding to the operating environment information and the log information of each type.
2. The bioinformatic software monitoring system of claim 1, wherein the analysis module is specifically configured to:
analyzing the operating environment information of each type in the kafka message system to acquire the CPU use condition, the IO use condition and the MEM use condition of the biological information software;
judging whether the CPU use condition, the IO use condition and the MEM use condition of the biological information software exceed preset resource limits or not;
and if so, informing the Docker container of a message that the biological information software exceeds the preset resource limit so as to be used for the Docker container to perform corresponding processing, returning an error code so as to allow a manager to obtain an error reason corresponding to the error code according to the error code, and adjusting the preset resource limit according to the error reason.
3. The bioinformatic software monitoring system of claim 1, wherein the analysis module is specifically configured to:
analyzing the log information of each category in the kafka message system, updating the running state of the biological information software according to the result of the log information analysis, and marking the running step of the biological information software; wherein the log information includes step information and exception information.
4. The bioinformatic software monitoring system according to any one of claims 1-3, further comprising a mysql database;
the mysql database is used for storing running environment information and log information of the biological information software running for multiple times;
correspondingly, the system further comprises an offline analysis module, wherein the offline analysis module is used for:
and performing off-line analysis on the running environment information and the log information in the mysql database.
5. The biological information software monitoring system according to claim 4, wherein the offline analysis module is specifically configured to:
performing off-line analysis on the running environment information and the log information of the biological information software which runs for many times to obtain the file size and the used resource size of the biological information software which runs each time;
and performing function fitting according to the file size of each operation of the biological information software and the size of the used resource, so as to allocate the resource for the biological information software by using the fitted function.
6. A biological information software monitoring method is characterized by comprising the following steps:
when a Docker container in a Docker cluster runs on a computing node where the Docker container is located, collecting running environment information and log information of biological information software packaged in the Docker container in real time based on a monitoring module packaged in the Docker container, classifying the running environment information and the log information, and pushing the classified running environment information and the log information to a kafka message system for storage;
analyzing the operating environment information and the log information of each type in the kafka message system based on an analysis module, and acquiring an analysis result corresponding to the operating environment information and the log information of each type.
7. The method for monitoring the biological information software as claimed in claim 6, wherein the step of obtaining the analysis result corresponding to each type of the operation environment information based on the analysis module analyzing each type of the operation environment information in the kafka message system comprises:
analyzing the operating environment information of each type in the kafka message system to acquire the CPU use condition, the IO use condition and the MEM use condition of the biological information software;
judging whether the CPU use condition, the IO use condition and the MEM use condition of the biological information software exceed preset resource limits or not;
and if so, informing the Docker container of a message that the biological information software exceeds the preset resource limit so as to be used for the Docker container to perform corresponding processing, returning an error code so as to allow a manager to obtain an error reason corresponding to the error code according to the error code, and adjusting the preset resource limit according to the error reason.
8. The method for monitoring bioinformation software according to claim 6, wherein the step of obtaining the analysis result corresponding to each category of log information based on the analysis module analyzing each category of log information in the kafka message system comprises:
analyzing the log information of each category in the kafka message system, updating the running state of the biological information software according to the result of the log information analysis, and marking the running step of the biological information software; wherein the log information includes step information and exception information.
9. The bioinformatic software monitoring method according to any one of claims 6 to 8, further comprising:
performing off-line analysis on the running environment information and the log information of the biological information software which runs for many times to obtain the file size and the used resource size of the biological information software which runs each time;
and performing function fitting according to the file size of each operation of the biological information software and the size of the used resource, so as to allocate the resource for the biological information software by using the fitted function.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method for software monitoring of biological information according to any one of claims 6 to 9 are implemented when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911398949.0A CN111211938B (en) | 2019-12-30 | 2019-12-30 | Biological information software monitoring system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911398949.0A CN111211938B (en) | 2019-12-30 | 2019-12-30 | Biological information software monitoring system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111211938A true CN111211938A (en) | 2020-05-29 |
| CN111211938B CN111211938B (en) | 2021-10-15 |
Family
ID=70788290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911398949.0A Active CN111211938B (en) | 2019-12-30 | 2019-12-30 | Biological information software monitoring system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111211938B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116450301A (en) * | 2023-06-14 | 2023-07-18 | 天津市天河计算机技术有限公司 | Container-based monitoring method, system, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150256481A1 (en) * | 2014-03-06 | 2015-09-10 | Jisto Inc. | Elastic Compute Cloud Based On Underutilized Server Resources Using A Distributed Container System |
| CN105224445A (en) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN107370816A (en) * | 2017-07-26 | 2017-11-21 | 郑州云海信息技术有限公司 | A kind of dispositions method and device of Web applications |
| CN109274556A (en) * | 2018-11-09 | 2019-01-25 | 四川长虹电器股份有限公司 | A kind of collection and analysis system of web log |
| US20190102409A1 (en) * | 2017-09-29 | 2019-04-04 | Oracle International Corporation | System and method for managing a blockchain cloud service |
| CN109743199A (en) * | 2018-12-25 | 2019-05-10 | 中国联合网络通信集团有限公司 | Containerization management system based on micro services |
-
2019
- 2019-12-30 CN CN201911398949.0A patent/CN111211938B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150256481A1 (en) * | 2014-03-06 | 2015-09-10 | Jisto Inc. | Elastic Compute Cloud Based On Underutilized Server Resources Using A Distributed Container System |
| CN105224445A (en) * | 2015-10-28 | 2016-01-06 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN107370816A (en) * | 2017-07-26 | 2017-11-21 | 郑州云海信息技术有限公司 | A kind of dispositions method and device of Web applications |
| US20190102409A1 (en) * | 2017-09-29 | 2019-04-04 | Oracle International Corporation | System and method for managing a blockchain cloud service |
| CN109274556A (en) * | 2018-11-09 | 2019-01-25 | 四川长虹电器股份有限公司 | A kind of collection and analysis system of web log |
| CN109743199A (en) * | 2018-12-25 | 2019-05-10 | 中国联合网络通信集团有限公司 | Containerization management system based on micro services |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116450301A (en) * | 2023-06-14 | 2023-07-18 | 天津市天河计算机技术有限公司 | Container-based monitoring method, system, equipment and medium |
| CN116450301B (en) * | 2023-06-14 | 2023-08-15 | 天津市天河计算机技术有限公司 | Container-based monitoring method, system, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111211938B (en) | 2021-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020259421A1 (en) | Method and apparatus for monitoring service system | |
| EP2524322B1 (en) | A virtualization and consolidation analysis engine for enterprise data centers | |
| EP2503733B1 (en) | Data collecting method, data collecting apparatus and network management device | |
| CN106713396B (en) | Server scheduling method and system | |
| US20210042578A1 (en) | Feature engineering orchestration method and apparatus | |
| CN110659307A (en) | Event stream correlation analysis method and system | |
| CN110955586A (en) | System fault prediction method, device and equipment based on log | |
| AU2021218159B2 (en) | Utilizing machine learning models to determine customer care actions for telecommunications network providers | |
| CN115373835A (en) | Task resource adjusting method and device for Flink cluster and electronic equipment | |
| CN112702184A (en) | Fault early warning method and device and computer-readable storage medium | |
| CN109542737A (en) | Platform alert processing method, device, electronic device and storage medium | |
| CN110557291A (en) | Network service monitoring system | |
| CN111211938B (en) | Biological information software monitoring system and method | |
| CN105471938B (en) | Server load management method and device | |
| CN112788010A (en) | Script processing method, device, medium and equipment based on threat event | |
| CN112000657A (en) | Data management method, device, server and storage medium | |
| CN112631577B (en) | Model scheduling method, model scheduler and model safety test platform | |
| CN114706893A (en) | Fault detection method, device, equipment and storage medium | |
| CN110493071B (en) | Message system resource balancing device, method and equipment | |
| CN113656239A (en) | Monitoring method and device for middleware and computer program product | |
| CN111581062A (en) | Service fault processing method and server | |
| CN111209102A (en) | Distributed task distribution method and system | |
| CN112948075B (en) | Task congestion processing method and device and electronic equipment | |
| JP2021018630A (en) | Alarm aggregation sorting device and alarm aggregation sorting method | |
| CN116089248B (en) | Write I/O burst distribution prediction method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |