CN111382022A - Method and device for monitoring real-time streaming computing platform, electronic equipment and storage medium - Google Patents
Method and device for monitoring real-time streaming computing platform, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111382022A CN111382022A CN201811614036.3A CN201811614036A CN111382022A CN 111382022 A CN111382022 A CN 111382022A CN 201811614036 A CN201811614036 A CN 201811614036A CN 111382022 A CN111382022 A CN 111382022A
- Authority
- CN
- China
- Prior art keywords
- storm
- running state
- state information
- page
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000004590 computer program Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 229940126655 NDI-034858 Drugs 0.000 description 6
- 241000290929 Nimbus Species 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method and a device for monitoring a real-time streaming computing platform, electronic equipment and a storage medium, wherein the method comprises the following steps: the method comprises the steps of obtaining running state information of Storm by accessing a UI page of the Storm of a real-time stream computing platform, obtaining the running state information of the Storm by calling an RPC server on the Storm if the running state information is failed, and obtaining the running state information of the Storm by analyzing information written back by the Storm and recorded in Zookeeper if the running state information is failed. According to the embodiment of the invention, the information acquisition is sequentially tried to be obtained for monitoring through the three channels, and only one channel can acquire the states of the Storm cluster and the Topology, so that the query performance and accuracy of the Storm cluster state monitoring are improved, and the problem of inaccurate monitoring is avoided.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a method and a device for monitoring a real-time streaming computing platform, electronic equipment and a storage medium.
Background
In an IT monitoring system, a large amount of open-source middleware is used, such as Storm, which is an open-source distributed, high-fault-tolerant real-time big data processing platform (i.e. real-time stream computing platform), and as more and more scenes cannot tolerate high delay, such as website statistics, recommendation systems, financial systems (high-frequency trading, stocks) and the like, the application of big data real-time processing solutions (i.e. stream computing) is becoming widespread, Storm is a mainstream in stream computing technology, and the core components of Storm include: nimbus, Supervisor, Topology. Nimbus is the Master of Storm, and is responsible for resource allocation and task scheduling. A Storm cluster has only one Nimbus. Supervisor is Slave of Storm and is responsible for receiving tasks assigned by Nimbus and managing all Worker (work processes). Topology: service topology, Storm's topology is the encapsulation of real-time computing application logic, and topology is also understood to be a topology consisting of a series of Spout (data source) and Bolt interrelated by data flow.
Because the IT monitoring system relies on Storm to process the event, the operation state of Storm needs to be monitored and managed in a centralized manner, the requirement on the accuracy of the Storm state is very high, if one node in the Storm cluster has a problem, the node needs to be reported in time, otherwise, the operation of the whole IT monitoring system is influenced. However, in the prior art, the Storm cluster state monitoring accuracy is not high, and the condition that the Storm running state information cannot be obtained often occurs, so that a solution is needed urgently.
Disclosure of Invention
The invention provides a method and a device for monitoring a real-time stream computing platform, electronic equipment and a storage medium, wherein three different acquisition modules are used for carrying out sequential verification, and only one module is required to acquire the state of a Storm cluster, so that the Storm monitoring reliability is ensured, the Storm cluster state monitoring accuracy is improved, and the normal operation of an IT monitoring system is further ensured.
According to one aspect of the present application, there is provided a method of monitoring a real-time streaming computing platform, comprising:
acquiring running state information of Storm by accessing a UI page of the real-time streaming computing platform Storm,
if the acquisition of the running state information fails, acquiring the running state information of Storm by calling a second acquisition module of an RPC server on Storm,
and if the running state information fails to be acquired, acquiring the running state information of the Storm through a third acquisition module for analyzing the information written back by the Storm recorded in the Zookeeper, thereby realizing monitoring.
Optionally, the method further comprises: and if the acquisition of the running state information of Storm by analyzing the information written back by Storm recorded in the Zookeeper fails, returning a failure prompt that the running state information of Storm cannot be acquired.
Optionally, the obtaining the running state information of Storm by accessing the UI page of Storm includes:
and acquiring UI page address information of Storm configured in the Zookeeper, and accessing the UI page according to the acquired UI page address information to acquire data displayed on the UI page of Storm to obtain running state information.
Optionally, the accessing the UI page according to the obtained UI page address information to obtain the data displayed on the Storm UI page includes:
accessing data presented on the RESTful interface of the UI page in the Storm UI page according to the acquired address information containing the IP address and the port number of the UI page,
and if the RESTful interface is blocked or the access is unsuccessful, returning a prompt that the Storm UI cannot be accessed.
Optionally, the obtaining the running state information of Storm by calling an RPC server on Storm includes:
starting a Thrift RPC client, connecting and calling an RPC server on Storm, and acquiring the running state information of the Storm.
In accordance with another aspect of the present application, there is provided an apparatus for monitoring a live stream computing platform, comprising:
a first obtaining module for obtaining the running state information of Storm by accessing the UI page of the real-time stream computing platform Storm,
a second obtaining module, configured to obtain running state information of Storm by calling an RPC server on Storm,
and the third acquisition module is used for acquiring the running state information of the Storm by analyzing the information written back by the Storm recorded in the Zookeeper so as to realize monitoring.
Optionally, the apparatus comprises: and the failure prompt module is used for returning a failure prompt that the Storm running state information cannot be acquired if the third acquisition module fails to acquire the Storm running state information.
Optionally, the first obtaining module is specifically configured to obtain UI page address information of Storm configured in Zookeeper, and access the UI page according to the obtained UI page address information to obtain data displayed on the UI page of Storm to obtain running state information;
the second acquisition module is specifically used for starting a Thrift RPC client, connecting and calling an RPC server on Storm, and acquiring the running state information of Storm.
According to another aspect of the present application, there is provided an electronic apparatus including: the system comprises a memory and a processor, wherein the memory and the processor are connected through an internal bus in a communication mode, the memory stores program instructions capable of being executed by the processor, and the program instructions are capable of realizing the method in one aspect of the application when being executed by the processor.
According to yet another aspect of the application, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements a method according to one aspect of the application.
According to the method and the device for monitoring the Storm of the real-time flow computing platform, the running state information of the Storm is obtained by accessing the UI page of the Storm, if the obtaining fails, the running state information of the Storm is obtained by calling the RPC server on the Storm, if the obtaining fails, the running state information of the Storm is obtained by analyzing the information written back by the Storm recorded in the Zookeeper, and the monitoring is realized.
Drawings
FIG. 1 is a flow diagram of a method of monitoring a real-time streaming computing platform according to one embodiment of the invention;
FIG. 2 is a flow chart illustrating a method for monitoring a real-time streaming computing platform according to another embodiment of the invention
FIG. 3 is a block diagram of an apparatus for monitoring a live stream computing platform according to one embodiment of the invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The IT monitoring system processes the event by relying on the Storm middleware, so the requirement on the accuracy of the state of the Storm middleware is very high, if one Storm node in the cluster has a problem, the Storm node needs to be reported in time, otherwise, the operation of the whole IT monitoring system is influenced.
In this regard, in the embodiment of the present invention, sequential verification is performed by three different acquisition modules, so that it is ensured that monitoring can be achieved by acquiring state information of the Storm cluster, and a process can be ended as long as one module can acquire the information, that is, in practical application, monitoring of the Storm cluster can be achieved by any one of the three acquisition modules of this embodiment.
However, a single monitoring means has a problem that it is difficult to obtain Storm state information, for example, when monitoring the Storm cluster state using a thrft of an RPC (Remote Procedure Call) framework, it is necessary to compile a source code of the Storm to obtain a Java source code related to the thrft Client using the thrft, if the Storm versions are not consistent, the process is cumbersome, time-consuming and labor-consuming, and problems encountered in the monitoring process are many. Here, the thread is a cross-Language service deployment framework, and defines the Interface and data type of the RPC through IDL (Interface Definition Language), and then generates codes of different languages through a thread compiler to conveniently generate cross-programming Language for communication between the RPC client and the server. The thread supports various transmission protocols, and a user can select a proper type according to actual requirements.
As another example, to reduce the use of Thrift to monitor Storm state, Storm cluster data is obtained via the REST interface provided by the Storm UI. The method has the disadvantages that an interface is provided by relying on the Storm UI, if the load of the Nimbus node of the Storm is too heavy and the node is down, the Storm UI of the Storm cluster which normally runs is failed to be opened or is always failed to be refreshed, and therefore the running state information of the Storm cluster cannot be accurately acquired.
Preferably, to avoid reporting an exception due to failure to complete data transmission within the timeout period during monitoring through the thread, and to avoid a failure of opening through the Storm UI or a failure of always refreshing, which leads to a problem of inaccurate monitoring. In the embodiment of the invention, firstly, a RESTful interface is provided through a Storm UI, and states of a Storm cluster and Topology and the like are obtained; if the StormUI providing interface is blocked and the access is not successful; monitoring the Storm cluster state through an RPC frame Thrift; if the thread is abnormal due to timeout; and finally, writing back the state information to the Zookeeper through Storm, and analyzing to obtain the state of the Storm cluster. The sequence verification is carried out through the three modes, and the process can be ended as long as one mode can obtain the state of the Storm cluster.
Fig. 1 is a flowchart of a method for monitoring a real-time streaming computing platform according to an embodiment of the present invention, and referring to fig. 1, the method for monitoring a real-time streaming computing platform includes the following steps:
step S101, obtaining the running state information of Storm by accessing the UI page of the real-time stream computing platform Storm,
step S102, if the running state information fails to be obtained, the running state information of Storm is obtained by calling RPC server on Storm,
step S103, if the running state information fails to be acquired, the running state information of Storm is acquired by analyzing the information written back by the Storm recorded in the Zookeeper.
In this implementation, if the third acquisition channel also fails to acquire information, that is, if acquisition of the running state information of Storm by analyzing the information written back by Storm recorded in Zookeeper fails, a failure prompt that the running state information of Storm cannot be acquired is returned.
As shown in fig. 1, in the method for monitoring a Storm of a real-time flow computing platform according to the embodiment, sequential verification is performed through three different information acquisition modules, and only one channel is required to acquire the state of the Storm cluster, so that query performance is improved, reliability of Storm monitoring and accuracy of state monitoring are ensured, and normal operation of an IT monitoring system is facilitated.
The following describes implementation steps of the method for monitoring the Storm of the real-time streaming computing platform according to an embodiment of the present invention with reference to fig. 2, where the flow starts, step S201 is executed, and Storm configuration information is loaded.
It should be noted that, when initializing data, before starting the system, the Storm cluster address and Storm UI address information are stored in the Zookeeper. ZooKeeper is a distributed application program coordination service, is a manager of a cluster, and monitors the state of each node in the cluster to perform next reasonable operation according to feedback submitted by the node. Finally, the system has simple and easy-to-use interfaces and efficient and stable performance and functions and is provided for users. Programs always need to be configured, and if the programs are deployed in a scattered manner on a plurality of machines, it becomes difficult to change the configuration one by one. All the configurations are put on the Zookeeper and stored in a certain directory node of the Zookeeper, then all relevant application programs monitor the directory node, once the configuration information changes, each application program receives the notification of the Zookeeper, and then new configuration information is obtained from the Zookeeper and applied to the system.
In this step, the configured Storm UI address information is obtained from the Zookeeper. Here, the Storm UI address information includes IP address information and port number information of a UI (user interface) page.
Step S202, obtaining running state information through a RESTful interface provided by Storm UI;
in this embodiment, the UI page address information of Storm configured in Zookeeper is acquired, and the UI page is accessed according to the acquired UI page address information to acquire data displayed on the UI page of Storm, so as to obtain the running state information.
Specifically, data displayed on the RESTful interface of the UI page and the Storm UI page is accessed according to the acquired address information containing the IP address and the port number of the UI page, if the RESTful interface is blocked or the access is unsuccessful, a prompt that the Storm UI cannot be accessed is returned, namely if the Storm UI is abnormal, step S203 is executed, and a Storm UI abnormal event is generated to prompt that the acquisition mode is unsuccessful. If the Storm UI provides that the interface access is successful, the relevant state is directly returned. It should be noted that Storm provides UI interface to display operation state data, such as the number of times an exception occurs in a certain topology, and the stack of the occurred exceptions.
Step S204, if the information is correctly acquired, the flow is ended, otherwise step S205 is executed,
in this step, it is determined whether step S203 correctly obtains the running state information of Storm, where the running state information of Storm includes state information (e.g., normal, abnormal) of the master node Nimbus, the slave node Supervisor, and the service Topology of Storm. If so, the process ends, otherwise, the second mode is used for trying continuously.
Step S205, obtaining operation state information through Trift RPC;
specifically, the obtaining of the running state information through the Trift RPC includes starting a Thrift RPC client, connecting and calling an RPC server on Storm, and obtaining the running state information of Storm. The method is to access the bottom interface of Storm through function call and request service to acquire the running state information of Storm.
Step S206, if the information is correctly obtained, if yes, the process is ended, otherwise step S207 is executed,
in this step, it is determined whether step S205 correctly obtains the running state information of Storm, if yes, the monitoring process is ended, otherwise, the third method is continuously used to try.
Step S207, analyzing the information written back by Storm recorded in the Zookeeper, and acquiring the running state information.
If the Storm cluster state monitoring through the RPC framework Thrift fails, the Storm cluster state is analyzed and obtained through the state information written back to the Zookeeper by the Storm, and the state information is returned.
Specifically, the recorded information written back by Storm is obtained from the Zookeeper, and the running state information of Storm is obtained after analysis.
It should be noted that Zookeeper is a manager of the Storm cluster, Storm writes back information of a key index, for example, heartbeat information, to the Zookeeper at regular time, and the running state of the Storm cluster can be determined by acquiring the heartbeat information.
In step S208, if the information is correctly acquired, if so, the process is ended, otherwise, step S209 is executed to generate an event that the Storm running state cannot be acquired.
In step S208, it is determined whether the running state information of the Storm cluster is correctly acquired, and if the running state information of the Storm cluster is acquired, the monitoring information acquisition process is ended. If the monitoring information is not acquired, an event that the Storm running state cannot be acquired is generated, and prompt information is generated to prompt that the acquisition of the monitoring information fails.
Therefore, in the embodiment of the invention, the running state information of the Storm cluster is obtained by three different modules and trying in sequence. The first acquisition module for accessing the UI page of Storm is used as a first priority attempt mode because of the consideration of query performance, complexity and other factors, and the second acquisition module is used for acquiring under the condition that the first acquisition module cannot acquire the operating state information of the Storm cluster, and the like until the operating state information of the Storm cluster is acquired or acquisition fails. In addition, the reason why the third obtaining module is arranged at the end in the present embodiment is that, in general, the Storm running state information obtained by the third obtaining module is limited, and may only include very critical running state information, such as whether an abnormality occurs, and no record is made for details of the abnormality, so that the detailed query and monitoring requirements may not be met.
The method for monitoring a real-time streaming computing platform is the same as the method for monitoring a real-time streaming computing platform, and an embodiment of the present invention further provides an apparatus for monitoring a real-time streaming computing platform, and referring to fig. 3, the apparatus 300 for monitoring a real-time streaming computing platform includes:
a first obtaining module 301, configured to obtain running state information of Storm by accessing a UI page of the real-time streaming computing platform Storm,
a second obtaining module 302, configured to obtain the running state information of Storm by calling an RPC server on Storm,
a third obtaining module 303, configured to obtain the running state information of Storm by analyzing the information written back by Storm recorded in Zookeeper.
In one embodiment of the present invention, the apparatus 300 for monitoring a real-time streaming computing platform further comprises: and the failure prompt module is used for returning a failure prompt that the Storm running state information cannot be acquired if the third acquisition module fails to acquire the Storm running state information.
In an embodiment of the present invention, the first obtaining module 301 is specifically configured to obtain UI page address information of Storm configured in Zookeeper, and access a UI page according to the obtained UI page address information to obtain data displayed on the UI page of Storm to obtain running state information; the second obtaining module 302 is specifically configured to start a swift RPC client, connect and call an RPC server on Storm, and obtain running state information of Storm.
It should be noted that, for example, explanations of functions performed by modules in the apparatus for monitoring the real-time flow computing platform Storm shown in fig. 3 are consistent with the example explanations in the foregoing method embodiments, and are not repeated here.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 4, the electronic device includes a memory 401 and a processor 402, the memory 401 and the processor 402 are communicatively connected through an internal bus 403, the memory 401 stores program instructions that can be executed by the processor 402, and the program instructions, when executed by the processor 402, can implement the method for monitoring the real-time streaming computing platform Storm described above.
In addition, the logic instructions in the memory 401 may be implemented in the form of software functional units and may be stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part thereof, which essentially contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Another embodiment of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to perform the method of monitoring a real-time streaming computing platform Storm described above.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is to be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the description of the present invention, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description. Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
While the foregoing is directed to embodiments of the present invention, other modifications and variations of the present invention may be devised by those skilled in the art in light of the above teachings. It should be understood by those skilled in the art that the foregoing detailed description is for the purpose of illustrating the invention rather than the foregoing detailed description, and that the scope of the invention is defined by the claims.
Claims (10)
1. A method of monitoring a real-time streaming computing platform, comprising:
acquiring running state information of Storm by accessing a UI page of the real-time streaming computing platform Storm,
if the acquisition of the running state information fails, acquiring the running state information of Storm by calling an RPC server on the Storm,
and if the acquisition of the running state information fails, acquiring the running state information of the Storm by analyzing the information written back by the Storm recorded in the Zookeeper, and realizing monitoring.
2. The method of claim 1, further comprising: and if the acquisition of the running state information of Storm by analyzing the information written back by Storm recorded in the Zookeeper fails, returning a failure prompt that the running state information of Storm cannot be acquired.
3. The method of claim 1, wherein the obtaining the running state information of Storm by accessing the UI page of Storm comprises:
and acquiring UI page address information of Storm configured in the Zookeeper, and accessing the UI page according to the acquired UI page address information to acquire data displayed on the UI page of Storm to obtain running state information.
4. The method according to claim 3, wherein the accessing the UI page according to the obtained UI page address information to obtain the data shown on the UI page of Storm comprises:
accessing data presented on the RESTful interface of the UI page in the Storm UI page according to the acquired address information containing the IP address and the port number of the UI page,
and if the RESTful interface is blocked or the access is unsuccessful, returning a prompt that the Storm UI cannot be accessed.
5. The method of claim 1, wherein the obtaining the running state information of Storm by calling an RPC server on Storm comprises:
starting a Thrift RPC client, connecting and calling an RPC server on Storm, and acquiring the running state information of the Storm.
6. An apparatus for monitoring a real-time stream computing platform, comprising:
a first obtaining module for obtaining the running state information of Storm by accessing the UI page of the real-time stream computing platform Storm,
a second obtaining module, configured to obtain running state information of Storm by calling an RPC server on Storm,
and the third acquisition module is used for acquiring the running state information of the Storm by analyzing the information written back by the Storm recorded in the Zookeeper so as to realize monitoring.
7. The apparatus of claim 6, wherein the apparatus comprises: and the failure prompt module is used for returning a failure prompt that the Storm running state information cannot be acquired if the third acquisition module fails to acquire the Storm running state information.
8. The apparatus according to claim 6, wherein the first obtaining module is specifically configured to obtain UI page address information of Storm configured in Zookeeper, and access a UI page according to the obtained UI page address information to obtain data displayed on the UI page of Storm to obtain running state information;
the second acquisition module is specifically used for starting a Thrift RPC client, connecting and calling an RPC server on Storm, and acquiring the running state information of Storm.
9. An electronic device, characterized in that the electronic device comprises: a memory and a processor communicatively coupled via an internal bus, the memory storing program instructions executable by the processor, the program instructions when executed by the processor implementing the method of any of claims 1-5.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811614036.3A CN111382022B (en) | 2018-12-27 | 2018-12-27 | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811614036.3A CN111382022B (en) | 2018-12-27 | 2018-12-27 | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111382022A true CN111382022A (en) | 2020-07-07 |
| CN111382022B CN111382022B (en) | 2024-02-20 |
Family
ID=71214545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811614036.3A Active CN111382022B (en) | 2018-12-27 | 2018-12-27 | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111382022B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103401934A (en) * | 2013-08-06 | 2013-11-20 | 广州唯品会信息科技有限公司 | Method and system for acquiring log data |
| CN105630652A (en) * | 2016-02-02 | 2016-06-01 | 中国石油大学(华东) | Real-time big data platform Storm oriented runtime three-dimensional visualization system |
| CN105681308A (en) * | 2016-01-18 | 2016-06-15 | 中国石油大学(华东) | Attribute abstract system orienting towards real time big data platform Storm |
| US20160261727A1 (en) * | 2015-03-06 | 2016-09-08 | Connie W. Yang | Systems and methods of streaming data |
| CN106201739A (en) * | 2016-06-29 | 2016-12-07 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of remote invocation method of Storm based on Redis |
| US20170148264A1 (en) * | 2014-06-16 | 2017-05-25 | Kinderguardian Inc. | System and method for managing behavior |
| US20170249126A1 (en) * | 2016-02-26 | 2017-08-31 | International Business Machines Corporation | Easy storm topology design and execution |
| CN109039730A (en) * | 2018-07-25 | 2018-12-18 | 网宿科技股份有限公司 | Server cluster and server cluster configuration information management method |
-
2018
- 2018-12-27 CN CN201811614036.3A patent/CN111382022B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103401934A (en) * | 2013-08-06 | 2013-11-20 | 广州唯品会信息科技有限公司 | Method and system for acquiring log data |
| US20170148264A1 (en) * | 2014-06-16 | 2017-05-25 | Kinderguardian Inc. | System and method for managing behavior |
| US20160261727A1 (en) * | 2015-03-06 | 2016-09-08 | Connie W. Yang | Systems and methods of streaming data |
| CN105681308A (en) * | 2016-01-18 | 2016-06-15 | 中国石油大学(华东) | Attribute abstract system orienting towards real time big data platform Storm |
| CN105630652A (en) * | 2016-02-02 | 2016-06-01 | 中国石油大学(华东) | Real-time big data platform Storm oriented runtime three-dimensional visualization system |
| US20170249126A1 (en) * | 2016-02-26 | 2017-08-31 | International Business Machines Corporation | Easy storm topology design and execution |
| CN106201739A (en) * | 2016-06-29 | 2016-12-07 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of remote invocation method of Storm based on Redis |
| CN109039730A (en) * | 2018-07-25 | 2018-12-18 | 网宿科技股份有限公司 | Server cluster and server cluster configuration information management method |
Non-Patent Citations (1)
| Title |
|---|
| 周明阳，闫超，郭良浩，徐鹏，任岁玲: "基于Storm的分布式实时信号处理系统" * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111382022B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11132356B2 (en) | Optimizing data entries in a log | |
| WO2020238066A1 (en) | Software pressure test method and apparatus, and storage medium and server | |
| US20150186129A1 (en) | Method and system for deploying a program module | |
| CN108804215B (en) | Task processing method and device and electronic equipment | |
| US9766927B1 (en) | Data flow management in processing workflows | |
| CN110244991B (en) | Micro-service dependency analysis method and device | |
| CN106411970B (en) | A kind of fault handling method based on service call, device and system | |
| CN111355622A (en) | Container traffic monitoring method, system and computer readable storage medium | |
| CN111124830B (en) | Micro-service monitoring method and device | |
| US10831565B2 (en) | Fault tolerant adapter system to consume database as a service | |
| CN104834602A (en) | Program issuing method, device and system | |
| CN113778486A (en) | Containerization processing method, device, medium and equipment for code pipeline | |
| CN108037984B (en) | Resource management method, system and readable storage medium for data analysis | |
| CN113127335B (en) | System testing method and device | |
| CN112905323A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN109902028A (en) | Automated testing method, device, equipment and the storage medium of ACL characteristic | |
| CN112486592A (en) | Distributed data processing method, device, server and readable storage medium | |
| CN109324892B (en) | Distributed management method, distributed management system and device | |
| CN117076096A (en) | Task flow execution method and device, computer readable medium and electronic equipment | |
| US10970055B2 (en) | Identifying software and hardware bottlenecks | |
| US10922145B2 (en) | Scheduling software jobs having dependencies | |
| CN114090268B (en) | Container management method and container management system | |
| CN111382022A (en) | Method and device for monitoring real-time streaming computing platform, electronic equipment and storage medium | |
| US9542171B2 (en) | Managing an application modification process | |
| WO2021036987A1 (en) | Method and device for achieving operation and maintenance monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 818, 8 / F, 34 Haidian Street, Haidian District, Beijing 100080 Applicant after: ULTRAPOWER SOFTWARE Co.,Ltd. Address before: 100089 Beijing city Haidian District wanquanzhuang Road No. 28 Wanliu new building 6 storey block A Room 601 Applicant before: ULTRAPOWER SOFTWARE Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |