CN111382022B - Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform - Google Patents
Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform Download PDFInfo
- Publication number
- CN111382022B CN111382022B CN201811614036.3A CN201811614036A CN111382022B CN 111382022 B CN111382022 B CN 111382022B CN 201811614036 A CN201811614036 A CN 201811614036A CN 111382022 B CN111382022 B CN 111382022B
- Authority
- CN
- China
- Prior art keywords
- storm
- state information
- running state
- page
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 229940126655 NDI-034858 Drugs 0.000 description 6
- 241000290929 Nimbus Species 0.000 description 6
- 230000005856 abnormality Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method, a device, electronic equipment and a storage medium for monitoring a real-time stream computing platform, wherein the method comprises the following steps: acquiring the running state information of the Storm by accessing a UI page of the Storm of the real-time stream computing platform, if the running state information is failed to be acquired, acquiring the running state information of the Storm by calling an RPC server on the Storm, and if the running state information is failed to be acquired, acquiring the running state information of the Storm by analyzing the information which is recorded in the Zookeeper and is written back by the Storm. According to the embodiment of the invention, information is tried to be acquired for monitoring through the three channels in sequence, so long as one channel can acquire the states of the Storm cluster and the Topology, the inquiry performance and the accuracy of the Storm cluster state monitoring are improved, and the problem of inaccurate monitoring is avoided.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a method, an apparatus, an electronic device, and a storage medium for monitoring a real-time streaming computing platform.
Background
The use of a large number of open-source middleware in IT monitoring systems, such as Storm, which is an open-source distributed, highly fault-tolerant real-time big data processing platform (i.e. real-time stream computing platform), has become widespread with more and more scenes intolerant to high delays, such as website statistics, recommendation systems, financial systems (high frequency trade, stocks) and the like, big data real-time processing solutions (i.e. stream computing) are becoming mainstream in stream computing technology, and the core components of Storm include: nimbus, supervisor, topology. Nimbus is a Master of Storm, responsible for resource allocation and task scheduling. One Storm cluster has only one Nimbus. Supervisor is a storage Slave and is responsible for receiving tasks distributed by Nimbus and managing all works. Topology: service topology, storm's topology is the encapsulation of real-time computing application logic, which is also understood to be a topology consisting of a series of Spout and Bolt that are related to each other by data streams.
Because IT monitoring system relies on Storm to process events, IT needs to monitor and manage the running state of Storm, and the accuracy requirement for Storm state is very high, if a node in Storm cluster has a problem, IT needs to report in time, otherwise IT can affect the running of the whole IT monitoring system. However, in the prior art, the accuracy rate of monitoring the Storm cluster state is not high, and the condition that Storm running state information cannot be obtained often occurs, so that the problem needs to be solved.
Disclosure of Invention
The invention provides a method, a device, electronic equipment and a storage medium for monitoring a real-time stream computing platform, which are used for sequentially verifying through three different acquisition modules, so that only one module acquires the state of a Storm cluster, the reliability of Storm monitoring is ensured, the accuracy of Storm cluster state monitoring is improved, and the normal operation of an IT monitoring system is further ensured.
According to one aspect of the present application, there is provided a method of monitoring a real-time streaming computing platform, comprising:
acquiring the running state information of the Storm by accessing the UI page of the real-time stream computing platform Storm,
if the acquisition of the running state information fails, acquiring the running state information of the Storm by calling a second acquisition module of an RPC server on the Storm,
and if the acquisition of the running state information fails, acquiring the running state information of the Storm by a third acquisition module for analyzing the information written back by the Storm recorded in the Zookeeper, so as to realize monitoring.
Optionally, the method further comprises: if the information recorded in the Zookeeper and written back by the Storm is failed to acquire the running state information of the Storm, a failure prompt that the running state information of the Storm cannot be acquired is returned.
Optionally, the acquiring the running state information of the Storm by accessing the UI page of the Storm includes:
acquiring UI page address information of a Storm configured in a Zookeeper, and accessing a UI page according to the acquired UI page address information to acquire data displayed on the UI page of the Storm to acquire running state information.
Optionally, accessing the UI page according to the acquired UI page address information to acquire the data shown on the UI page of the Storm includes:
accessing the RESTful interface of the UI page according to the acquired address information containing the IP address and the port number of the UI page to display data on the UI page of Storm,
if the RESTful interface blocks or the access is unsuccessful, a prompt that the Storm UI cannot access is returned.
Optionally, the acquiring the running state information of the Storm by calling the RPC server on the Storm includes:
and starting the thread RPC client, connecting and calling the RPC server on the Storm, and acquiring the running state information of the Storm.
According to another aspect of the present application, there is provided an apparatus for monitoring a real-time streaming computing platform, comprising:
a first acquisition module for acquiring the running state information of the Storm by accessing the UI page of the real-time stream computing platform Storm,
a second obtaining module, configured to obtain the running state information of the Storm by calling the RPC server on the Storm,
and the third acquisition module is used for acquiring the running state information of the Storm by analyzing the information written back by the Storm recorded in the Zookeeper, so as to realize monitoring.
Optionally, the apparatus comprises: and the failure prompt module is used for returning a failure prompt that the running state information of the Storm cannot be acquired if the third acquisition module fails to acquire the information.
Optionally, the first obtaining module is specifically configured to obtain UI page address information of a Storm configured in the Zookeeper, access the UI page according to the obtained UI page address information to obtain data displayed on the UI page of the Storm, and obtain running state information;
the second obtaining module is specifically configured to start the thread RPC client, connect to and call the RPC server on the Storm, and obtain the running state information of the Storm.
According to another aspect of the present application, there is provided an electronic device including: the device comprises a memory and a processor, wherein the memory is in communication connection with the processor through an internal bus, the memory stores program instructions capable of being executed by the processor, and the program instructions can realize the method according to one aspect of the application when being executed by the processor.
According to yet another aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as described in one aspect of the present application.
According to the method and the device for monitoring the Storm of the real-time stream computing platform, the running state information of the Storm is acquired by accessing the UI page of the Storm, if the acquisition fails, the running state information of the Storm is acquired by calling the RPC server on the Storm, and if the acquisition fails, the running state information of the Storm is acquired by analyzing the information which is recorded in the Zookeeper and is written back by the Storm, so that the monitoring is realized, and therefore, the sequential verification is carried out through three different information acquisition channels, as long as the state of a Storm cluster is acquired through one channel, the inquiry performance is improved, the reliability of the Storm monitoring and the accuracy of the state monitoring are ensured, and the normal running of an IT monitoring system is further ensured.
Drawings
FIG. 1 is a flow chart of a method of monitoring a real-time streaming computing platform in accordance with one embodiment of the present invention;
FIG. 2 is a flow chart of a method for monitoring a real-time streaming computing platform according to another embodiment of the present invention
FIG. 3 is a block diagram of an apparatus for monitoring a real-time streaming computing platform in accordance with one embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The IT monitoring system processes the event by relying on the Storm middleware, so that the requirement on the accuracy of the state of the Storm middleware is very high, if one Storm node in the cluster has a problem, the problem needs to be reported in time, and otherwise, the operation of the whole IT monitoring system can be influenced.
In this regard, in the embodiment of the present invention, the three different acquisition modules are used for sequential verification, so that it is ensured that the state information of the Storm cluster can be acquired to realize monitoring, and the flow can be ended as long as one module can acquire the information, that is, in practical application, the monitoring of the Storm cluster can be realized by any one of the three acquisition modules in the embodiment.
However, a single monitoring means has the problems that it is difficult to obtain the state information of the Storm, for example, when the condition of the Storm cluster is monitored by using the thread of the RPC (Remote Procedure Call) framework, the Java source code related to the thread Client needs to be obtained by compiling the source code of the Storm by using the thread, if the versions of the Storm are inconsistent, the process is tedious, time and labor are wasted, and the problems encountered in the monitoring process are more numerous, for example, if one party receives and transmits data and fails to complete data transmission within a set time under the condition that the timeout mechanism of the server and the Client is set, the other party can report an abnormality due to forced disconnection. Here, thread is a cross-language service deployment framework, where thread defines the interface and data type of RPC through IDL (Interface Definition Language ), and then generates code in different languages through a thread compiler to conveniently generate a cross-programming language for RPC client and server communication. Thread supports multiple transport protocols, and a user can select an appropriate type according to actual requirements.
For another example, to reduce the use of thread to monitor Storm status, storm cluster data is acquired through a REST interface provided by the Storm UI. The method has the defects that interfaces are needed to be provided by depending on a Storm UI, if the Nimbus nodes of the Storm are overloaded and the nodes are down, the Storm UI of the Storm cluster which operates normally fails to open or fails to refresh all the time, so that the operation state information of the Storm cluster cannot be accurately acquired.
Preferably, the exception is avoided when the data transmission is not completed within the timeout time through thread monitoring, and the problem of inaccurate monitoring caused by failure of opening or continuous refreshing of the Storm UI is avoided. In the embodiment of the invention, a RESTful interface is provided through a Storm UI first, and states of Storm clusters, the morphology and the like are obtained; if the Storm UI provides an interface, blocking occurs and the access is unsuccessful; monitoring the Storm cluster state through the RPC framework thread; if thread is abnormal due to overtime; and finally, writing state information back to the Zookeeper through the Storm, and analyzing to obtain the state of the Storm cluster. The sequence verification is carried out in the three modes, and the process can be ended as long as the state of the Storm cluster can be obtained in one mode.
FIG. 1 is a flow chart of a method of monitoring a real-time streaming computing platform according to one embodiment of the present invention, see FIG. 1, comprising the steps of:
step S101, obtaining the running state information of Storm by accessing the UI page of Storm of the real-time stream computing platform,
step S102, if the operation state information acquisition fails, the operation state information of the Storm is acquired by calling an RPC server on the Storm,
step S103, if the operation state information acquisition fails, the operation state information of the Storm is acquired by analyzing the information written back by the Storm recorded in the Zookeeper.
In this embodiment, if the third acquisition channel fails to acquire the running state information of the Storm, that is, if the running state information of the Storm is failed to acquire by analyzing the information written back by the Storm recorded in the Zookeeper, a failure prompt that the running state information of the Storm cannot be acquired is returned.
As can be seen from fig. 1, in the method for monitoring a real-time streaming computing platform Storm in this embodiment, three different information acquisition modules are used to sequentially verify, so long as a channel acquires a condition of a Storm cluster, query performance is improved, reliability of Storm monitoring and accuracy of condition monitoring are ensured, and normal operation of an IT monitoring system is facilitated.
The implementation steps of the method for monitoring a real-time streaming computing platform Storm according to the embodiment of the present invention will be described with reference to fig. 2, where the flow starts, step S201 is executed, and Storm configuration information is loaded.
When initializing data, before the system is started, the Storm cluster address and Storm UI address information are stored in a Zookeeper. The ZooKeeper is a distributed application coordination service, is a manager of the cluster, monitors the states of all nodes in the cluster, and performs next reasonable operation according to feedback submitted by the nodes. Finally, a simple and easy-to-use interface and a system with high performance and stable functions are provided for users. Programs always need to be configured, and if the programs are distributed and deployed on multiple machines, it becomes difficult to change the configuration one by one. All the configurations are put on the Zookeeper, stored in a certain directory node of the Zookeeper, all related application programs monitor the directory node, once the configuration information changes, each application program can receive the notification of the Zookeeper, and then new configuration information is acquired from the Zookeeper and applied to the system.
In the step, the configured Storm UI address information is acquired from the Zookeeper. Here, the Storm UI address information includes IP address information and port number information of UI (User Interface) pages.
Step S202, acquiring running state information through a RESTful interface provided by a Storm UI;
in this embodiment, UI page address information of a Storm configured in a Zookeeper is obtained, and a UI page is accessed according to the obtained UI page address information to obtain data displayed on the UI page of the Storm, so as to obtain running state information.
Specifically, the RESTful interface of the UI page is accessed according to the acquired address information including the IP address and the port number of the UI page to display data on the UI page of Storm, if the RESTful interface is blocked or the access is unsuccessful, a prompt that the Storm UI cannot be accessed is returned, that is, if an abnormality occurs, step S203 is executed, and a Storm UI abnormal event is generated to prompt that the acquisition mode is unsuccessful. If the Storm UI provides that the interface access is successful, the relevant state is returned directly. Note that, the Storm provides UI interface to show running state data, such as the number of times of occurrence of an abnormality of a certain topology, and stacks of occurrence of the abnormality.
Step S204, if the information is correctly acquired, the flow is ended, otherwise step S205 is executed,
in this step, it is determined whether the operation state information of Storm is correctly acquired in step S203, where the operation state information of Storm includes state information (such as normal and abnormal) of master node Nimbus, slave node super, and service Topology of Storm. If yes, the flow ends, otherwise the attempt continues using the second approach.
Step S205, acquiring running state information through Trift RPC;
specifically, acquiring the running state information through the Trift RPC comprises starting a thread RPC client, connecting and calling an RPC server on the Storm, and acquiring the running state information of the Storm. This is by function call, access to the underlying interface of Storm, request service to get the running state information of Storm.
Step S206, if the information is correctly acquired, the flow ends, otherwise step S207 is executed,
in this step, it is determined whether the operation state information of Storm is correctly obtained in step S205, if yes, the monitoring process is ended once, otherwise, the third mode is continuously used for trying.
And S207, analyzing the information written back by Storm recorded in the Zookeeper, and obtaining the running state information.
If the monitoring of the Storm cluster state fails through the RPC framework thread, the analysis is carried out to obtain the Storm cluster state through the state information written back to the Zookeeper by the Storm, and the state information is returned.
Specifically, recorded information written back by Storm is obtained from the Zookeeper, and the running state information of Storm is obtained after analysis.
It should be noted that, the Zookeeper is a manager of the Storm cluster, and the Storm can write back the information of the key index, such as heartbeat information, into the Zookeeper at regular time, and the running state of the Storm cluster can be determined by acquiring the heartbeat information.
Step S208, if the information is correctly acquired, the flow is ended, otherwise, step S209 is executed to generate an event that the Storm running state cannot be acquired.
In step S208, it is determined whether the running state information of the Storm cluster is correctly acquired, and if so, the primary monitoring information acquisition process is ended. If the monitoring information is not acquired, generating an event that the storage running state cannot be acquired, and generating prompt information to prompt that the monitoring information acquisition fails.
It can be known that in the embodiment of the present invention, the operation state information of the Storm cluster is obtained by sequentially performing the attempts by three different modules. The first acquisition module accessing the UI page of the Storm is used as a first priority attempt mode because factors such as query performance and complexity are considered, and the first acquisition module is not used for acquiring the UI page of the Storm, and the second acquisition module is used for acquiring the UI page of the Storm until the running state information of the Storm cluster is acquired or the acquisition fails. In addition, the reason why the order of the third obtaining module is ranked at the end in this embodiment is that, in general, the Storm running state information obtained by using the third obtaining module is limited, and may only include very critical running state information, for example, whether an abnormality occurs, and details of the abnormality are not recorded, so that the detailed query and monitoring requirements may not be satisfied.
As well as the method for monitoring a real-time streaming computing platform described above, the embodiment of the present invention further provides an apparatus for monitoring a real-time streaming computing platform, referring to fig. 3, an apparatus 300 for monitoring a real-time streaming computing platform includes:
a first obtaining module 301, configured to obtain running state information of a Storm by accessing a UI page of the real-time streaming computing platform Storm,
a second obtaining module 302, configured to obtain the running state information of the Storm by calling the RPC server on the Storm,
and a third obtaining module 303, configured to obtain the running state information of the Storm by analyzing the information written back by the Storm recorded in the Zookeeper.
In one embodiment of the present invention, the apparatus 300 for monitoring a real-time streaming computing platform further comprises: and the failure prompt module is used for returning a failure prompt that the running state information of the Storm cannot be acquired if the third acquisition module fails to acquire the information.
In one embodiment of the present invention, the first obtaining module 301 is specifically configured to obtain UI page address information of a Storm configured in a Zookeeper, access a UI page according to the obtained UI page address information to obtain data displayed on the UI page of the Storm, and obtain running state information; the second obtaining module 302 is specifically configured to start the thread RPC client, connect to and call the RPC server on the Storm, and obtain the running state information of the Storm.
It should be noted that, the explanation of each function executed by each module in the apparatus for monitoring a real-time streaming computing platform Storm shown in fig. 3 is consistent with the explanation of the foregoing method embodiment, and will not be repeated here.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 4, the electronic device includes a memory 401 and a processor 402, where the memory 401 and the processor 402 are communicatively connected through an internal bus 403, and the memory 401 stores program instructions that can be executed by the processor 402, and the program instructions when executed by the processor 402 can implement the method for monitoring a real-time streaming computing platform Storm described above.
Further, the logic instructions in the memory 401 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Another embodiment of the present invention provides a computer readable storage medium storing computer instructions for causing a computer to perform the method for monitoring a real-time streaming computing platform Storm described above.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the description of the present invention, numerous specific details are set forth. It may be evident, however, that the embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description. Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
The foregoing is merely a specific embodiment of the invention and other modifications and variations can be made by those skilled in the art in light of the above teachings. It is to be understood by persons skilled in the art that the foregoing detailed description is provided for the purpose of illustrating the invention more fully, and that the scope of the invention is defined by the appended claims.
Claims (5)
1. A method for monitoring a real-time streaming computing platform, applied to an IT monitoring system, comprising:
acquiring running state information of a Storm by accessing a UI page of the Storm of a real-time stream computing platform, acquiring UI page address information of the Storm configured in a Zookeeper, and accessing the UI page according to the acquired UI page address information to acquire data displayed on the UI page of the Storm to acquire the running state information;
if the acquisition of the running state information fails, starting a thread RPC client, connecting and calling an RPC server on the Storm, and acquiring the running state information of the Storm;
if the acquisition of the running state information fails, the running state information of the Storm is acquired by analyzing the information which is recorded in the Zookeeper and is written back by the Storm, so that monitoring is realized;
if the information recorded in the Zookeeper and written back by the Storm is failed to acquire the running state information of the Storm, a failure prompt that the running state information of the Storm cannot be acquired is returned.
2. The method of claim 1, wherein accessing the UI page according to the acquired UI page address information to acquire data presented on the UI page of Storm comprises:
accessing the RESTful interface of the UI page according to the acquired address information containing the IP address and the port number of the UI page to display data on the UI page of Storm,
if the RESTful interface blocks or the access is unsuccessful, a prompt that the Storm UI cannot access is returned.
3. An apparatus for monitoring a real-time streaming computing platform, for use in an IT monitoring system, comprising:
the first acquisition module is used for acquiring running state information of the Storm by accessing a UI page of the Storm of the real-time stream computing platform, acquiring UI page address information of the Storm configured in the Zookeeper, and accessing the UI page according to the acquired UI page address information to acquire data displayed on the UI page of the Storm to acquire the running state information;
the second acquisition module is used for starting the thread RPC client, connecting and calling the RPC server on the Storm, and acquiring the running state information of the Storm;
the third acquisition module is used for acquiring the running state information of the Storm by analyzing the information written back by the Storm recorded in the Zookeeper, so as to realize monitoring; if the information recorded in the Zookeeper and written back by the Storm is failed to acquire the running state information of the Storm, a failure prompt that the running state information of the Storm cannot be acquired is returned.
4. An electronic device, the electronic device comprising: the memory and the processor are in communication connection through an internal bus, and the memory stores program instructions capable of being executed by the processor, and the program instructions are capable of implementing the method of any one of claims 1-2 when executed by the processor.
5. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811614036.3A CN111382022B (en) | 2018-12-27 | 2018-12-27 | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811614036.3A CN111382022B (en) | 2018-12-27 | 2018-12-27 | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111382022A CN111382022A (en) | 2020-07-07 |
| CN111382022B true CN111382022B (en) | 2024-02-20 |
Family
ID=71214545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811614036.3A Active CN111382022B (en) | 2018-12-27 | 2018-12-27 | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111382022B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103401934A (en) * | 2013-08-06 | 2013-11-20 | 广州唯品会信息科技有限公司 | Method and system for acquiring log data |
| CN105630652A (en) * | 2016-02-02 | 2016-06-01 | 中国石油大学(华东) | Real-time big data platform Storm oriented runtime three-dimensional visualization system |
| CN105681308A (en) * | 2016-01-18 | 2016-06-15 | 中国石油大学(华东) | Attribute abstract system orienting towards real time big data platform Storm |
| CN106201739A (en) * | 2016-06-29 | 2016-12-07 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of remote invocation method of Storm based on Redis |
| CN109039730A (en) * | 2018-07-25 | 2018-12-18 | 网宿科技股份有限公司 | Server cluster and server cluster configuration information management method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170148264A1 (en) * | 2014-06-16 | 2017-05-25 | Kinderguardian Inc. | System and method for managing behavior |
| US10027728B2 (en) * | 2015-03-06 | 2018-07-17 | Ebay Inc. | Systems and methods of streaming data |
| US20170249126A1 (en) * | 2016-02-26 | 2017-08-31 | International Business Machines Corporation | Easy storm topology design and execution |
-
2018
- 2018-12-27 CN CN201811614036.3A patent/CN111382022B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103401934A (en) * | 2013-08-06 | 2013-11-20 | 广州唯品会信息科技有限公司 | Method and system for acquiring log data |
| CN105681308A (en) * | 2016-01-18 | 2016-06-15 | 中国石油大学(华东) | Attribute abstract system orienting towards real time big data platform Storm |
| CN105630652A (en) * | 2016-02-02 | 2016-06-01 | 中国石油大学(华东) | Real-time big data platform Storm oriented runtime three-dimensional visualization system |
| CN106201739A (en) * | 2016-06-29 | 2016-12-07 | 上海浦东发展银行股份有限公司信用卡中心 | A kind of remote invocation method of Storm based on Redis |
| CN109039730A (en) * | 2018-07-25 | 2018-12-18 | 网宿科技股份有限公司 | Server cluster and server cluster configuration information management method |
Non-Patent Citations (1)
| Title |
|---|
| 周明阳，闫超，郭良浩，徐鹏，任岁玲 .基于Storm的分布式实时信号处理系统.计算机应用.2017,第0-第2部分. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111382022A (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020238066A1 (en) | Software pressure test method and apparatus, and storage medium and server | |
| CN112910945B (en) | Request link tracking method and service request processing method | |
| CN108768791B (en) | Information acquisition configuration management system and method | |
| US11132356B2 (en) | Optimizing data entries in a log | |
| CN110309161B (en) | Data synchronization method and device and server | |
| US20150205699A1 (en) | Service modeling and virtualization | |
| JP6130518B2 (en) | Instance host configuration | |
| US7519908B2 (en) | Application server configuration tool | |
| US20070033441A1 (en) | System for and method of multi-location test execution | |
| CN106411970B (en) | A kind of fault handling method based on service call, device and system | |
| US10831565B2 (en) | Fault tolerant adapter system to consume database as a service | |
| EP3365774B1 (en) | System and method for booting application servers in parallel | |
| CN109361777B (en) | Synchronization method, synchronization system and related device for distributed cluster node states | |
| CN110933152B (en) | Preheating method, device and system and electronic equipment | |
| CN114338684A (en) | Energy management system and method | |
| Tang et al. | Application centric lifecycle framework in cloud | |
| CN117076096A (en) | Task flow execution method and device, computer readable medium and electronic equipment | |
| CN110929130B (en) | Public security level audit data query method based on distributed scheduling | |
| CN111382022B (en) | Method, device, electronic equipment and storage medium for monitoring real-time stream computing platform | |
| US10970055B2 (en) | Identifying software and hardware bottlenecks | |
| CN109324892B (en) | Distributed management method, distributed management system and device | |
| CN116775420A (en) | Information creation cloud platform resource display and early warning method and system based on Flink flow calculation | |
| US20120173658A1 (en) | Schema Validation Proxy | |
| CN115858499A (en) | Database partition processing method and device, computer equipment and storage medium | |
| CN111381921B (en) | Front-end and back-end separation system and method based on Ambari |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 818, 8 / F, 34 Haidian Street, Haidian District, Beijing 100080 Applicant after: ULTRAPOWER SOFTWARE Co.,Ltd. Address before: 100089 Beijing city Haidian District wanquanzhuang Road No. 28 Wanliu new building 6 storey block A Room 601 Applicant before: ULTRAPOWER SOFTWARE Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |