CN109614164B - Method, device and equipment for realizing configurable plug-in and readable storage medium - Google Patents
Method, device and equipment for realizing configurable plug-in and readable storage medium Download PDFInfo
- Publication number
- CN109614164B CN109614164B CN201811448379.7A CN201811448379A CN109614164B CN 109614164 B CN109614164 B CN 109614164B CN 201811448379 A CN201811448379 A CN 201811448379A CN 109614164 B CN109614164 B CN 109614164B
- Authority
- CN
- China
- Prior art keywords
- plug
- data acquisition
- node
- acquisition module
- instruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
- G06F9/44526—Plug-ins; Add-ons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a method, a device, equipment and a readable storage medium for realizing configurable plug-ins, wherein the method comprises the following steps: when the fact that the data acquisition module corresponding to the ZooKeeper is successfully registered is detected, whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not is detected, wherein one data acquisition module corresponds to one or more plugin nodes; when the management instruction is detected, generating a notification message according to the management instruction; and broadcasting the notification message to a data acquisition module which has subscribed to the management event corresponding to the management instruction in the ZooKeeper so that the data acquisition module can execute the operation corresponding to the management instruction. The invention realizes pluggable configuration of the plug-in of the data acquisition module, and realizes the operations of adding and deleting the plug-in under the condition of maintaining the whole ZooKeeper open source framework unchanged.
Description
Technical Field
The invention relates to the technical field of ZooKeeper, in particular to a method, a device, equipment and a readable storage medium for realizing configurable plug-ins.
Background
In the task scheduling process, a Spring's tasks scheduler is currently used, and the method is to utilize the container injection characteristic of an open source frame Spring, inject each task plug-in into a container when the Spring frame is started, and then trigger the function of the task plug-in at a specified time through the scheduling plug-in node, thereby completing task scheduling. Task scheduling by Spring's quartz scheduler has the disadvantage that plug-ins cannot be dynamically added, deleted. If a new plugin needs to be released, the whole Spring framework needs to be recompiled, packaged and deployed, and the new plugin can take effect. Similarly, if a certain plug-in needs to be logged out, the above procedure also needs to be repeated, so that it can be seen that, in the task scheduling process, the plug-in cannot implement pluggable configuration, i.e., the plug-in cannot implement operations such as adding and deleting the plug-in under the condition of maintaining the whole open source framework unchanged.
Disclosure of Invention
The invention mainly aims to provide a method, a device, equipment and a readable storage medium for realizing configuration of a plug-in, and aims to solve the technical problem that the plug-in cannot realize pluggable configuration in the task scheduling process.
In order to achieve the above object, the present invention provides a method for implementing configuration of a plug-in, including the steps of:
when the fact that the data acquisition module corresponding to the ZooKeeper is successfully registered is detected, whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not is detected, wherein one data acquisition module corresponds to one or more plugin nodes;
when the management instruction is detected, generating a notification message according to the management instruction;
and broadcasting the notification message to a data acquisition module which has subscribed to the management event corresponding to the management instruction in the ZooKeeper so that the data acquisition module can execute the operation corresponding to the management instruction.
Preferably, when the management instruction is a delete instruction, the step of broadcasting the notification message to a data collection module that has subscribed to a management event corresponding to the management instruction in the ZooKeeper, so that the data collection module executes an operation corresponding to the management instruction includes:
and broadcasting the notification message to a data acquisition module which has subscribed to the deletion event corresponding to the deletion instruction in the ZooKeeper, so that after the data acquisition module receives the deletion instruction, if a plug-in node corresponding to the deletion instruction is detected to be a main plug-in node, after the main plug-in node is logged out according to the deletion instruction, one plug-in node is selected from standby plug-in nodes to serve as the main plug-in node.
Preferably, after the step of broadcasting the notification message to the data collection module that has subscribed to the management event corresponding to the management instruction in the ZooKeeper, for the data collection module to execute the operation corresponding to the management instruction, the method further includes:
generating a task scheduling message after detecting a task scheduling instruction;
and broadcasting the task scheduling message to a target plug-in node subscribed with the task scheduling message so that the target plug-in node can execute the task corresponding to the task scheduling instruction.
Preferably, after the step of broadcasting the task scheduling message to the target plug-in node subscribed to the task scheduling message for the target plug-in node to execute the task corresponding to the task scheduling instruction, the method further includes:
and determining a first target plug-in node in the target plug-in nodes, which acquires the task scheduling message, and distributing the task corresponding to the task scheduling instruction to the first target plug-in node, which acquires the task scheduling message.
Preferably, after the step of determining a first target plug-in node from the target plug-in nodes that obtained the task scheduling message and distributing the task corresponding to the task scheduling instruction to the first target plug-in node that obtained the task scheduling message, the method further includes:
and moving the task corresponding to the task scheduling instruction from a newly added task list to an executed task list so as to prohibit other target plug-in nodes except the first target plug-in node for acquiring the task scheduling message from acquiring the task corresponding to the task scheduling instruction from the newly added task list.
Preferably, before the step of detecting whether a management instruction for managing a plug-in node in the ZooKeeper is detected after the data acquisition module in the ZooKeeper is successfully registered, the method further includes:
after receiving a registration request triggered when the data acquisition module is started, checking whether only one data acquisition module in the running state in a host corresponding to the data acquisition module is available, and detecting whether the data acquisition module in the running state establishes communication connection with the ZooKeeper;
and if only one data acquisition module in the running state is arranged in the host, and the data acquisition module in the running state establishes communication connection with the ZooKeeper, executing the registration request.
Preferably, if there is only one data acquisition module in the host in the running state, and the data acquisition module in the running state establishes a communication connection with the ZooKeeper, after the step of executing the registration request, the method further includes:
and creating a node list of the host and a node list of the data acquisition module in a running state under a root directory of the registration namespace of the ZooKeeper.
Preferably, before the step of detecting whether a management instruction for managing a plug-in node in the ZooKeeper is detected after the data acquisition module corresponding to the ZooKeeper is detected to be successfully registered, the method further includes:
and when the data acquisition module is detected to be started, generating a task plug-in list of the host corresponding to the data acquisition module in the ZooKeeper.
In addition, to achieve the above object, the present invention further provides an apparatus for implementing configurable plug-ins, where the apparatus for implementing configurable plug-ins includes:
the detection module is used for detecting whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not after the registration of the data acquisition module corresponding to the ZooKeeper is detected to be successful, wherein one data acquisition module corresponds to one or more plugin nodes;
the generating module is used for generating a notification message according to the management instruction after the management instruction is detected;
and the broadcasting module is used for broadcasting the notification message to a data acquisition module which has subscribed the management event corresponding to the management instruction in the ZooKeeper so that the data acquisition module can execute the operation corresponding to the management instruction.
In addition, to achieve the above object, the present invention further provides an apparatus for implementing configurable plug-ins, which includes a memory, a processor and a program stored in the memory and capable of running on the processor, wherein when the program is executed by the processor, the method for implementing configurable plug-ins as described above is implemented.
In addition, to achieve the above object, the present invention also provides a computer readable storage medium, which stores a program for implementing plug-in configuration, and when the program for implementing plug-in configuration is executed by a processor, the method for implementing plug-in configuration as described above is implemented.
According to the method, after the registration of the data acquisition module corresponding to the ZooKeeper is detected to be successful, whether a management instruction for managing the plug-in nodes in the ZooKeeper is detected or not is detected; when the management instruction is detected, generating a notification message according to the management instruction; the notification message is broadcasted to the data acquisition modules subscribed in the ZooKeeper and corresponding to the management events of the management instructions, so that the data acquisition modules execute the operation corresponding to the management instructions, and by setting the data acquisition modules, each data acquisition module can correspond to one or more plugin nodes and manage the plugin nodes, thereby realizing pluggable configuration of plugins of the data acquisition modules, realizing the operations of adding, deleting and the like of the plugin nodes under the condition of maintaining the whole ZooKeeper open source framework unchanged, namely realizing the operations of adding, deleting and the like of the plugins.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a first embodiment of a method for implementing plug-in configuration according to the present invention;
FIG. 3 is a block diagram of a framework to which embodiments of the present invention are applied;
FIG. 4 is a flow chart illustrating a second embodiment of a method for implementing plug-in configuration according to the present invention;
fig. 5 is a flow chart illustrating a third embodiment of the method for implementing plug-in configuration according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present invention.
It should be noted that fig. 1 is a schematic structural diagram of a hardware operating environment for implementing a device with configurable plug-ins. The equipment for realizing the configurable plug-in the embodiment of the invention can be terminal equipment such as a PC, a portable computer and the like.
As shown in fig. 1, the device that the implementation plug-in is configurable may include: a processor 1001, such as a CPU, a network interface 1004, a memory 1005, and a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the architecture of the device implementing the plug-in configurable shown in fig. 1 does not constitute a limitation of the device implementing the plug-in configurable, and may include more or fewer components than shown, or some components combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a program that is configurable to implement a plug-in. The operating system is a program for managing and controlling hardware and software resources of the device for realizing the configurable plug-in, and supports the operation of the program for realizing the configurable plug-in and other software or programs.
In the device for implementing configurable plug-ins shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and communicating data with the backend server; and processor 1001 may be used to call an implementation plug-in configurable program stored in memory 1005 and perform the following operations:
when the fact that the data acquisition module corresponding to the ZooKeeper is successfully registered is detected, whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not is detected, wherein one data acquisition module corresponds to one or more plugin nodes;
when the management instruction is detected, generating a notification message according to the management instruction;
and broadcasting the notification message to a data acquisition module which has subscribed to the management event corresponding to the management instruction in the ZooKeeper so that the data acquisition module can execute the operation corresponding to the management instruction.
Further, when the management instruction is a delete instruction, the step of broadcasting the notification message to a data collection module that has subscribed to a management event corresponding to the management instruction in the ZooKeeper, so that the data collection module executes an operation corresponding to the management instruction includes:
and broadcasting the notification message to a data acquisition module which has subscribed to the deletion event corresponding to the deletion instruction in the ZooKeeper, so that after the data acquisition module receives the deletion instruction, if a plug-in node corresponding to the deletion instruction is detected to be a main plug-in node, after the main plug-in node is logged out according to the deletion instruction, one plug-in node is selected from standby plug-in nodes to serve as the main plug-in node.
Further, after the step of broadcasting the notification message to the data collection module in the ZooKeeper, which has subscribed to the management event corresponding to the management instruction, so that the data collection module executes the operation corresponding to the management instruction, the processor 1001 may further be configured to call a configurable program based on a block chain implementation plug-in stored in the memory 1005, and execute the following steps:
generating a task scheduling message after detecting a task scheduling instruction;
and broadcasting the task scheduling message to a target plug-in node subscribed with the task scheduling message so that the target plug-in node can execute the task corresponding to the task scheduling instruction.
Further, after the step of broadcasting the task scheduling message to the target plug-in node subscribed to the task scheduling message for the target plug-in node to execute the task corresponding to the task scheduling instruction, the processor 1001 may further be configured to call a configurable program based on a blockchain implementation plug-in stored in the memory 1005, and perform the following steps:
and determining a first target plug-in node in the target plug-in nodes, which acquires the task scheduling message, and distributing the task corresponding to the task scheduling instruction to the first target plug-in node, which acquires the task scheduling message.
Further, after determining the target plug-in node from which the task scheduling message is first obtained, and distributing the task corresponding to the task scheduling instruction to the target plug-in node from which the task scheduling message is first obtained, the processor 1001 may be further configured to invoke a configurable program based on a blockchain stored in the memory 1005, and execute the following steps:
and moving the task corresponding to the task scheduling instruction from a newly added task list to an executed task list so as to prohibit other target plug-in nodes except the first target plug-in node for acquiring the task scheduling message from acquiring the task corresponding to the task scheduling instruction from the newly added task list.
Further, before the step of detecting whether a management instruction for managing a plugin node in the ZooKeeper is detected after the data acquisition module in the ZooKeeper is detected to be successfully registered, the processor 1001 may be further configured to call a configurable program for implementing a plugin based on a block chain stored in the memory 1005, and execute the following steps:
after receiving a registration request triggered when the data acquisition module is started, checking whether only one data acquisition module in the running state in a host corresponding to the data acquisition module is available, and detecting whether the data acquisition module in the running state establishes communication connection with the ZooKeeper;
and if only one data acquisition module in the running state is arranged in the host, and the data acquisition module in the running state establishes communication connection with the ZooKeeper, executing the registration request.
Further, if there is only one data acquisition module in the host in the running state, and the data acquisition module in the running state establishes a communication connection with the ZooKeeper, after the step of executing the registration request, the processor 1001 may be further configured to call a configurable program based on a block chain implementation plug-in stored in the memory 1005, and execute the following steps:
and creating a node list of the host and a node list of the data acquisition module in a running state under a root directory of the registration namespace of the ZooKeeper.
Further, before the step of detecting whether a management instruction for managing a plugin node in the ZooKeeper is detected after the registration of the data acquisition module corresponding to the ZooKeeper is detected to be successful, the processor 1001 may be further configured to call a configurable program for implementing a plugin based on a block chain stored in the memory 1005, and execute the following steps:
and when the data acquisition module is detected to be started, generating a task plug-in list of the host corresponding to the data acquisition module in the ZooKeeper.
Based on the above structure, various embodiments of the method for realizing the configurable plug-in are provided.
Referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of a method for implementing configurable plug-ins according to the present invention.
While a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than that shown or described herein.
The method for realizing the configurable plug-in is applied to a server or a terminal, and the terminal can comprise a mobile terminal such as a mobile phone, a wearable device, a tablet computer, a notebook computer, a palm computer, a Personal Digital Assistant (PDA) and the like, and a fixed terminal such as a digital TV, a desktop computer and the like. In various embodiments of the method for implementing the configurable plug-in, the execution body is omitted for ease of description to illustrate the various embodiments. The method for realizing the configurable plug-in comprises the following steps:
step S10, after detecting that the registration of the data acquisition module corresponding to the ZooKeeper is successful, detecting whether a management instruction for managing the plugin nodes in the ZooKeeper is detected, wherein one data acquisition module corresponds to one or more plugin nodes.
ZooKeeper is a distributed, open source code distributed application program coordination service, is an open source implementation of Chubby of Google, is an important component of Hadoop and Hbase, is software for providing a consistency service for distributed applications, and provides services including: configuration maintenance, domain name service, distributed synchronization and group service, etc. The data acquisition module is ICP (Intelligent Capacity Planner) -ZK (zooKeeper), and the ICP-ZK is the data acquisition module of the ICP, and the data acquisition module realizes the characteristics of high availability of a distributed type, support of thermal deployment, horizontal expansion and the like based on the ZK. The ICP is an intelligent capacity management system, the resource use condition, the application system capacity display, the monitoring and the analysis are established on the basis of data reported by a monitoring, configuration management center and an external system through interfaces, a capacity model is established, the future trend is predicted according to the historical condition, the resource bottleneck of the capacity model is analyzed, and reference is provided for resource or capacity adjustment.
The method for realizing the configuration of the plug-ins in the embodiment of the present invention is applied to the framework structure shown in fig. 3, as can be seen from fig. 3, ZooKeeper is connected with ICP-ZK (data acquisition module), a plurality of plug-in nodes (Worker) are set in the data acquisition module, and the data acquisition module is used as a plug-in and can correspond to one or a plurality of plug-in nodes. In the data acquisition module, a main plug-in node (Scheduler Master) and a standby plug-in node (Scheduler Slave) are set. Data can be collected in each data storage module storing data through the plug-in node, such as collecting data from TDSQL (Tencent distributed MySQL), Hive (data warehouse tool), HBase, Spark, or CMDB (Configuration Management Database), etc. The ZooKeeper can generate tasks and then distribute the tasks to the plug-in nodes, and the ZooKeeper is managed by an ICP server (ICP WEB).
And when the data acquisition module corresponding to the ZooKeeper is detected to be successfully registered, detecting whether a management instruction for managing the plug-in node in the ZooKeeper is detected. The management instruction is sent to the plug-in node of the data acquisition module by the ZooKeeper, and the management instruction includes but is not limited to a delete instruction and an add instruction.
Step S20, when the management instruction is detected, a notification message is generated according to the management instruction.
Step S30, broadcasting the notification message to a data collection module in the ZooKeeper that has subscribed to the management event corresponding to the management instruction, so that the data collection module executes an operation corresponding to the management instruction.
And when the management instruction is detected, automatically generating a notification message according to the management instruction. If the management instruction is a deletion instruction for deleting the existing plug-in node, a deletion notification message is generated; and when the management instruction is an adding instruction for adding a new plug-in node, generating an adding notification message. And after the notification message is generated, the notification message is broadcasted to a data acquisition module which subscribes the management event corresponding to the management instruction in the ZooKeeper so that the data acquisition module can execute the operation corresponding to the management instruction. It should be noted that, after the data acquisition module establishes a communication connection with the ZooKeeper, it may be determined that the data acquisition module is a data acquisition module that has subscribed to the management event corresponding to the management instruction. After the data acquisition module receives the notification message, the data acquisition module starts a binary resource manager (Class Loader), and executes an operation corresponding to the management instruction through the binary resource manager. If the management instruction is a deletion instruction, the data acquisition module executes logout operation of a plugin node corresponding to the logout deletion instruction through the binary resource manager; and when the management instruction is an adding instruction, the data acquisition module executes the loading operation of the plug-in node corresponding to the loading adding instruction through the binary resource manager. The Class Loader is used for dynamically loading a Class file into a virtual machine and converting the Class file into an instance of a Java.
Further, when the management instruction is a deletion instruction, step S30 includes:
step a, broadcasting the notification message to a data acquisition module which has subscribed the deletion event corresponding to the deletion instruction in the ZooKeeper, so that after the data acquisition module receives the deletion instruction, if it is detected that a plug-in node corresponding to the deletion instruction is a main plug-in node, after the main plug-in node is cancelled according to the deletion instruction, a plug-in node is selected from standby plug-in nodes as the main plug-in node.
Further, when the management instruction is a deletion instruction, the notification message is broadcast to the data acquisition module subscribed in the ZooKeeper and corresponding to the deletion event of the deletion instruction. And when the data acquisition module subscribed to the deletion event receives the deletion instruction, detecting whether the plug-in node corresponding to the deletion instruction is the main plug-in node. If the plug-in node corresponding to the deletion instruction is the main plug-in node, the data acquisition module subscribed to the deletion event selects one plug-in node from the standby plug-in nodes as the main plug-in node after the main plug-in node is cancelled according to the deletion instruction. It should be noted that one or more standby plug-in nodes may exist in one data collection module. In the process of selecting one plug-in node from the standby plug-in nodes as the main plug-in node, one plug-in node can be randomly selected from the standby plug-in nodes as the main plug-in node, and the plug-in node can be selected from the standby plug-in nodes as the main plug-in node from front to back according to the arrangement sequence of the standby plug-in nodes. In the embodiment of the invention, the main plug-in node carries the main node identification, and the standby plug-in node carries the standby node identification, so that whether the plug-in node is the main plug-in node or the standby plug-in node can be determined through the identification carried by the plug-in node. If the primary node identification may be set to "a", the standby node identification may be set to "a". When a certain standby plug-in node is selected as the main plug-in node, the identifier of the plug-in node is correspondingly modified from the standby node identifier to the main node identifier.
Further, when the management instruction is an add instruction, step S30 includes:
and broadcasting the notification message to a data acquisition module which has subscribed to the adding event corresponding to the adding instruction in the ZooKeeper, so that the data acquisition module determines the type of the added plug-in node after receiving the adding instruction, and adds the plug-in node of which the type is determined, wherein the added plug-in node comprises a main plug-in node or a standby plug-in node.
Further, the method for realizing the configurable plug-in further comprises the following steps:
and b, when detecting that the data acquisition module is started, generating a task plug-in list of the host corresponding to the data acquisition module in the ZooKeeper.
Further, after the data acquisition module is detected to be started, a task plug-in list of the host corresponding to the data acquisition module is generated in the ZooKeeper. It should be noted that the data acquisition module is installed in the host. Specifically, a task plug-in list is generated under the host namespace in ZooKeeper. Net is a form of code organization used by various languages, classified by name space, distinguishing different code functions, and is also part of the full name of all classes in vs. It should be noted that, in the ZooKeeper, the tasks to be executed are stored in advance, and after the data acquisition module is started, the corresponding task plug-in list can be generated through the preset timing task and the tasks to be executed, or the task plug-in list can be generated by manually adding the tasks by the operator corresponding to the ZooKeeper. Specifically, in the task plug-in list, the task that can be executed and the node name of the node that can execute the task plug-in are included. When the task needs to be executed, the corresponding task can be executed through the plug-in nodes in the task plug-in list.
In the embodiment, after the data acquisition module corresponding to the ZooKeeper is detected to be successfully registered, whether a management instruction for managing the plug-in node in the ZooKeeper is detected; when the management instruction is detected, generating a notification message according to the management instruction; the notification message is broadcasted to the data acquisition modules subscribed in the ZooKeeper and corresponding to the management events of the management instructions, so that the data acquisition modules execute the operation corresponding to the management instructions, and by setting the data acquisition modules, each data acquisition module can correspond to one or more plugin nodes and manage the plugin nodes, thereby realizing pluggable configuration of plugins of the data acquisition modules, realizing the operations of adding, deleting and the like of the plugin nodes under the condition of maintaining the whole ZooKeeper open source framework unchanged, namely realizing the operations of adding, deleting and the like of the plugins.
Further, a second embodiment of the method for implementing the configurable plug-in of the invention is proposed.
The difference between the second embodiment of the method for realizing the configurable plug-in and the first embodiment of the method for realizing the configurable plug-in is that, referring to fig. 4, the method for realizing the configurable plug-in further comprises:
in step S40, when a task scheduling instruction is detected, a task scheduling message is generated.
Step S50, broadcasting the task scheduling message to a target plug-in node subscribed to the task scheduling message, so that the target plug-in node executes a task corresponding to the task scheduling instruction.
And generating a task scheduling message after detecting the task scheduling instruction. The task scheduling instruction can be generated by a corresponding timing task at regular time, and can also be triggered manually by a corresponding operator through a configuration interface of a corresponding application program. It should be noted that, after the plug-in node of the data acquisition module successfully establishes a communication connection with the ZooKeeper, the data acquisition module may trigger a monitoring event, and the plug-in node of the data acquisition module subscribes to the task scheduling message generated by the ZooKeeper through the monitoring event. Therefore, after generating the task scheduling message, the ZooKeeper broadcasts the task scheduling message to the plug-in nodes subscribed to the task scheduling message, and marks the plug-in nodes subscribed to the task scheduling message as target plug-in nodes. And after the target plug-in node receives the task scheduling message, executing the task corresponding to the task scheduling instruction.
Further, in order to avoid resource waste caused by the fact that a plurality of plugin nodes execute the same task at the same time or mutual influence among the plurality of plugin nodes, the method for realizing plugin configuration further includes:
and c, determining a first target plug-in node which acquires the task scheduling message in the target plug-in nodes, and distributing the task corresponding to the task scheduling instruction to the first target plug-in node which acquires the task scheduling message.
And further, determining a first target plug-in node which acquires the task scheduling message in the target plug-in nodes, and distributing the task corresponding to the task scheduling instruction to the first target plug-in node which acquires the task scheduling message. It should be noted that, in the embodiment of the present invention, only the first target plugin node that acquires the task scheduling message may successfully execute the task corresponding to the task scheduling instruction, and all other target plugin nodes may not successfully execute the task corresponding to the task scheduling instruction, that is, only the first target plugin node that acquires the task scheduling message may acquire the relevant information for executing the task corresponding to the task scheduling instruction, and all other target plugin nodes may not acquire the relevant information for executing the task corresponding to the task scheduling instruction.
Further, the method for realizing the configurable plug-in further comprises the following steps:
and d, moving the task corresponding to the task scheduling instruction from a newly added task list to an executed task list so as to prohibit other target plug-in nodes except the first target plug-in node for acquiring the task scheduling message from acquiring the task corresponding to the task scheduling instruction from the newly added task list.
And after distributing the task corresponding to the task scheduling instruction to the first target plug-in node which acquires the task scheduling message, moving the task corresponding to the task scheduling instruction from the newly added task list to the execution task list so as to prohibit other target plug-in nodes except the first acquired task scheduling message from acquiring the task corresponding to the task scheduling instruction from the newly added task list. It should be noted that, in the embodiment of the present invention, a new task list, an execution task list, and a completion task list are created. And the newly added task list is used for storing the tasks distributed by the ZooKeeper after the ZooKeeper distributes the tasks, and each plug-in node can acquire the tasks from the newly added task list. When a certain task exists in the executed task list, indicating that the task is in a state of being executed; when a task exists in the completed task list, it indicates that the task has been performed. It can be understood that when a task is moved from the newly added task list to the executed task list, it indicates that the task cannot be re-acquired by other plug-in nodes, so that multiple plug-in nodes are prevented from executing the same task, and flexible task scheduling is realized.
Further, a third embodiment of the method for implementing the plug-in configurable method of the present invention is presented.
The third embodiment of the method for implementing plug-in configuration differs from the first or second embodiment of the method for implementing plug-in configuration in that, with reference to fig. 5, the method for implementing plug-in configuration further includes:
step S60, after receiving a registration request triggered when the data acquisition module is started, checking whether there is only one data acquisition module in the running state in the host corresponding to the data acquisition module, and detecting whether the data acquisition module in the running state establishes a communication connection with the ZooKeeper.
After receiving a registration request triggered when the data acquisition module is started, checking whether only one data acquisition module in the running state in the host corresponding to the data acquisition module is available, and detecting whether the data acquisition module in the running state establishes communication connection with the ZooKeeper. It should be noted that, in the host, there is at least one data acquisition module. When a plurality of data acquisition modules exist in the host, in order to avoid disordered operation of the plurality of data acquisition modules, only one data acquisition module in the host can be in an operation state at a time. In this embodiment, whether the data acquisition module is in the operating state can be determined by the status identifier of each data acquisition module in the host, and then whether only one data acquisition module in the operating state is in the host is determined. If the available state identifier "1" indicates that the data acquisition module is in the running state, and the state identifier "0" indicates that the data acquisition module is in the pause state. Therefore, when only one data acquisition module carrying the state identifier of the running state exists in the host, only one data acquisition module in the running state in the host can be determined.
Specifically, when the data acquisition module in the running state and the ZooKeeper can mutually transmit a message, it indicates that the data acquisition module in the running state and the ZooKeeper establish a communication connection, and at this time, it indicates that the ZooKeeper passes through self-checking.
Step S70, if there is only one data acquisition module in the host in the running state, and the data acquisition module in the running state establishes a communication connection with the ZooKeeper, the registration request is executed.
And if only one data acquisition module in the running state is arranged in the host, and the data acquisition module in the running state establishes communication connection with the ZooKeeper, executing the registration request. The registration request carries an IP (Internet Protocol, Protocol for interconnection between networks) address of the host and a module identifier of the data acquisition module in a running state. In the process of executing the registration request, the registration request is executed according to the IP address and the module identification of the host. Such as storing the IP address and module identification in a specific database, or adding a specific identification to the IP address and module identification to complete the registration operation. In this embodiment, the specific representation form of the module identifier is not limited, and a unique module identifier exists in each data acquisition module.
Further, if more than one data acquisition module in the running state in the host computer and/or the data acquisition module in the running state does not establish communication connection with the ZooKeeper, the execution of the registration request is refused.
Further, the method for realizing the configurable plug-in further comprises the following steps:
and e, creating a node list of the host and a node list of the data acquisition module in a running state under the root directory of the registration namespace of the ZooKeeper.
Further, after executing the registration request, a node list of the host and a node list of the data acquisition module in a running state are created under a root directory of a registration namespace of the ZooKeeper, so that the running state of the host is recorded through the node list of the host, and the running state of the data acquisition module is recorded through the node list of the data acquisition module. It should be noted that, after the registration operation of the registration request is completed, the operator corresponding to the ZooKeeper may perform the operation of the host resource through the configuration interface corresponding to the application program, for example, start the data acquisition module in the host or log out the data acquisition module in the host, thereby implementing the dynamic lateral expansion of the host resource.
In this embodiment, dynamic lateral capacity expansion of host resources is achieved by executing a registration request triggered when a data acquisition module in a host is started.
In addition, an embodiment of the present invention further provides an apparatus for implementing configurable plug-ins, where the apparatus for implementing configurable plug-ins includes:
the detection module is used for detecting whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not after the registration of the data acquisition module corresponding to the ZooKeeper is detected to be successful, wherein one data acquisition module corresponds to one or more plugin nodes;
the generating module is used for generating a notification message according to the management instruction after the management instruction is detected;
and the broadcasting module is used for broadcasting the notification message to a data acquisition module which has subscribed the management event corresponding to the management instruction in the ZooKeeper so that the data acquisition module can execute the operation corresponding to the management instruction.
Further, the broadcasting module is further configured to broadcast the notification message to a data acquisition module that has subscribed to the deletion event corresponding to the deletion instruction in the ZooKeeper, so that after the data acquisition module receives the deletion instruction, if it is detected that the plugin node corresponding to the deletion instruction is a main plugin node, after the main plugin node is logged out according to the deletion instruction, one plugin node is selected from standby plugin nodes as the main plugin node.
Furthermore, the generating module is further configured to generate a task scheduling message after detecting the task scheduling instruction;
the broadcast module is further configured to broadcast the task scheduling message to a target plug-in node that has subscribed to the task scheduling message, so that the target plug-in node executes a task corresponding to the task scheduling instruction.
Further, the apparatus for implementing plug-in configuration further comprises:
the determining module is used for determining a target plug-in node which first acquires the task scheduling message in the target plug-in nodes;
and the distribution module is used for distributing the task corresponding to the task scheduling instruction to the first target plug-in node which acquires the task scheduling message.
Further, the apparatus for implementing plug-in configuration further comprises:
and the moving module is used for moving the task corresponding to the task scheduling instruction from a newly added task list to an executed task list so as to prohibit other target plug-in nodes except the first target plug-in node for acquiring the task scheduling message from acquiring the task corresponding to the task scheduling instruction from the newly added task list.
Further, the apparatus for implementing plug-in configuration further comprises:
the checking module is used for checking whether only one data acquisition module in the running state in the host corresponding to the data acquisition module is available or not after receiving a registration request triggered when the data acquisition module is started, and detecting whether the data acquisition module in the running state is in communication connection with the ZooKeeper or not;
and the execution module is used for executing the registration request if only one data acquisition module in the running state is arranged in the host and the data acquisition module in the running state establishes communication connection with the ZooKeeper.
Further, the apparatus for implementing plug-in configuration further comprises:
and the creating module is used for creating a node list of the host and a node list of the data acquisition module in a running state under the root directory of the registration namespace of the ZooKeeper.
Further, the generating module is further configured to generate a task plug-in list of the host corresponding to the data acquisition module in the ZooKeeper after detecting that the data acquisition module is started.
The specific implementation of the device for implementing the configuration of the plug-in is basically the same as that of the above embodiments of the method for implementing the configuration of the plug-in, and is not described herein again.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, where a program for implementing plug-in configuration is stored, and when executed by a processor, the program for implementing plug-in configuration implements the steps of implementing the plug-in configurable method as described above.
The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the embodiments of the method for implementing the configurable plug-in, and is not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for realizing configurable plug-ins, characterized in that the method for realizing configurable plug-ins comprises the following steps:
when the fact that the data acquisition module corresponding to the ZooKeeper is successfully registered is detected, whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not is detected, wherein one data acquisition module corresponds to one or more plugin nodes;
when the management instruction is detected, generating a notification message according to the management instruction;
broadcasting the notification message to a data acquisition module which has subscribed to the management event corresponding to the management instruction in the ZooKeeper, so that the data acquisition module executes the operation corresponding to the management instruction;
when the management instruction is a delete instruction, the step of broadcasting the notification message to a data acquisition module, which has subscribed to a management event corresponding to the management instruction, in the ZooKeeper, so that the data acquisition module executes an operation corresponding to the management instruction includes:
broadcasting the notification message to a data acquisition module which has subscribed to the deletion event corresponding to the deletion instruction in the ZooKeeper, so that after the data acquisition module receives the deletion instruction, if a plug-in node corresponding to the deletion instruction is detected to be a main plug-in node, after the main plug-in node is cancelled according to the deletion instruction, one plug-in node is selected from standby plug-in nodes to serve as the main plug-in node;
or, when the management instruction is an add instruction, the step of broadcasting the notification message to a data acquisition module that has subscribed to a management event corresponding to the management instruction in the ZooKeeper, so that the data acquisition module executes an operation corresponding to the management instruction includes:
and broadcasting the notification message to a data acquisition module which has subscribed to the adding event corresponding to the adding instruction in the ZooKeeper, so that the data acquisition module determines the type of the added plug-in node after receiving the adding instruction, and adds the plug-in node of which the type is determined, wherein the added plug-in node comprises a main plug-in node or a standby plug-in node.
2. The method for implementing plug-in configuration as claimed in claim 1, wherein after the step of broadcasting the notification message to the data collection module in the ZooKeeper, which has subscribed to the management event corresponding to the management instruction, for the data collection module to perform the operation corresponding to the management instruction, the method further comprises:
generating a task scheduling message after detecting a task scheduling instruction;
and broadcasting the task scheduling message to a target plug-in node subscribed with the task scheduling message so that the target plug-in node can execute the task corresponding to the task scheduling instruction.
3. The method for implementing plug-in configuration as claimed in claim 2, wherein after the step of broadcasting the task scheduling message to the target plug-in node subscribed to the task scheduling message for the target plug-in node to execute the task corresponding to the task scheduling instruction, further comprising:
and determining a first target plug-in node in the target plug-in nodes, which acquires the task scheduling message, and distributing the task corresponding to the task scheduling instruction to the first target plug-in node, which acquires the task scheduling message.
4. The method as claimed in claim 3, wherein after the step of determining a first target plug-in node from among the target plug-in nodes that acquired the task scheduling message and distributing the task corresponding to the task scheduling instruction to the first target plug-in node that acquired the task scheduling message, the method further comprises:
and moving the task corresponding to the task scheduling instruction from a newly added task list to an executed task list so as to prohibit other target plug-in nodes except the first target plug-in node for acquiring the task scheduling message from acquiring the task corresponding to the task scheduling instruction from the newly added task list.
5. The method according to any one of claims 1 to 4, wherein before the step of detecting whether a management instruction for managing a plugin node in a ZooKeeper is detected after detecting that the data collection module in the ZooKeeper is successfully registered, the method further comprises:
after receiving a registration request triggered when the data acquisition module is started, checking whether only one data acquisition module in the running state in a host corresponding to the data acquisition module is available, and detecting whether the data acquisition module in the running state establishes communication connection with the ZooKeeper;
and if only one data acquisition module in the running state is arranged in the host, and the data acquisition module in the running state establishes communication connection with the ZooKeeper, executing the registration request.
6. The method according to claim 5, wherein if there is only one data collection module in the host in the running state and the data collection module in the running state establishes a communication connection with the ZooKeeper, the step of executing the registration request further includes:
and creating a node list of the host and a node list of the data acquisition module in a running state under a root directory of the registration namespace of the ZooKeeper.
7. The method according to any one of claims 1 to 4, wherein before the step of detecting whether a management instruction for managing a plugin node in a ZooKeeper is detected after detecting that the registration of the data acquisition module corresponding to the ZooKeeper is successful, the method further comprises:
and when the data acquisition module is detected to be started, generating a task plug-in list of the host corresponding to the data acquisition module in the ZooKeeper.
8. An apparatus for implementing configurable plug-ins, the apparatus comprising:
the detection module is used for detecting whether a management instruction for managing the plugin nodes in the ZooKeeper is detected or not after the registration of the data acquisition module corresponding to the ZooKeeper is detected to be successful, wherein one data acquisition module corresponds to one or more plugin nodes;
the generating module is used for generating a notification message according to the management instruction after the management instruction is detected;
a broadcasting module, configured to broadcast the notification message to a data acquisition module that has subscribed to the management event corresponding to the management instruction in the ZooKeeper, so that the data acquisition module executes an operation corresponding to the management instruction;
the broadcasting module is further configured to broadcast the notification message to a data acquisition module that has subscribed to a deletion event corresponding to a deletion instruction in the ZooKeeper, so that after the data acquisition module receives the deletion instruction, if it is detected that a plugin node corresponding to the deletion instruction is a master plugin node, after the master plugin node is cancelled according to the deletion instruction, one plugin node is selected from standby plugin nodes as the master plugin node; and broadcasting the notification message to a data acquisition module which has subscribed to the addition instruction and corresponds to the addition event in the ZooKeeper, so that the data acquisition module determines the type of the added plug-in node after receiving the addition instruction, and adds the plug-in node of which the type is determined, wherein the added plug-in node comprises a main plug-in node or a standby plug-in node.
9. An apparatus for implementing plug-in configurability, comprising a memory, a processor, and a program for implementing plug-in configurability stored on the memory and executable on the processor, the program for implementing plug-in configurability implementing steps of the method for implementing plug-in configurability according to any one of claims 1 to 7 when executed by the processor.
10. A computer-readable storage medium having stored thereon a program for implementing a plug-in configurable, the program for implementing a plug-in configurable implementing the steps of implementing a plug-in configurable method according to any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811448379.7A CN109614164B (en) | 2018-11-29 | 2018-11-29 | Method, device and equipment for realizing configurable plug-in and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811448379.7A CN109614164B (en) | 2018-11-29 | 2018-11-29 | Method, device and equipment for realizing configurable plug-in and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614164A CN109614164A (en) | 2019-04-12 |
| CN109614164B true CN109614164B (en) | 2021-09-28 |
Family
ID=66004899
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811448379.7A Active CN109614164B (en) | 2018-11-29 | 2018-11-29 | Method, device and equipment for realizing configurable plug-in and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614164B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109992531B (en) * | 2019-04-15 | 2020-11-10 | 成都四方伟业软件股份有限公司 | Data storage method and device |
| CN110069295B (en) * | 2019-05-06 | 2022-06-17 | 百度在线网络技术（北京）有限公司 | Block chain processing method, device, equipment and medium |
| CN111458561B (en) * | 2020-03-27 | 2022-07-29 | 深圳渊联技术有限公司 | Equipment sampling device, configuration and management method, management server and management system |
| CN113010531B (en) * | 2021-02-05 | 2022-11-01 | 成都库珀创新科技有限公司 | Block chain BAAS system task scheduling framework based on directed acyclic graph |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103353861B (en) * | 2013-06-18 | 2016-06-29 | 中国科学院计算技术研究所 | Realize method and the device of distributed I/O resource pool |
| CN103729436A (en) * | 2013-12-27 | 2014-04-16 | 中国科学院信息工程研究所 | Distributed metadata management method and system |
| US9411534B2 (en) * | 2014-07-02 | 2016-08-09 | Hedvig, Inc. | Time stamp generation for virtual disks |
| CN105656653B (en) * | 2014-11-14 | 2019-07-19 | 华为技术有限公司 | Increase method of network entry, the device and system of node in distributed coordination system newly |
| CN104618142B (en) * | 2014-12-30 | 2017-07-07 | 北京奇安信科技有限公司 | News subscribing method, treatment node device and bus apparatus |
| CN108737543B (en) * | 2018-05-21 | 2021-09-24 | 高新兴智联科技有限公司 | Distributed Internet of things middleware and working method |
-
2018
- 2018-11-29 CN CN201811448379.7A patent/CN109614164B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614164A (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109614164B (en) | Method, device and equipment for realizing configurable plug-in and readable storage medium | |
| CN108052675B (en) | Log management method, system and computer readable storage medium | |
| CN108365993B (en) | Block link point dynamic changing method, system and computer readable storage medium | |
| US10162688B2 (en) | Processing event messages for user requests to execute program code | |
| KR20200012981A (en) | Network slice management methods, devices, and computer readable storage media | |
| US10129264B2 (en) | Method and apparatus for implementing document sharing between user groups | |
| CN111026635B (en) | Software project testing system, method, device and storage medium | |
| CN110413346B (en) | Parameter updating method and device | |
| CN109672580B (en) | Full link monitoring method, device, terminal equipment and storage medium | |
| CN111130825B (en) | Network slice template generation method, device, equipment and storage medium | |
| US11119911B2 (en) | Garbage collection method and device | |
| JP2018527683A (en) | Method and apparatus for loading terminal application | |
| EP3249871A1 (en) | Method and device for updating network service descriptor | |
| CN111431740A (en) | Data transmission method, device, equipment and computer readable storage medium | |
| CN113220431B (en) | Cross-cloud distributed data task scheduling method, device and storage medium | |
| CN109618176B (en) | Processing method, equipment and storage medium for live broadcast service | |
| CN105760240A (en) | Distributed task processing method and device | |
| CN111083219A (en) | Request processing method, device, equipment and computer readable storage medium | |
| WO2017204699A1 (en) | Automatic network management system, method and computer program product | |
| CN108062244B (en) | Reptile task canceling method and device | |
| CN106936660B (en) | Data acquisition method and device | |
| CN108132836B (en) | Task distribution method and device and electronic equipment | |
| WO2017088347A1 (en) | Method, device and system for managing user usage information of application based on paas platform | |
| CN111240998A (en) | Test case processing method and device | |
| CN112685102A (en) | Gateway plug-in hot loading method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |