CN113626047B - FLUME automatic deployment system and method - Google Patents
FLUME automatic deployment system and method Download PDFInfo
- Publication number
- CN113626047B CN113626047B CN202110994424.4A CN202110994424A CN113626047B CN 113626047 B CN113626047 B CN 113626047B CN 202110994424 A CN202110994424 A CN 202110994424A CN 113626047 B CN113626047 B CN 113626047B
- Authority
- CN
- China
- Prior art keywords
- flume
- deployed
- server
- user
- automatic deployment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000013515 script Methods 0.000 claims abstract description 67
- 238000012544 monitoring process Methods 0.000 claims abstract description 22
- 238000012423 maintenance Methods 0.000 claims abstract description 18
- 238000009434 installation Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 8
- 238000007726 management method Methods 0.000 description 20
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 3
- IXJYMUFPNFFKIB-FMONCPFKSA-N pomp protocol Chemical compound S=C1N=CNC2=C1NC=N2.O=C1C=C[C@]2(C)[C@H]3C(=O)C[C@](C)([C@@](CC4)(O)C(=O)CO)[C@@H]4[C@@H]3CCC2=C1.C=1N=C2N=C(N)N=C(N)C2=NC=1CN(C)C1=CC=C(C(=O)N[C@@H](CCC(O)=O)C(O)=O)C=C1.C([C@H](C[C@]1(C(=O)OC)C=2C(=C3C([C@]45[C@H]([C@@]([C@H](OC(C)=O)[C@]6(CC)C=CCN([C@H]56)CC4)(O)C(=O)OC)N3C=O)=CC=2)OC)C[C@@](C2)(O)CC)N2CCC2=C1NC1=CC=CC=C21 IXJYMUFPNFFKIB-FMONCPFKSA-N 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 210000001503 joint Anatomy 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 239000002071 nanotube Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
Abstract
The invention provides a FLUME automatic deployment system and a method, which can be used in the distributed technical field, wherein the system comprises: the log monitoring subsystem comprises an operation and maintenance server module and is used for generating FLUME configuration files according to FLUME log acquisition rules configured by a user; the automatic deployment subsystem comprises: the server management module is used for receiving the information of the plurality of servers to be deployed imported by the user, and the environment management module is used for receiving the information of the plurality of environments to be deployed newly built by the user; the script management module is used for receiving FLUME automatic deployment scripts configured by a user; and after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script. The invention can realize FLUME automatic deployment on a plurality of target servers.
Description
Technical Field
The invention relates to the field of distributed technology, in particular to a FLUME automatic deployment system and method.
Background
A log monitoring system is generally composed of several key modules of log collection, analysis processing, index calculation and visual display, wherein the log collection is commonly used FLUME as a collection tool. When the product is accessed, FLUME is required to be deployed on a monitoring server of the log to be acquired, and the customized configuration of the configuration file is performed. In recent years, as more and more products are turned to a distributed micro-service architecture, the number of deployed nodes is greatly increased, and the workload of the traditional manual deployment FLUME for collecting tools and configuring one by one is too great, so that the requirements of efficient operation and maintenance cannot be met.
Disclosure of Invention
The embodiment of the invention provides a FLUME automatic deployment system for realizing automatic deployment of FLUME on a plurality of target servers, which comprises the following steps:
a log monitoring subsystem and an automatic deployment subsystem, wherein,
The log monitoring subsystem comprises an operation and maintenance server side module, wherein the operation and maintenance server side module is used for generating FLUME configuration files according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule binds a group of server information to be deployed;
the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein,
A server management module for receiving a plurality of server information to be deployed imported by a user,
The environment management module is used for receiving a plurality of pieces of environment information to be deployed newly created by a user, wherein each environment to be deployed is used for managing a group of servers to be deployed;
The script management module is used for receiving FLUME automatic deployment scripts configured by a user, wherein the FLUME automatic deployment scripts comprise a version of a FLUME installation package and server information to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, and when the FLUME automatic deployment script is executed, automatically deploying the script by pulling FLUME the installation package and the corresponding FLUME configuration file according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed.
The embodiment of the invention provides a FLUME automatic deployment method for realizing automatic deployment of FLUME on a plurality of target servers, which comprises the following steps:
receiving a plurality of server information to be deployed imported by a user,
Receiving information of a plurality of environments to be deployed newly created by a user, wherein each environment to be deployed is used for managing a group of servers to be deployed;
Generating FLUME configuration files according to FLUME log collection rules configured by a user, wherein each FLUME log collection rule binds a group of server information to be deployed;
receiving FLUME automatic deployment scripts configured by a user, wherein the FLUME automatic deployment scripts comprise a version of a FLUME installation package and server information to be deployed;
After receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, and when the FLUME automatic deployment script is executed, automatically deploying the script by pulling FLUME the installation package and the corresponding FLUME configuration file according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the automatic deployment method of FLUME is realized when the processor executes the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the automatic deployment method FLUME.
In the embodiment of the invention, the log monitoring subsystem comprises an operation and maintenance server module, wherein the operation and maintenance server module is used for generating FLUME configuration files according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule is bound with a group of server information to be deployed; the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein the server management module is used for receiving a plurality of server information to be deployed imported by a user, and the environment management module is used for receiving a plurality of environment information to be deployed newly built by the user, wherein each environment to be deployed is used for managing a group of servers to be deployed; the script management module is used for receiving FLUME automatic deployment scripts configured by a user, wherein the FLUME automatic deployment scripts comprise a version of a FLUME installation package and server information to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, and when the FLUME automatic deployment script is executed, automatically deploying the script by pulling FLUME the installation package and the corresponding FLUME configuration file according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed. The system and the method provided by the embodiment of the invention have the following beneficial effects:
firstly, automatic deployment of FLUME is realized, installation and configuration of FLUME can be completed on a plurality of servers in parallel by one key, and manual operation cost is reduced;
secondly, friendly FLUME log collection rule configuration is provided, a user does not need to learn and understand complex fields in FLUME configuration files, FLUME configuration files are automatically generated by the background according to FLUME log collection rules, and the use threshold is reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a schematic diagram of a FLUME auto-deploy system in an embodiment of the invention;
FIG. 2 is a schematic diagram of a FLUME auto-deploy system in an embodiment of the invention;
FIG. 3 is another schematic diagram of a FLUME auto-deploy system in an embodiment of the invention;
FIG. 4 is a detailed flow chart of FLUME automatic deployment in an embodiment of the present invention;
FIG. 5 is a flow chart of a method for automatic deployment FLUME in an embodiment of the present invention;
Fig. 6 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are open-ended terms, meaning including, but not limited to. The description of the reference terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The order of steps involved in the embodiments is illustrative of the practice of the application, and is not limited and may be suitably modified as desired.
The terms involved in the present invention will be explained first.
FLUME: FLUME is a highly available, highly reliable distributed mass log collection, aggregation, transmission system provided by Cloudera.
Kafka: and the message buffer queue is used for putting the message into the queue first when the data volume is large, delaying for a certain time to process, and avoiding the collapse of subsequent processing logic.
POMP: the enterprise self-research log monitoring subsystem collects logs from servers of all products by FLUME, and visually displays indexes such as transaction amount, success rate, response time and the like of the products through log analysis and calculation processing to monitor the running state of the products in real time.
The product is as follows: enterprise-specific concepts refer to a system that provides a function or service to the outside. One product typically contains multiple servers (multiple services corresponding to the running state).
SADP-X86: the enterprise self-research automatic deployment platform suitable for the X86 architecture provides an on-line execution function of a self-defined script and a job, can be used for automatically issuing FLUME acquisition tools and carrying out customized configuration on FLUME configuration files.
FIG. 1 is a schematic diagram of a system for automatic deployment FLUME in an embodiment of the present invention, as shown in FIG. 1, the system includes: a log monitoring subsystem 101 and an automatic deployment subsystem 102, wherein,
The log monitoring subsystem 101 comprises an operation and maintenance server side module 1011, wherein the operation and maintenance server side module 1011 is used for generating FLUME configuration files according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule is bound with a group of server information to be deployed;
the automatic deployment subsystem 102 includes a server management module 1021, an environment management module 1021, and a script management module 1023, wherein,
A server management module 1021 for receiving a plurality of server information to be deployed imported by a user,
The environment management module 1022 is configured to receive a plurality of environment information to be deployed newly created by a user, where each environment to be deployed is configured to nanotube a group of servers to be deployed;
The script management module 1023 is configured to receive FLUME automatic deployment scripts configured by a user, where the FLUME automatic deployment scripts include a version of a FLUME installation package and server information to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, and when the FLUME automatic deployment script is executed, automatically deploying the script by pulling FLUME the installation package and the corresponding FLUME configuration file according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed.
In specific implementation, the server management module is configured to receive information of multiple servers to be deployed imported by a user, where the servers to be deployed are all target products. After receiving the script execution instruction of the user, the script management module issues the FLUME automatic deployment scripts to the corresponding servers to be deployed and executes the scripts, and as can be seen, the embodiment of the invention supports FLUME one-key execution of the automatic deployment scripts, wherein if a plurality of corresponding servers to be deployed exist, the execution is performed in parallel.
FIG. 2 is a schematic diagram of a FLUME automatic deployment system in an embodiment of the present invention, where a log monitoring subsystem may be POMP-based, and an automatic deployment subsystem may be SADP-X86-based, where the system provided in the embodiment of the present invention fully utilizes the CMDB capabilities of an existing enterprise platform (and SADP-X86), and the functions of the modules in the log monitoring subsystem and the automatic deployment subsystem are implemented on the platform basis, without repeated development, and without repeated linkage and docking between the implementation subsystems, and provides a complete use flow and solution, thereby reducing development costs.
FIG. 3 is another schematic diagram of a system for automated deployment FLUME in an embodiment of the invention, in which the log monitoring subsystem further includes an operation and maintenance presentation end module 1012 for receiving a plurality of FLUME log collection rules configured by a user via a page.
In the above embodiment, the operation and maintenance display module includes a navigation page for guiding the user to configure FLUME the log collection rule, and the user can conveniently configure the operation and maintenance display module based on the navigation page, where one FLUME log collection rule includes:
(1) Basic information such as rule name and belonged product
(2) Log acquisition source information: multiple journal collection paths may be created under one collection rule, and the delivery of journals under each path to the topic of Kakfa.
(3) Collecting advanced option configuration: if so, whether directory-level regular expressions are contained, whether stack log multi-line merge mode is used, log file cleaning policies, etc.
(4) FLUME log collection rule binding server information to be deployed: a set of servers is selected on which to use the current FLUME log collection rules. When selecting a server, the log monitoring subsystem queries a server list under the environment information to be deployed by calling a server query interface of the SADP-X86 automatic deployment subsystem for the user to select.
In one embodiment, the system further includes an FTP server module 103 for storing FLUME the configuration file and different versions of FLUME installation packages;
the operation and maintenance server module is further used for: and publishing FLUME the configuration file to the FTP server module.
In one embodiment, the server information includes an IP of a server;
The FTP server module is specifically configured to: and storing FLUME the configuration file according to the IP address book of the server.
In the above embodiment, if a plurality FLUME of log collection rules bind to one server, the FLUME configuration files are combined.
In an embodiment, the automatic deployment subsystem further includes a server query interface 1024, configured to display the configured environment information to be deployed after receiving a server query instruction of the user, and receive a set of server information to be deployed in one environment information to be deployed selected by the user;
The operation and maintenance display end module is also used for: and sending a server query instruction to a server query interface to obtain a group of server information to be deployed, and receiving the binding of the group of server information to be deployed and FLUME log acquisition rules by a user.
In one embodiment, the log monitoring subsystem further comprises Kafka1013;
After being started, FLUME delivers the logs collected based on FLUME log collection rules into Kafka.
In the above embodiment, if FLUME receives the collected log in Kafka after startup, FLUME deployment is verified to be successful, and delivery is classified according to topic.
Based on fig. 2, a detailed flowchart of automatic FLUME deployment is given in the embodiment of the present invention, see fig. 4, and the detailed steps include:
step 401, a user imports a server to be deployed of a target product in an SADP-X86 automatic deployment subsystem;
step 402, a user establishes a to-be-deployed environment in an SADP-X86 automatic deployment subsystem, and adds a group of to-be-deployed servers to the to-be-deployed environment;
Step 403, the user creates FLUME acquisition rules in POMP log monitoring subsystem and binds a group of server information to be deployed; after FLUME collection rule configuration is completed, when a user clicks "release", the log monitoring Ping Zi system generates FLUME configuration files according to FLUME log collection rules configured by the user, wherein each FLUME log collection rule binds a group of server information to be deployed, that is, each server IP corresponds to a FLUME configuration file, and uploads the configuration files to a corresponding directory of the FTP server module.
Step 404, configuring FLUME an automatic deployment script in the SADP-X86 automatic deployment subsystem by a user;
In step 405, the user issues a script execution instruction to the SADP-X86 automatic deployment subsystem, and the SADP-X86 automatic deployment subsystem will automatically pull FLUME the installation package and the corresponding FLUME configuration file from the FTP server module to deploy, and start FLUME after the deployment is completed.
In summary, the beneficial effects of the system provided in the embodiment of the invention are as follows:
firstly, automatic deployment of FLUME is realized, installation and configuration of FLUME can be completed on a plurality of servers in parallel by one key, and manual operation cost is reduced;
secondly, friendly FLUME log collection rule configuration is provided, a user does not need to learn and understand complex fields in FLUME configuration files, FLUME configuration files are automatically generated by the background according to FLUME log collection rules, and the use threshold is reduced.
Thirdly, the system provided by the embodiment of the invention fully utilizes the CMDB capability of the existing platform (and SADP-X86) of the enterprise, the module functions of the log monitoring subsystem and the automatic deployment subsystem are realized on the basis of the platform, repeated development is not needed, linkage and butt joint among the subsystems are realized, a complete use flow and a complete solution are provided, and the development cost is reduced.
The embodiment of the invention also provides a FLUME automatic deployment method, the principle of which is similar to that of the FLUME automatic deployment system, and the description is omitted here.
Fig. 5 is a flowchart of a method for automatic deployment FLUME in an embodiment of the present invention, which specifically includes:
step 501, receiving a plurality of server information to be deployed imported by a user;
Step 502, receiving a plurality of environment information to be deployed newly created by a user, wherein each environment to be deployed is used for managing a group of servers to be deployed;
Step 503, generating FLUME configuration files according to FLUME log collection rules configured by a user, wherein each FLUME log collection rule binds a group of server information to be deployed;
Step 504, receiving FLUME automatic deployment scripts configured by a user, wherein the FLUME automatic deployment scripts comprise a version of a FLUME installation package and server information to be deployed;
Step 505, after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, when the FLUME automatic deployment script is executed, automatically deploying the installation package and a corresponding FLUME configuration file from a pull FLUME according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed.
In an embodiment, the method further comprises:
A plurality FLUME of log collection rules configured by a user are received.
In an embodiment, the method further comprises:
Store FLUME the configuration file and different versions of FLUME installation packages.
In an embodiment, the method further comprises:
and storing FLUME the configuration file according to the IP address book of the server.
In an embodiment, the method further comprises:
after receiving a server query instruction of a user, displaying configured environment information to be deployed;
receiving a group of server information to be deployed in one environment information to be deployed selected by a user;
And receiving the binding of the user to the group of server information to be deployed and FLUME log acquisition rules.
In one embodiment, FLUME delivers the log collected based on FLUME log collection rules to Kafka after startup.
In summary, the method provided in the embodiment of the invention has the following beneficial effects:
firstly, automatic deployment of FLUME is realized, installation and configuration of FLUME can be completed on a plurality of servers in parallel by one key, and manual operation cost is reduced;
secondly, friendly FLUME log collection rule configuration is provided, a user does not need to learn and understand complex fields in FLUME configuration files, FLUME configuration files are automatically generated by the background according to FLUME log collection rules, and the use threshold is reduced.
Thirdly, the system provided by the embodiment of the invention fully utilizes the CMDB capability of the existing platform (and SADP-X86) of the enterprise, the module functions of the log monitoring subsystem and the automatic deployment subsystem are realized on the basis of the platform, repeated development is not needed, linkage and butt joint among the subsystems are realized, a complete use flow and a complete solution are provided, and the development cost is reduced.
An embodiment of the present invention further provides a computer device, and fig. 6 is a schematic diagram of a computer device in the embodiment of the present invention, where the computer device is capable of implementing all the steps in the method for automatic FLUME deployment in the foregoing embodiment, and the computer device specifically includes the following contents:
A processor 601, a memory 602, a communication interface (Communications Interface) 603, and a communication bus 604;
wherein the processor 601, the memory 602, and the communication interface 603 complete communication with each other through the communication bus 604; the communication interface 603 is configured to implement information transmission between related devices such as a server device, a detection device, and a user device;
The processor 601 is configured to invoke a computer program in the memory 602, where execution of the computer program by the processor implements all the steps in the method for automatic deployment FLUME in the above embodiment.
Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which when executed by a processor, implements all the steps of the method for automatic deployment of FLUME in the above embodiments.
Those skilled in the art will appreciate that embodiments of the invention may be provided as a method, system, or computer program business system. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program business systems according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. A system for automated deployment FLUME comprising: a log monitoring subsystem and an automatic deployment subsystem, wherein,
The log monitoring subsystem comprises an operation and maintenance server side module, wherein the operation and maintenance server side module is used for generating FLUME configuration files according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule binds a group of server information to be deployed;
the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein,
The server management module is used for receiving the information of the plurality of servers to be deployed imported by the user;
The environment management module is used for receiving a plurality of pieces of environment information to be deployed newly created by a user, wherein each environment to be deployed is used for managing a group of servers to be deployed;
The script management module is used for receiving FLUME automatic deployment scripts configured by a user, wherein the FLUME automatic deployment scripts comprise a version of a FLUME installation package and server information to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, wherein when the FLUME automatic deployment script is executed, the automatic deployment is performed by automatically pulling FLUME the installation package and a corresponding FLUME configuration file according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed;
The automatic deployment subsystem further comprises a server query interface, a server query interface and a server deployment interface, wherein the server query interface is used for displaying configured environment information to be deployed after receiving a server query instruction of a user and receiving a group of server information to be deployed in one environment information to be deployed selected by the user;
The operation and maintenance display end module is also used for: a server query instruction is sent to a server query interface to obtain a group of server information to be deployed, and binding of a user to the group of server information to be deployed and FLUME log acquisition rules is received;
the log monitoring subsystem further comprises Kafka;
After being started, FLUME delivers the logs collected based on FLUME log collection rules into Kafka.
2. The automated deployed system of claim FLUME, wherein the journal monitoring subsystem further comprises an operations and maintenance presentation module that receives the user-configured plurality of FLUME journal gathering rules via pages.
3. The system for automatic deployment of FLUME of claim 1, further comprising an FTP server module for storing FLUME configuration files and different versions of FLUME installation packages;
the operation and maintenance server module is further used for: and publishing FLUME the configuration file to the FTP server module.
4. The system for automatic deployment of FLUME of claim 3, wherein said server information comprises IP of a server;
The FTP server module is specifically configured to: and storing FLUME the configuration file according to the IP address book of the server.
5. A method of FLUME automatic deployment, comprising:
receiving a plurality of server information to be deployed imported by a user,
Receiving information of a plurality of environments to be deployed newly created by a user, wherein each environment to be deployed is used for managing a group of servers to be deployed;
Generating FLUME configuration files according to FLUME log collection rules configured by a user, wherein each FLUME log collection rule binds a group of server information to be deployed;
receiving FLUME automatic deployment scripts configured by a user, wherein the FLUME automatic deployment scripts comprise a version of a FLUME installation package and server information to be deployed;
After receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the script, wherein when the FLUME automatic deployment script is executed, the automatic deployment is performed by automatically pulling FLUME the installation package and a corresponding FLUME configuration file according to the version of the FLUME installation package and the information of the server to be deployed, and starting FLUME after the deployment is completed;
The method further comprises the steps of: after receiving a server query instruction of a user, displaying configured environment information to be deployed; receiving a group of server information to be deployed in one environment information to be deployed selected by a user; binding the group of server information to be deployed by a user and FLUME log acquisition rules is received;
After being started, FLUME delivers the logs collected based on FLUME log collection rules into Kafka.
6. The method for automated deployment of FLUME, further comprising:
A plurality FLUME of log collection rules configured by a user are received.
7. The method for automated deployment of FLUME, further comprising:
Store FLUME the configuration file and different versions of FLUME installation packages.
8. The method for automated deployment of FLUME of claim 7, further comprising:
and storing FLUME the configuration file according to the IP address book of the server.
9.A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 5 to 8 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program for executing the method of any one of claims 5 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994424.4A CN113626047B (en) | 2021-08-27 | 2021-08-27 | FLUME automatic deployment system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994424.4A CN113626047B (en) | 2021-08-27 | 2021-08-27 | FLUME automatic deployment system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113626047A CN113626047A (en) | 2021-11-09 |
| CN113626047B true CN113626047B (en) | 2024-04-19 |
Family
ID=78388053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110994424.4A Active CN113626047B (en) | 2021-08-27 | 2021-08-27 | FLUME automatic deployment system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626047B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150291A (en) * | 2022-07-05 | 2022-10-04 | 中国银行股份有限公司 | Operation and maintenance system and method based on event driving |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9578063B1 (en) * | 2015-11-20 | 2017-02-21 | International Business Machines Corporation | Application self-service for assured log management in cloud environments |
| CN107908414A (en) * | 2017-11-22 | 2018-04-13 | 中国银行股份有限公司 | A kind of method, apparatus and system of the deployment of application server version |
| CN111580832A (en) * | 2020-04-29 | 2020-08-25 | 电科云(北京)科技有限公司 | Application release system and method applied to heterogeneous multi-cloud environment |
| CN112035124A (en) * | 2020-09-03 | 2020-12-04 | 中国银行股份有限公司 | Application deployment method and device |
-
2021
- 2021-08-27 CN CN202110994424.4A patent/CN113626047B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9578063B1 (en) * | 2015-11-20 | 2017-02-21 | International Business Machines Corporation | Application self-service for assured log management in cloud environments |
| CN107908414A (en) * | 2017-11-22 | 2018-04-13 | 中国银行股份有限公司 | A kind of method, apparatus and system of the deployment of application server version |
| CN111580832A (en) * | 2020-04-29 | 2020-08-25 | 电科云(北京)科技有限公司 | Application release system and method applied to heterogeneous multi-cloud environment |
| CN112035124A (en) * | 2020-09-03 | 2020-12-04 | 中国银行股份有限公司 | Application deployment method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113626047A (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110535831B (en) | Kubernetes and network domain-based cluster security management method and device and storage medium | |
| CN112118565B (en) | Multi-tenant service gray level publishing method, device, computer equipment and storage medium | |
| US10171377B2 (en) | Orchestrating computing resources between different computing environments | |
| US11177999B2 (en) | Correlating computing network events | |
| US8478800B1 (en) | Log streaming facilities for computing applications | |
| US20160124793A1 (en) | Log analytics for problem diagnosis | |
| CN111752799A (en) | Service link tracking method, device, equipment and storage medium | |
| CN113987074A (en) | Distributed service full-link monitoring method and device, electronic equipment and storage medium | |
| US20190109920A1 (en) | Browser resource pre-pulling method, terminal and storage medium | |
| CN107766509B (en) | Method and device for static backup of webpage | |
| US20210096981A1 (en) | Identifying differences in resource usage across different versions of a software application | |
| CN108234245A (en) | The screening technique of log content and daily record data, device, system, readable medium | |
| CN107809383A (en) | A kind of map paths method and device based on MVC | |
| US11777836B2 (en) | Protocol and state analysis in a dynamic routing network | |
| CN113626047B (en) | FLUME automatic deployment system and method | |
| CN111190805A (en) | Training log visualization system, method and equipment of deep learning training platform | |
| CN108920295A (en) | The processing method of system exception, apparatus and system | |
| CN115794064B (en) | Configuration method and device of task processing flow, electronic equipment and storage medium | |
| CN106202470B (en) | The processing method and system of script | |
| US9298539B1 (en) | Automated error recovery for workflows | |
| CN112825525B (en) | Method and apparatus for processing transactions | |
| US8214846B1 (en) | Method and system for threshold management | |
| CN109101636A (en) | A kind of method, apparatus and system carrying out data acquisition in cloud by visual configuration | |
| CN115221529B (en) | Method and system for injecting abnormity of front-end webpage | |
| CN115203165A (en) | Block chain management method, device, server and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |