CN113626047A - System and method for FLUME automatic deployment - Google Patents
System and method for FLUME automatic deployment Download PDFInfo
- Publication number
- CN113626047A CN113626047A CN202110994424.4A CN202110994424A CN113626047A CN 113626047 A CN113626047 A CN 113626047A CN 202110994424 A CN202110994424 A CN 202110994424A CN 113626047 A CN113626047 A CN 113626047A
- Authority
- CN
- China
- Prior art keywords
- flume
- server
- deployed
- user
- deployment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013515 script Methods 0.000 claims abstract description 65
- 238000012544 monitoring process Methods 0.000 claims abstract description 23
- 238000012423 maintenance Methods 0.000 claims abstract description 17
- 238000009434 installation Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 18
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000007726 management method Methods 0.000 description 25
- 238000010586 diagram Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 3
- IXJYMUFPNFFKIB-FMONCPFKSA-N pomp protocol Chemical compound S=C1N=CNC2=C1NC=N2.O=C1C=C[C@]2(C)[C@H]3C(=O)C[C@](C)([C@@](CC4)(O)C(=O)CO)[C@@H]4[C@@H]3CCC2=C1.C=1N=C2N=C(N)N=C(N)C2=NC=1CN(C)C1=CC=C(C(=O)N[C@@H](CCC(O)=O)C(O)=O)C=C1.C([C@H](C[C@]1(C(=O)OC)C=2C(=C3C([C@]45[C@H]([C@@]([C@H](OC(C)=O)[C@]6(CC)C=CCN([C@H]56)CC4)(O)C(=O)OC)N3C=O)=CC=2)OC)C[C@@](C2)(O)CC)N2CCC2=C1NC1=CC=CC=C21 IXJYMUFPNFFKIB-FMONCPFKSA-N 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 210000001503 joint Anatomy 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
- Stored Programmes (AREA)
Abstract
The invention provides a system and a method for FLUME automatic deployment, which can be used in the field of distributed technology, and the system comprises: the system comprises a log monitoring subsystem and an automatic deployment subsystem, wherein the log monitoring subsystem comprises an operation and maintenance server module and is used for generating a FLUME configuration file according to a FLUME log acquisition rule configured by a user; the automated deployment subsystem includes: the system comprises a server management module, an environment management module and a deployment management module, wherein the server management module is used for receiving information of a plurality of servers to be deployed, which are imported by a user, and the environment management module is used for receiving information of a plurality of environments to be deployed, which are newly built by the user; the script management module is used for receiving a FLUME automatic deployment script configured by a user; and after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script. The invention can realize the automatic deployment of the FLUME on a plurality of target servers.
Description
Technical Field
The invention relates to the field of distributed technology, in particular to a system and a method for FLUME automatic deployment.
Background
A log monitoring system generally comprises a plurality of key modules of log collection, analysis processing, index calculation and visual display, wherein the common FLUME for log collection is used as a collection tool. When a product is accessed, the flash needs to be deployed to a monitoring server of a log to be collected, and the customized configuration of a configuration file is performed. In recent years, as more and more products turn to a distributed micro-service architecture, the number of deployed nodes is greatly increased, and the workload of a traditional manual deployment method of FLUME acquisition tools and one-to-one configuration method is too large, so that the requirement of efficient operation and maintenance cannot be met.
Disclosure of Invention
The embodiment of the invention provides a system for automatically deploying FLUME, which is used for realizing the automatic deployment of the FLUME on a plurality of target servers and comprises the following steps:
a log monitoring subsystem and an automated deployment subsystem, wherein,
the log monitoring subsystem comprises an operation and maintenance server module, wherein the operation and maintenance server module is used for generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule is bound with a group of server information to be deployed;
the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein,
the server management module is used for receiving a plurality of pieces of information of the servers to be deployed, which are imported by users,
the system comprises an environment management module, a deployment management module and a deployment management module, wherein the environment management module is used for receiving a plurality of pieces of to-be-deployed environment information newly built by a user, and each to-be-deployed environment is used for managing a group of to-be-deployed servers;
the system comprises a script management module, a service module and a service module, wherein the script management module is used for receiving a FLUME automatic deployment script configured by a user, and the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
The embodiment of the invention provides a method for automatically deploying FLUME, which is used for realizing the automatic deployment of the FLUME on a plurality of target servers and comprises the following steps:
receiving a plurality of pieces of information of servers to be deployed, which are imported by a user,
receiving a plurality of pieces of to-be-deployed environment information newly built by a user, wherein each to-be-deployed environment is used for managing a group of to-be-deployed servers;
generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed;
receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises a FLUME installation package version and information of a server to be deployed;
after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
The embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the method for automatically deploying flash is implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for executing the method for automatically deploying flash is stored in the computer-readable storage medium.
In the embodiment of the invention, the log monitoring subsystem comprises an operation and maintenance server module and an operation and maintenance server module, and is used for generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed; the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein the server management module is used for receiving information of a plurality of servers to be deployed imported by a user, the environment management module is used for receiving information of a plurality of environments to be deployed newly built by the user, and each environment to be deployed is used for hosting a group of servers to be deployed; the system comprises a script management module, a service module and a service module, wherein the script management module is used for receiving a FLUME automatic deployment script configured by a user, and the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished. The system and the method provided by the embodiment of the invention have the following beneficial effects:
firstly, the automatic deployment of the FLUME is realized, the installation and the configuration of the FLUME can be finished on a plurality of servers in parallel by one key, and the manual operation cost is reduced;
secondly, friendly FLUME log acquisition rule configuration is provided, a user does not need to learn and understand complex fields in the FLUME configuration file, the background automatically generates the FLUME configuration file according to the FLUME log acquisition rule, and the use threshold is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a schematic diagram of a system for automatic FLUME deployment in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a system for automatic FLUME deployment in an embodiment of the present invention;
FIG. 3 is another schematic diagram of a system for automatic FLUME deployment in an embodiment of the present invention;
FIG. 4 is a detailed flowchart of the automatic deployment of FLUME in the embodiment of the present invention;
FIG. 5 is a flowchart of a method for automatic FLUME deployment in an embodiment of the present invention;
FIG. 6 is a diagram of a computer device in an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are used in an open-ended fashion, i.e., to mean including, but not limited to. Reference to the description of the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in the embodiments is for illustrative purposes to illustrate the implementation of the present application, and the sequence of steps is not limited and can be adjusted as needed.
The terms to which the present invention relates will be explained first.
FLUME: FLUME is a highly available and highly reliable distributed system for collecting, aggregating and transmitting mass logs provided by Cloudera.
Kafka: and the message buffer queue is used for putting the message into the queue firstly when the data volume is large, delaying for a certain time for processing, and avoiding dragging the subsequent processing logic.
POMP: a log monitoring subsystem for enterprise self-research collects logs from a server of each product by using FLUME, visually displays indexes such as transaction amount, success rate and response time of the product through log analysis and calculation processing, and monitors the running state of the product in real time.
The product is as follows: a business-specific concept refers to a system that provides certain functions or services to the outside. A product typically contains multiple servers (corresponding to multiple services in an active state).
SADP-X86: the invention relates to an automatic deployment platform which is self-developed by enterprises and suitable for an X86 architecture, wherein the platform provides an online execution function of self-defined scripts and operations, and can be used for automatically issuing a FLUME acquisition tool and carrying out customized configuration on a FLUME configuration file.
Fig. 1 is a schematic diagram of a system for automatically deploying flash in an embodiment of the present invention, as shown in fig. 1, the system includes: a log monitoring subsystem 101 and an automated deployment subsystem 102, wherein,
the log monitoring subsystem 101 comprises an operation and maintenance server module 1011, wherein the operation and maintenance server module 1011 is configured to generate a FLUME configuration file according to FLUME log collection rules configured by a user, and each FLUME log collection rule is bound with a group of information of a server to be deployed;
the automated deployment subsystem 102 includes a server management module 1021, an environment management module 1021, and a script management module 1023, wherein,
a server management module 1021 for receiving a plurality of pieces of information of servers to be deployed imported by users,
the environment management module 1022 is configured to receive information of a plurality of environments to be deployed newly created by a user, where each environment to be deployed is used for hosting a group of servers to be deployed;
the script management module 1023 is used for receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
In specific implementation, the server management module is used for receiving information of a plurality of servers to be deployed, which are imported by a user, and the servers to be deployed are target products. After receiving a script execution instruction of a user, the script management module issues the FLUME automatic deployment script to a corresponding server to be deployed and executes the script.
Fig. 2 is a schematic diagram of a system for flute automatic deployment in an embodiment of the present invention, where a log monitoring subsystem may be based on POMP, and an automatic deployment subsystem may be based on SADP-X86, the system proposed in the embodiment of the present invention fully utilizes CMDB capability of an existing platform (and SADP-X86) of an enterprise, and module functions in the log monitoring subsystem and the automatic deployment subsystem are implemented on the basis of the above-mentioned platform, without repeated development, but instead, linkage and docking between the subsystems are implemented, and a complete usage flow and solution are provided, so as to reduce development cost.
Fig. 3 is another schematic diagram of the system for automatically deploying flash in the embodiment of the present invention, and in an embodiment, the log monitoring subsystem further includes an operation and maintenance exhibition end module 1012, configured to receive, through a page, a plurality of flash log collection rules configured by a user.
In the above embodiment, the operation and maintenance display module includes a navigation page for guiding a user to configure a flash log collection rule, where the user can conveniently configure the flash log collection rule based on the navigation page, and one flash log collection rule includes:
(1) basic information such as rule name and product thereof
(2) Collecting source information by the log: multiple log collection paths can be created under one collection rule, and the log is delivered to topic of Kakfa under each path.
(3) Collecting advanced option configuration: such as whether to include directory level regular expressions, whether to use a stacked log multi-row merge pattern, log file cleanup policies, etc.
(4) Information of a server to be deployed bound by the FLUME log acquisition rule: a set of servers is selected on which to use this current flash log collection rule. When a server is selected, the log monitoring subsystem queries a server list under the to-be-deployed environment information by calling a server query interface of the SADP-X86 automatic deployment subsystem, so that a user can select the server.
In one embodiment, the system further comprises an FTP server module 103, configured to store the flash configuration file and different versions of the flash installation package;
the operation and maintenance server module is further used for: and issuing the FLUME configuration file to the FTP server module.
In one embodiment, the server information includes an IP of the server;
the FTP server module is specifically configured to: the FLUME configuration file is stored per IP subdirectory of the server.
In the above embodiment, if a plurality of flash log collection rules are bound to one server, the flash configuration files are merged.
In an embodiment, the automatic deployment subsystem further includes a server query interface 1024, configured to, after receiving a server query instruction from a user, display configured environment information to be deployed, and receive a set of information about a server to be deployed in one piece of environment information to be deployed selected by the user;
the operation and maintenance display end module is also used for: and sending a server query instruction to a server query interface, acquiring a group of information of the server to be deployed, and receiving the binding of the group of information of the server to be deployed and the FLUME log acquisition rule by a user.
In one embodiment, the log monitoring subsystem further comprises Kafka 1013;
and after the FLUME is started, delivering the logs collected based on the FLUME log collection rule into Kafka.
In the above embodiment, if the collected logs are received in Kafka after the flash is started, it is verified that the flash is successfully deployed, and delivery is performed according to topic in a classified manner.
Based on fig. 2, an embodiment of the present invention provides a detailed flowchart of flash automatic deployment, and as shown in fig. 4, the detailed steps include:
In summary, the system provided in the embodiment of the present invention has the following beneficial effects:
firstly, the automatic deployment of the FLUME is realized, the installation and the configuration of the FLUME can be finished on a plurality of servers in parallel by one key, and the manual operation cost is reduced;
secondly, friendly FLUME log acquisition rule configuration is provided, a user does not need to learn and understand complex fields in the FLUME configuration file, the background automatically generates the FLUME configuration file according to the FLUME log acquisition rule, and the use threshold is reduced.
Thirdly, the system provided by the embodiment of the invention fully utilizes the CMDB capability of the existing platform (and SADP-X86) of the enterprise, the module functions in the log monitoring subsystem and the automatic deployment subsystem are realized on the basis of the platform, repeated development is not needed, linkage and butt joint among the subsystems are realized, a complete use flow and a solution are provided, and the development cost is reduced.
The embodiment of the invention also provides a method for FLUME automatic deployment, the principle of which is similar to that of a system for FLUME automatic deployment, and the description is omitted here.
Fig. 5 is a flowchart of a method for automatically deploying flash in an embodiment of the present invention, which specifically includes:
and 505, after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is completed.
In an embodiment, the method further comprises:
and receiving a plurality of FLUME log collection rules configured by a user.
In an embodiment, the method further comprises:
the FLUME configuration file and different versions of the FLUME installation package are stored.
In an embodiment, the method further comprises:
the FLUME configuration file is stored per IP subdirectory of the server.
In an embodiment, the method further comprises:
after a server query instruction of a user is received, displaying configured environment information to be deployed;
receiving a group of information of servers to be deployed in one piece of information of environments to be deployed selected by a user;
and receiving the binding of the set of information of the server to be deployed and the FLUME log acquisition rule of the user.
In one embodiment, the FLUME logs collected based on the FLUME log collection rules are posted to Kafka after the FLUME is started.
In summary, the method provided in the embodiment of the present invention has the following beneficial effects:
firstly, the automatic deployment of the FLUME is realized, the installation and the configuration of the FLUME can be finished on a plurality of servers in parallel by one key, and the manual operation cost is reduced;
secondly, friendly FLUME log acquisition rule configuration is provided, a user does not need to learn and understand complex fields in the FLUME configuration file, the background automatically generates the FLUME configuration file according to the FLUME log acquisition rule, and the use threshold is reduced.
Thirdly, the system provided by the embodiment of the invention fully utilizes the CMDB capability of the existing platform (and SADP-X86) of the enterprise, the module functions in the log monitoring subsystem and the automatic deployment subsystem are realized on the basis of the platform, repeated development is not needed, linkage and butt joint among the subsystems are realized, a complete use flow and a solution are provided, and the development cost is reduced.
An embodiment of the present invention further provides a computer device, fig. 6 is a schematic diagram of a computer device in an embodiment of the present invention, where the computer device is capable of implementing all steps in the method for automatically deploying flash in the foregoing embodiment, and the computer device specifically includes the following contents:
a processor (processor)601, a memory (memory)602, a communication Interface (Communications Interface)603, and a communication bus 604;
the processor 601, the memory 602 and the communication interface 603 complete mutual communication through the communication bus 604; the communication interface 603 is used for implementing information transmission among related devices such as server-side devices, detection devices, user-side devices and the like;
the processor 601 is used to call the computer program in the memory 602, and when the processor executes the computer program, the processor implements all the steps in the method for automatically deploying flash in the above embodiments.
An embodiment of the present invention further provides a computer-readable storage medium, which is capable of implementing all the steps in the method for automatically deploying flash in the foregoing embodiment, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps of the method for automatically deploying flash in the foregoing embodiment.
It will be appreciated by one skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program service system embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program business systems according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (14)
1. A system for FLUME automatic deployment, comprising: a log monitoring subsystem and an automated deployment subsystem, wherein,
the log monitoring subsystem comprises an operation and maintenance server module, wherein the operation and maintenance server module is used for generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule is bound with a group of server information to be deployed;
the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein,
the server management module is used for receiving information of a plurality of servers to be deployed, which is imported by a user;
the system comprises an environment management module, a deployment management module and a deployment management module, wherein the environment management module is used for receiving a plurality of pieces of to-be-deployed environment information newly built by a user, and each to-be-deployed environment is used for managing a group of to-be-deployed servers;
the system comprises a script management module, a service module and a service module, wherein the script management module is used for receiving a FLUME automatic deployment script configured by a user, and the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
2. The system for flute automatic deployment as claimed in claim 1, wherein the log monitoring subsystem further comprises an operation and maintenance exhibition end module for receiving a plurality of flute log collection rules configured by a user through a page.
3. The system for automated FLUME deployment of claim 1, further comprising an FTP server module for storing FLUME configuration files and different versions of FLUME installation packages;
the operation and maintenance server module is further used for: and issuing the FLUME configuration file to the FTP server module.
4. A FLUME auto-deploy system as claimed in claim 3, wherein said server information comprises the IP of the server;
the FTP server module is specifically configured to: the FLUME configuration file is stored per IP subdirectory of the server.
5. The FLUME automatic deployment system of claim 1, wherein the automatic deployment subsystem further comprises a server query interface for displaying configured environment information to be deployed after receiving a server query instruction of a user, and receiving a set of information of the server to be deployed in one environment information to be deployed selected by the user;
the operation and maintenance display end module is also used for: and sending a server query instruction to a server query interface, acquiring a group of information of the server to be deployed, and receiving the binding of the group of information of the server to be deployed and the FLUME log acquisition rule by a user.
6. The FLUME auto-deploy system of claim 1, wherein the log monitoring subsystem further comprises Kafka;
and after the FLUME is started, delivering the logs collected based on the FLUME log collection rule into Kafka.
7. A method for FLUME automatic deployment, comprising:
receiving a plurality of pieces of information of servers to be deployed, which are imported by a user,
receiving a plurality of pieces of to-be-deployed environment information newly built by a user, wherein each to-be-deployed environment is used for managing a group of to-be-deployed servers;
generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed;
receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises a FLUME installation package version and information of a server to be deployed;
after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
8. The method for FLUME automated deployment as recited in claim 7, further comprising:
and receiving a plurality of FLUME log collection rules configured by a user.
9. The method for FLUME automated deployment as recited in claim 7, further comprising:
the FLUME configuration file and different versions of the FLUME installation package are stored.
10. A method of flash auto-deployment as claimed in claim 9, further comprising:
the FLUME configuration file is stored per IP subdirectory of the server.
11. The method for FLUME automated deployment as recited in claim 7, further comprising:
after a server query instruction of a user is received, displaying configured environment information to be deployed;
receiving a group of information of servers to be deployed in one piece of information of environments to be deployed selected by a user;
and receiving the binding of the set of information of the server to be deployed and the FLUME log acquisition rule of the user.
12. The method for FLUME automatic deployment of claim 7, wherein the FLUME delivers the collected logs based on FLUME log collection rules into Kafka after startup.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 7 to 12 when executing the computer program.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 7 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994424.4A CN113626047B (en) | 2021-08-27 | 2021-08-27 | FLUME automatic deployment system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994424.4A CN113626047B (en) | 2021-08-27 | 2021-08-27 | FLUME automatic deployment system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113626047A true CN113626047A (en) | 2021-11-09 |
| CN113626047B CN113626047B (en) | 2024-04-19 |
Family
ID=78388053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110994424.4A Active CN113626047B (en) | 2021-08-27 | 2021-08-27 | FLUME automatic deployment system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626047B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150291A (en) * | 2022-07-05 | 2022-10-04 | 中国银行股份有限公司 | Operation and maintenance system and method based on event driving |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9578063B1 (en) * | 2015-11-20 | 2017-02-21 | International Business Machines Corporation | Application self-service for assured log management in cloud environments |
| CN107908414A (en) * | 2017-11-22 | 2018-04-13 | 中国银行股份有限公司 | A kind of method, apparatus and system of the deployment of application server version |
| CN111580832A (en) * | 2020-04-29 | 2020-08-25 | 电科云(北京)科技有限公司 | Application release system and method applied to heterogeneous multi-cloud environment |
| CN112035124A (en) * | 2020-09-03 | 2020-12-04 | 中国银行股份有限公司 | Application deployment method and device |
-
2021
- 2021-08-27 CN CN202110994424.4A patent/CN113626047B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9578063B1 (en) * | 2015-11-20 | 2017-02-21 | International Business Machines Corporation | Application self-service for assured log management in cloud environments |
| CN107908414A (en) * | 2017-11-22 | 2018-04-13 | 中国银行股份有限公司 | A kind of method, apparatus and system of the deployment of application server version |
| CN111580832A (en) * | 2020-04-29 | 2020-08-25 | 电科云(北京)科技有限公司 | Application release system and method applied to heterogeneous multi-cloud environment |
| CN112035124A (en) * | 2020-09-03 | 2020-12-04 | 中国银行股份有限公司 | Application deployment method and device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150291A (en) * | 2022-07-05 | 2022-10-04 | 中国银行股份有限公司 | Operation and maintenance system and method based on event driving |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113626047B (en) | 2024-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11909604B2 (en) | Automatic provisioning of monitoring for containerized microservices | |
| US10171377B2 (en) | Orchestrating computing resources between different computing environments | |
| CN104536899B (en) | A kind of software deployment and its maintaining method based on Intelligent cluster | |
| CN109951547B (en) | Transaction request parallel processing method, device, equipment and medium | |
| CN110083455B (en) | Graph calculation processing method, graph calculation processing device, graph calculation processing medium and electronic equipment | |
| CN111752799A (en) | Service link tracking method, device, equipment and storage medium | |
| CN113987074A (en) | Distributed service full-link monitoring method and device, electronic equipment and storage medium | |
| CN102880503A (en) | Data analysis system and data analysis method | |
| US20170085653A1 (en) | Method, device and system for message distribution | |
| CN107370796A (en) | A kind of intelligent learning system based on Hyper TF | |
| CN103810272A (en) | Data processing method and system | |
| CN110442502A (en) | A kind of method, apparatus buried a little, equipment and storage medium | |
| CN112737800A (en) | Service node fault positioning method, call chain generation method and server | |
| CN112685499A (en) | Method, device and equipment for synchronizing process data of work service flow | |
| CN110750453B (en) | HTML 5-based intelligent mobile terminal testing method, system, server and storage medium | |
| CN113626047A (en) | System and method for FLUME automatic deployment | |
| CN111190805A (en) | Training log visualization system, method and equipment of deep learning training platform | |
| WO2016091141A1 (en) | Method and apparatus for information collection | |
| CN112825525A (en) | Method and apparatus for processing transactions | |
| CN112579406A (en) | Log call chain generation method and device | |
| CN112764837B (en) | Data reporting method, device, storage medium and terminal | |
| CN106202470B (en) | The processing method and system of script | |
| CN109257256A (en) | Apparatus monitoring method, device, computer equipment and storage medium | |
| US11200138B1 (en) | Policy-based request tracing using a computer | |
| KR100645529B1 (en) | Log management system capable of log processing and method using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |