CN113626047A - System and method for FLUME automatic deployment - Google Patents

System and method for FLUME automatic deployment Download PDF

Info

Publication number
CN113626047A
CN113626047A CN202110994424.4A CN202110994424A CN113626047A CN 113626047 A CN113626047 A CN 113626047A CN 202110994424 A CN202110994424 A CN 202110994424A CN 113626047 A CN113626047 A CN 113626047A
Authority
CN
China
Prior art keywords
flume
server
deployed
user
deployment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110994424.4A
Other languages
Chinese (zh)
Other versions
CN113626047B (en
Inventor
尹充
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bank of China Ltd
Original Assignee
Bank of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bank of China Ltd filed Critical Bank of China Ltd
Priority to CN202110994424.4A priority Critical patent/CN113626047B/en
Publication of CN113626047A publication Critical patent/CN113626047A/en
Application granted granted Critical
Publication of CN113626047B publication Critical patent/CN113626047B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/60Software deployment
    • G06F8/61Installation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/70Software maintenance or management
    • G06F8/71Version control; Configuration management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Stored Programmes (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention provides a system and a method for FLUME automatic deployment, which can be used in the field of distributed technology, and the system comprises: the system comprises a log monitoring subsystem and an automatic deployment subsystem, wherein the log monitoring subsystem comprises an operation and maintenance server module and is used for generating a FLUME configuration file according to a FLUME log acquisition rule configured by a user; the automated deployment subsystem includes: the system comprises a server management module, an environment management module and a deployment management module, wherein the server management module is used for receiving information of a plurality of servers to be deployed, which are imported by a user, and the environment management module is used for receiving information of a plurality of environments to be deployed, which are newly built by the user; the script management module is used for receiving a FLUME automatic deployment script configured by a user; and after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script. The invention can realize the automatic deployment of the FLUME on a plurality of target servers.

Description

System and method for FLUME automatic deployment
Technical Field
The invention relates to the field of distributed technology, in particular to a system and a method for FLUME automatic deployment.
Background
A log monitoring system generally comprises a plurality of key modules of log collection, analysis processing, index calculation and visual display, wherein the common FLUME for log collection is used as a collection tool. When a product is accessed, the flash needs to be deployed to a monitoring server of a log to be collected, and the customized configuration of a configuration file is performed. In recent years, as more and more products turn to a distributed micro-service architecture, the number of deployed nodes is greatly increased, and the workload of a traditional manual deployment method of FLUME acquisition tools and one-to-one configuration method is too large, so that the requirement of efficient operation and maintenance cannot be met.
Disclosure of Invention
The embodiment of the invention provides a system for automatically deploying FLUME, which is used for realizing the automatic deployment of the FLUME on a plurality of target servers and comprises the following steps:
a log monitoring subsystem and an automated deployment subsystem, wherein,
the log monitoring subsystem comprises an operation and maintenance server module, wherein the operation and maintenance server module is used for generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule is bound with a group of server information to be deployed;
the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein,
the server management module is used for receiving a plurality of pieces of information of the servers to be deployed, which are imported by users,
the system comprises an environment management module, a deployment management module and a deployment management module, wherein the environment management module is used for receiving a plurality of pieces of to-be-deployed environment information newly built by a user, and each to-be-deployed environment is used for managing a group of to-be-deployed servers;
the system comprises a script management module, a service module and a service module, wherein the script management module is used for receiving a FLUME automatic deployment script configured by a user, and the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
The embodiment of the invention provides a method for automatically deploying FLUME, which is used for realizing the automatic deployment of the FLUME on a plurality of target servers and comprises the following steps:
receiving a plurality of pieces of information of servers to be deployed, which are imported by a user,
receiving a plurality of pieces of to-be-deployed environment information newly built by a user, wherein each to-be-deployed environment is used for managing a group of to-be-deployed servers;
generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed;
receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises a FLUME installation package version and information of a server to be deployed;
after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
The embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the method for automatically deploying flash is implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for executing the method for automatically deploying flash is stored in the computer-readable storage medium.
In the embodiment of the invention, the log monitoring subsystem comprises an operation and maintenance server module and an operation and maintenance server module, and is used for generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed; the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein the server management module is used for receiving information of a plurality of servers to be deployed imported by a user, the environment management module is used for receiving information of a plurality of environments to be deployed newly built by the user, and each environment to be deployed is used for hosting a group of servers to be deployed; the system comprises a script management module, a service module and a service module, wherein the script management module is used for receiving a FLUME automatic deployment script configured by a user, and the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished. The system and the method provided by the embodiment of the invention have the following beneficial effects:
firstly, the automatic deployment of the FLUME is realized, the installation and the configuration of the FLUME can be finished on a plurality of servers in parallel by one key, and the manual operation cost is reduced;
secondly, friendly FLUME log acquisition rule configuration is provided, a user does not need to learn and understand complex fields in the FLUME configuration file, the background automatically generates the FLUME configuration file according to the FLUME log acquisition rule, and the use threshold is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a schematic diagram of a system for automatic FLUME deployment in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a system for automatic FLUME deployment in an embodiment of the present invention;
FIG. 3 is another schematic diagram of a system for automatic FLUME deployment in an embodiment of the present invention;
FIG. 4 is a detailed flowchart of the automatic deployment of FLUME in the embodiment of the present invention;
FIG. 5 is a flowchart of a method for automatic FLUME deployment in an embodiment of the present invention;
FIG. 6 is a diagram of a computer device in an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are used in an open-ended fashion, i.e., to mean including, but not limited to. Reference to the description of the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in the embodiments is for illustrative purposes to illustrate the implementation of the present application, and the sequence of steps is not limited and can be adjusted as needed.
The terms to which the present invention relates will be explained first.
FLUME: FLUME is a highly available and highly reliable distributed system for collecting, aggregating and transmitting mass logs provided by Cloudera.
Kafka: and the message buffer queue is used for putting the message into the queue firstly when the data volume is large, delaying for a certain time for processing, and avoiding dragging the subsequent processing logic.
POMP: a log monitoring subsystem for enterprise self-research collects logs from a server of each product by using FLUME, visually displays indexes such as transaction amount, success rate and response time of the product through log analysis and calculation processing, and monitors the running state of the product in real time.
The product is as follows: a business-specific concept refers to a system that provides certain functions or services to the outside. A product typically contains multiple servers (corresponding to multiple services in an active state).
SADP-X86: the invention relates to an automatic deployment platform which is self-developed by enterprises and suitable for an X86 architecture, wherein the platform provides an online execution function of self-defined scripts and operations, and can be used for automatically issuing a FLUME acquisition tool and carrying out customized configuration on a FLUME configuration file.
Fig. 1 is a schematic diagram of a system for automatically deploying flash in an embodiment of the present invention, as shown in fig. 1, the system includes: a log monitoring subsystem 101 and an automated deployment subsystem 102, wherein,
the log monitoring subsystem 101 comprises an operation and maintenance server module 1011, wherein the operation and maintenance server module 1011 is configured to generate a FLUME configuration file according to FLUME log collection rules configured by a user, and each FLUME log collection rule is bound with a group of information of a server to be deployed;
the automated deployment subsystem 102 includes a server management module 1021, an environment management module 1021, and a script management module 1023, wherein,
a server management module 1021 for receiving a plurality of pieces of information of servers to be deployed imported by users,
the environment management module 1022 is configured to receive information of a plurality of environments to be deployed newly created by a user, where each environment to be deployed is used for hosting a group of servers to be deployed;
the script management module 1023 is used for receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
In specific implementation, the server management module is used for receiving information of a plurality of servers to be deployed, which are imported by a user, and the servers to be deployed are target products. After receiving a script execution instruction of a user, the script management module issues the FLUME automatic deployment script to a corresponding server to be deployed and executes the script.
Fig. 2 is a schematic diagram of a system for flute automatic deployment in an embodiment of the present invention, where a log monitoring subsystem may be based on POMP, and an automatic deployment subsystem may be based on SADP-X86, the system proposed in the embodiment of the present invention fully utilizes CMDB capability of an existing platform (and SADP-X86) of an enterprise, and module functions in the log monitoring subsystem and the automatic deployment subsystem are implemented on the basis of the above-mentioned platform, without repeated development, but instead, linkage and docking between the subsystems are implemented, and a complete usage flow and solution are provided, so as to reduce development cost.
Fig. 3 is another schematic diagram of the system for automatically deploying flash in the embodiment of the present invention, and in an embodiment, the log monitoring subsystem further includes an operation and maintenance exhibition end module 1012, configured to receive, through a page, a plurality of flash log collection rules configured by a user.
In the above embodiment, the operation and maintenance display module includes a navigation page for guiding a user to configure a flash log collection rule, where the user can conveniently configure the flash log collection rule based on the navigation page, and one flash log collection rule includes:
(1) basic information such as rule name and product thereof
(2) Collecting source information by the log: multiple log collection paths can be created under one collection rule, and the log is delivered to topic of Kakfa under each path.
(3) Collecting advanced option configuration: such as whether to include directory level regular expressions, whether to use a stacked log multi-row merge pattern, log file cleanup policies, etc.
(4) Information of a server to be deployed bound by the FLUME log acquisition rule: a set of servers is selected on which to use this current flash log collection rule. When a server is selected, the log monitoring subsystem queries a server list under the to-be-deployed environment information by calling a server query interface of the SADP-X86 automatic deployment subsystem, so that a user can select the server.
In one embodiment, the system further comprises an FTP server module 103, configured to store the flash configuration file and different versions of the flash installation package;
the operation and maintenance server module is further used for: and issuing the FLUME configuration file to the FTP server module.
In one embodiment, the server information includes an IP of the server;
the FTP server module is specifically configured to: the FLUME configuration file is stored per IP subdirectory of the server.
In the above embodiment, if a plurality of flash log collection rules are bound to one server, the flash configuration files are merged.
In an embodiment, the automatic deployment subsystem further includes a server query interface 1024, configured to, after receiving a server query instruction from a user, display configured environment information to be deployed, and receive a set of information about a server to be deployed in one piece of environment information to be deployed selected by the user;
the operation and maintenance display end module is also used for: and sending a server query instruction to a server query interface, acquiring a group of information of the server to be deployed, and receiving the binding of the group of information of the server to be deployed and the FLUME log acquisition rule by a user.
In one embodiment, the log monitoring subsystem further comprises Kafka 1013;
and after the FLUME is started, delivering the logs collected based on the FLUME log collection rule into Kafka.
In the above embodiment, if the collected logs are received in Kafka after the flash is started, it is verified that the flash is successfully deployed, and delivery is performed according to topic in a classified manner.
Based on fig. 2, an embodiment of the present invention provides a detailed flowchart of flash automatic deployment, and as shown in fig. 4, the detailed steps include:
step 401, a user imports a server to be deployed of a target product in an automatic SADP-X86 deployment subsystem;
step 402, a user creates a to-be-deployed environment in an automatic SADP-X86 deployment subsystem, and adds a group of to-be-deployed servers to the to-be-deployed environment;
step 403, a user creates a FLUME collection rule in the POMP log monitoring flat subsystem and binds a group of information of servers to be deployed; after the FLUME acquisition rule configuration is completed, when a user clicks 'release', the log monitoring flat subsystem generates a FLUME configuration file according to the FLUME log acquisition rule configured by the user, wherein each FLUME log acquisition rule is bound with a group of server information to be deployed, namely each server IP corresponds to one FLUME configuration file, and the FLUME configuration files are uploaded to a corresponding directory of the FTP server module.
Step 404, configuring a FLUME automatic deployment script in the SADP-X86 automatic deployment subsystem by a user;
step 405, the user issues a script execution instruction to the SADP-X86 automatic deployment subsystem, the SADP-X86 automatic deployment subsystem automatically pulls the flash installation package and the corresponding flash configuration file from the FTP server module for deployment, and the flash is started after the deployment is completed.
In summary, the system provided in the embodiment of the present invention has the following beneficial effects:
firstly, the automatic deployment of the FLUME is realized, the installation and the configuration of the FLUME can be finished on a plurality of servers in parallel by one key, and the manual operation cost is reduced;
secondly, friendly FLUME log acquisition rule configuration is provided, a user does not need to learn and understand complex fields in the FLUME configuration file, the background automatically generates the FLUME configuration file according to the FLUME log acquisition rule, and the use threshold is reduced.
Thirdly, the system provided by the embodiment of the invention fully utilizes the CMDB capability of the existing platform (and SADP-X86) of the enterprise, the module functions in the log monitoring subsystem and the automatic deployment subsystem are realized on the basis of the platform, repeated development is not needed, linkage and butt joint among the subsystems are realized, a complete use flow and a solution are provided, and the development cost is reduced.
The embodiment of the invention also provides a method for FLUME automatic deployment, the principle of which is similar to that of a system for FLUME automatic deployment, and the description is omitted here.
Fig. 5 is a flowchart of a method for automatically deploying flash in an embodiment of the present invention, which specifically includes:
step 501, receiving information of a plurality of servers to be deployed, which is imported by a user;
step 502, receiving information of a plurality of environments to be deployed newly built by a user, wherein each environment to be deployed is used for managing a group of servers to be deployed;
step 503, generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed;
step 504, receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises a FLUME installation package version and information of a server to be deployed;
and 505, after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is completed.
In an embodiment, the method further comprises:
and receiving a plurality of FLUME log collection rules configured by a user.
In an embodiment, the method further comprises:
the FLUME configuration file and different versions of the FLUME installation package are stored.
In an embodiment, the method further comprises:
the FLUME configuration file is stored per IP subdirectory of the server.
In an embodiment, the method further comprises:
after a server query instruction of a user is received, displaying configured environment information to be deployed;
receiving a group of information of servers to be deployed in one piece of information of environments to be deployed selected by a user;
and receiving the binding of the set of information of the server to be deployed and the FLUME log acquisition rule of the user.
In one embodiment, the FLUME logs collected based on the FLUME log collection rules are posted to Kafka after the FLUME is started.
In summary, the method provided in the embodiment of the present invention has the following beneficial effects:
firstly, the automatic deployment of the FLUME is realized, the installation and the configuration of the FLUME can be finished on a plurality of servers in parallel by one key, and the manual operation cost is reduced;
secondly, friendly FLUME log acquisition rule configuration is provided, a user does not need to learn and understand complex fields in the FLUME configuration file, the background automatically generates the FLUME configuration file according to the FLUME log acquisition rule, and the use threshold is reduced.
Thirdly, the system provided by the embodiment of the invention fully utilizes the CMDB capability of the existing platform (and SADP-X86) of the enterprise, the module functions in the log monitoring subsystem and the automatic deployment subsystem are realized on the basis of the platform, repeated development is not needed, linkage and butt joint among the subsystems are realized, a complete use flow and a solution are provided, and the development cost is reduced.
An embodiment of the present invention further provides a computer device, fig. 6 is a schematic diagram of a computer device in an embodiment of the present invention, where the computer device is capable of implementing all steps in the method for automatically deploying flash in the foregoing embodiment, and the computer device specifically includes the following contents:
a processor (processor)601, a memory (memory)602, a communication Interface (Communications Interface)603, and a communication bus 604;
the processor 601, the memory 602 and the communication interface 603 complete mutual communication through the communication bus 604; the communication interface 603 is used for implementing information transmission among related devices such as server-side devices, detection devices, user-side devices and the like;
the processor 601 is used to call the computer program in the memory 602, and when the processor executes the computer program, the processor implements all the steps in the method for automatically deploying flash in the above embodiments.
An embodiment of the present invention further provides a computer-readable storage medium, which is capable of implementing all the steps in the method for automatically deploying flash in the foregoing embodiment, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps of the method for automatically deploying flash in the foregoing embodiment.
It will be appreciated by one skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program service system embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program business systems according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (14)

1. A system for FLUME automatic deployment, comprising: a log monitoring subsystem and an automated deployment subsystem, wherein,
the log monitoring subsystem comprises an operation and maintenance server module, wherein the operation and maintenance server module is used for generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, and each FLUME log acquisition rule is bound with a group of server information to be deployed;
the automatic deployment subsystem comprises a server management module, an environment management module and a script management module, wherein,
the server management module is used for receiving information of a plurality of servers to be deployed, which is imported by a user;
the system comprises an environment management module, a deployment management module and a deployment management module, wherein the environment management module is used for receiving a plurality of pieces of to-be-deployed environment information newly built by a user, and each to-be-deployed environment is used for managing a group of to-be-deployed servers;
the system comprises a script management module, a service module and a service module, wherein the script management module is used for receiving a FLUME automatic deployment script configured by a user, and the FLUME automatic deployment script comprises the version of a FLUME installation package and information of a server to be deployed; after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
2. The system for flute automatic deployment as claimed in claim 1, wherein the log monitoring subsystem further comprises an operation and maintenance exhibition end module for receiving a plurality of flute log collection rules configured by a user through a page.
3. The system for automated FLUME deployment of claim 1, further comprising an FTP server module for storing FLUME configuration files and different versions of FLUME installation packages;
the operation and maintenance server module is further used for: and issuing the FLUME configuration file to the FTP server module.
4. A FLUME auto-deploy system as claimed in claim 3, wherein said server information comprises the IP of the server;
the FTP server module is specifically configured to: the FLUME configuration file is stored per IP subdirectory of the server.
5. The FLUME automatic deployment system of claim 1, wherein the automatic deployment subsystem further comprises a server query interface for displaying configured environment information to be deployed after receiving a server query instruction of a user, and receiving a set of information of the server to be deployed in one environment information to be deployed selected by the user;
the operation and maintenance display end module is also used for: and sending a server query instruction to a server query interface, acquiring a group of information of the server to be deployed, and receiving the binding of the group of information of the server to be deployed and the FLUME log acquisition rule by a user.
6. The FLUME auto-deploy system of claim 1, wherein the log monitoring subsystem further comprises Kafka;
and after the FLUME is started, delivering the logs collected based on the FLUME log collection rule into Kafka.
7. A method for FLUME automatic deployment, comprising:
receiving a plurality of pieces of information of servers to be deployed, which are imported by a user,
receiving a plurality of pieces of to-be-deployed environment information newly built by a user, wherein each to-be-deployed environment is used for managing a group of to-be-deployed servers;
generating a FLUME configuration file according to FLUME log acquisition rules configured by a user, wherein each FLUME log acquisition rule is bound with a group of information of a server to be deployed;
receiving a FLUME automatic deployment script configured by a user, wherein the FLUME automatic deployment script comprises a FLUME installation package version and information of a server to be deployed;
after receiving a script execution instruction of a user, issuing the FLUME automatic deployment script to a corresponding server to be deployed and executing the FLUME automatic deployment script, automatically pulling the FLUME installation package and a corresponding FLUME configuration file for deployment according to the version of the FLUME installation package and the information of the server to be deployed when the FLUME automatic deployment script is executed, and starting the FLUME after the deployment is finished.
8. The method for FLUME automated deployment as recited in claim 7, further comprising:
and receiving a plurality of FLUME log collection rules configured by a user.
9. The method for FLUME automated deployment as recited in claim 7, further comprising:
the FLUME configuration file and different versions of the FLUME installation package are stored.
10. A method of flash auto-deployment as claimed in claim 9, further comprising:
the FLUME configuration file is stored per IP subdirectory of the server.
11. The method for FLUME automated deployment as recited in claim 7, further comprising:
after a server query instruction of a user is received, displaying configured environment information to be deployed;
receiving a group of information of servers to be deployed in one piece of information of environments to be deployed selected by a user;
and receiving the binding of the set of information of the server to be deployed and the FLUME log acquisition rule of the user.
12. The method for FLUME automatic deployment of claim 7, wherein the FLUME delivers the collected logs based on FLUME log collection rules into Kafka after startup.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 7 to 12 when executing the computer program.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 7 to 12.
CN202110994424.4A 2021-08-27 2021-08-27 FLUME automatic deployment system and method Active CN113626047B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110994424.4A CN113626047B (en) 2021-08-27 2021-08-27 FLUME automatic deployment system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110994424.4A CN113626047B (en) 2021-08-27 2021-08-27 FLUME automatic deployment system and method

Publications (2)

Publication Number Publication Date
CN113626047A true CN113626047A (en) 2021-11-09
CN113626047B CN113626047B (en) 2024-04-19

Family

ID=78388053

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110994424.4A Active CN113626047B (en) 2021-08-27 2021-08-27 FLUME automatic deployment system and method

Country Status (1)

Country Link
CN (1) CN113626047B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115150291A (en) * 2022-07-05 2022-10-04 中国银行股份有限公司 Operation and maintenance system and method based on event driving

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9578063B1 (en) * 2015-11-20 2017-02-21 International Business Machines Corporation Application self-service for assured log management in cloud environments
CN107908414A (en) * 2017-11-22 2018-04-13 中国银行股份有限公司 A kind of method, apparatus and system of the deployment of application server version
CN111580832A (en) * 2020-04-29 2020-08-25 电科云(北京)科技有限公司 Application release system and method applied to heterogeneous multi-cloud environment
CN112035124A (en) * 2020-09-03 2020-12-04 中国银行股份有限公司 Application deployment method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9578063B1 (en) * 2015-11-20 2017-02-21 International Business Machines Corporation Application self-service for assured log management in cloud environments
CN107908414A (en) * 2017-11-22 2018-04-13 中国银行股份有限公司 A kind of method, apparatus and system of the deployment of application server version
CN111580832A (en) * 2020-04-29 2020-08-25 电科云(北京)科技有限公司 Application release system and method applied to heterogeneous multi-cloud environment
CN112035124A (en) * 2020-09-03 2020-12-04 中国银行股份有限公司 Application deployment method and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115150291A (en) * 2022-07-05 2022-10-04 中国银行股份有限公司 Operation and maintenance system and method based on event driving

Also Published As

Publication number Publication date
CN113626047B (en) 2024-04-19

Similar Documents

Publication Publication Date Title
US11909604B2 (en) Automatic provisioning of monitoring for containerized microservices
US10171377B2 (en) Orchestrating computing resources between different computing environments
CN104536899B (en) A kind of software deployment and its maintaining method based on Intelligent cluster
CN109951547B (en) Transaction request parallel processing method, device, equipment and medium
CN110083455B (en) Graph calculation processing method, graph calculation processing device, graph calculation processing medium and electronic equipment
CN111752799A (en) Service link tracking method, device, equipment and storage medium
CN113987074A (en) Distributed service full-link monitoring method and device, electronic equipment and storage medium
CN102880503A (en) Data analysis system and data analysis method
US20170085653A1 (en) Method, device and system for message distribution
CN107370796A (en) A kind of intelligent learning system based on Hyper TF
CN103810272A (en) Data processing method and system
CN110442502A (en) A kind of method, apparatus buried a little, equipment and storage medium
CN112737800A (en) Service node fault positioning method, call chain generation method and server
CN112685499A (en) Method, device and equipment for synchronizing process data of work service flow
CN110750453B (en) HTML 5-based intelligent mobile terminal testing method, system, server and storage medium
CN113626047A (en) System and method for FLUME automatic deployment
CN111190805A (en) Training log visualization system, method and equipment of deep learning training platform
WO2016091141A1 (en) Method and apparatus for information collection
CN112825525A (en) Method and apparatus for processing transactions
CN112579406A (en) Log call chain generation method and device
CN112764837B (en) Data reporting method, device, storage medium and terminal
CN106202470B (en) The processing method and system of script
CN109257256A (en) Apparatus monitoring method, device, computer equipment and storage medium
US11200138B1 (en) Policy-based request tracing using a computer
KR100645529B1 (en) Log management system capable of log processing and method using the same

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant