CN112860276A - Distributed file system deployment method, device, server and storage medium - Google Patents
Distributed file system deployment method, device, server and storage medium Download PDFInfo
- Publication number
- CN112860276A CN112860276A CN202110119938.5A CN202110119938A CN112860276A CN 112860276 A CN112860276 A CN 112860276A CN 202110119938 A CN202110119938 A CN 202110119938A CN 112860276 A CN112860276 A CN 112860276A
- Authority
- CN
- China
- Prior art keywords
- module
- file
- installation
- distributed
- deployment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 238000009434 installation Methods 0.000 claims abstract description 166
- 238000013515 script Methods 0.000 claims abstract description 94
- 238000012544 monitoring process Methods 0.000 claims description 97
- 238000004891 communication Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a distributed file system deployment method, a distributed file system deployment device, a server and a storage medium, relates to the technical field of storage, and is applied to any node in a distributed system cluster. The method comprises the following steps: installing the installation files of the distributed file system into a preset directory of a preset operating system by operating the installation package of the distributed file system; calling an execution file of a preset deployment module in the installation file from a preset directory by executing a preset deployment script in the installation package, so that the preset deployment module calls each configuration module in the installation file to generate a configuration file; and calling an execution file of the configuration synchronization module in the installation file from a preset directory by executing a preset synchronization configuration script in the installation package so as to synchronize the configuration file to other nodes in the distributed system cluster. By the method and the device, rapid deployment of each node in the distributed system cluster can be achieved.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a distributed file system deployment method, apparatus, server, and storage medium.
Background
With the rise and popularization of cloud computing, the distributed shared storage system is valued by the industry, and the distributed file system ceph is rapidly the hottest open source distributed storage system due to the excellent characteristics of stability, high availability and expandability.
The existing installation and deployment schemes of various ceph distributed storage systems cannot be used in an offline state or a secret-related environment.
The manual deployment scheme which can be deployed in an off-line state or a confidential environment is complex, a complete set of three-node cluster deployment needs nearly hundreds of instructions, the technical requirements on operation and maintenance personnel are very high, the time required by the deployment process is long, and the deployment process is easy to go on business.
Disclosure of Invention
In view of the above-mentioned shortcomings in the prior art, an object of the present invention is to provide a method, an apparatus, a server and a storage medium for deploying a distributed file system, so as to implement rapid deployment of a distributed system cluster in a state of not accessing the internet.
In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:
in a first aspect, an embodiment of the present application provides a distributed file system deployment method, which is applied to any node in a distributed system cluster, where nodes in the distributed system cluster are connected through local area network communication; the method comprises the following steps:
installing installation files of a distributed file system into a preset directory of a preset operating system by operating an installation package of the distributed file system;
calling the execution files of the deployment modules from the preset directory by executing the execution scripts of the deployment modules in the installation files so that the deployment modules call the configuration modules in the installation files to generate configuration files;
and calling the execution file of the configuration synchronization module from the preset directory by executing the execution script of the configuration synchronization module in the installation file so as to synchronize the configuration file to other nodes in the distributed system cluster.
Optionally, the method further includes:
and calling an execution file of the monitoring module from the preset directory by executing the adding and deleting script of the monitoring module in the installation file, and adding or removing the monitoring module in the installation file of the distributed file system.
Optionally, the method further includes:
and calling an execution file of the object storage module from the preset directory by executing the adding and deleting script of the object storage module in the installation file, and adding or removing the object storage module in the installation file of the distributed file system.
Optionally, the method further includes:
and calling an execution file of the daemon module of the metadata service from the preset directory by executing the adding and deleting script of the daemon module of the metadata service in the installation file, and adding or removing the daemon module of the metadata service in the installation file of the distributed file system.
Optionally, the method further includes:
and calling an execution file of the monitoring cluster state module from the preset directory by executing an adding and deleting script of the monitoring cluster state module in the installation file, and adding or removing the monitoring cluster state module in the installation file of the distributed file system.
Optionally, the method further includes:
and configuring the starting and stopping of the monitoring cluster state module in the distributed file system by executing the enabling script of the monitoring cluster state module in the installation file.
Optionally, the method further includes:
and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
In a second aspect, an embodiment of the present application further provides a distributed file system deployment apparatus, which is applied to any node in a distributed system cluster, where nodes in the distributed system cluster are connected through lan communication, and the apparatus includes:
the installation module is used for installing the installation files of the distributed file system into a preset directory of a preset operating system by operating the installation package of the distributed file system;
the configuration file generation module is used for calling the execution files of the deployment modules from the preset directory by executing the execution scripts of the deployment modules in the installation files so that the deployment modules call the configuration modules in the installation files to generate configuration files;
and the synchronization module is used for calling the execution file of the configuration synchronization module from the preset directory by executing the execution script of the configuration synchronization module in the installation file so as to synchronize the configuration file to other nodes in the distributed system cluster.
Optionally, the apparatus further comprises:
and the monitoring module adding and deleting module is used for calling the execution file of the monitoring module from the preset directory by executing the adding and deleting script of the monitoring module in the installation file, and adding or removing the monitoring module in the installation file of the distributed file system.
Optionally, the apparatus further comprises:
and the object storage module adding and deleting module is used for calling the execution file of the object storage module from the preset directory by executing the adding and deleting script of the object storage module in the installation file, and adding or removing the object storage module in the installation file of the distributed file system.
Optionally, the apparatus further comprises:
and the daemon module adding and deleting module is used for calling the execution file of the daemon module of the metadata service from the preset directory through the adding and deleting script of the daemon module of the metadata service in the execution installation file, and adding or removing the daemon module of the metadata service in the installation file of the distributed file system.
Optionally, the apparatus further comprises:
and the monitoring cluster state module adding and deleting module is used for calling the execution file of the monitoring cluster state module from the preset directory by executing the adding and deleting script of the monitoring cluster state module in the installation file, and adding or removing the monitoring cluster state module in the installation file of the distributed file system.
Optionally, the apparatus further comprises:
and the start-stop module is used for configuring the start-stop of the monitoring cluster state module in the distributed file system by executing the enabling script of the monitoring cluster state module in the installation file.
Optionally, the apparatus further comprises:
and the node information recording module is used for recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
In a third aspect, an embodiment of the present application further provides a server, including: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, when an electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to execute the steps of the distributed file system deployment method.
In a fourth aspect, an embodiment of the present application further provides a storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the distributed file system deployment method as described in any one of the above are executed.
The beneficial effect of this application is:
the distributed file system deployment method, the distributed file system deployment device, the server and the storage medium are applied to any node in a distributed system cluster, all nodes in the distributed system cluster are in communication connection through a local area network, and installation files of the distributed file system are installed in a preset directory of a preset operating system by operating an installation package of the distributed file system; the deployment module calls the execution files of the deployment modules from the preset directory by executing the execution scripts of the deployment modules in the installation files so that the deployment modules call the configuration modules in the installation files to generate the configuration files, and calls the execution files of the configuration synchronization modules from the preset directory by executing the execution scripts of the configuration synchronization modules in the installation files so as to synchronize the configuration files to other nodes in the distributed system cluster. According to the scheme provided by the application, the installation files of the distributed file system are installed on each node by operating the installation package of the distributed file system, the distributed file system is deployed on each node by executing the execution script of the deployment module to generate the configuration files of each node, the configuration files of each node are synchronized by executing the synchronous configuration script, the rapid deployment of each node in a distributed system cluster can be realized, and the rapid synchronization of each node can be realized through a local area network when each node is in an off-line state.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic diagram of a distributed system cluster according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a first method for deploying a distributed file system according to an embodiment of the present application;
fig. 3 is a flowchart illustrating a second method for deploying a distributed file system according to an embodiment of the present application;
fig. 4 is a flowchart illustrating a third method for deploying a distributed file system according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a distributed file system deployment apparatus according to an embodiment of the present application;
fig. 6 is a schematic diagram of a server according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not intended to indicate or imply relative importance.
In the description of the present invention, "connected" and/or "connected" should be understood as "electrically connected", "communicatively connected", and the like, if circuits, modules, units, and the like, which are connected and/or "connected" have electrical signals or data transfer therebetween.
The distributed file system deployment method provided by the embodiment of the application is applied to any node in a distributed system cluster, and all nodes in the distributed system cluster are in communication connection through a local area network.
Fig. 1 is a schematic diagram of a distributed system cluster provided in an embodiment of the present application, as shown in fig. 1, a distributed system cluster 10 includes a plurality of servers 100 to be deployed with a distributed file system, each server 100 is used as a node in the distributed system cluster 10, and the plurality of servers 100 in the distributed system cluster 10 are connected through a local area network.
Fig. 2 is a schematic flowchart of a first method for deploying a distributed file system according to an embodiment of the present application; as shown in fig. 2, the method includes:
s100: and installing the installation files of the distributed file system into a preset directory of a preset operating system by operating the installation package of the distributed file system.
Specifically, the installation package of the distributed file system includes a pre-designed installation file with a plurality of components, and the installation file of each component includes an execution script of the component and an execution file of a module corresponding to the script.
Copying the installation package of the distributed file system to a server 100 to be deployed with the distributed file system, and running the installation package of the distributed file system in a preset operating system of the server 100 to install the installation files of the distributed file system in a preset directory of the preset operating system. For example, the preset operating system is an operating system of a server, and may be a Linux system, the preset directory is a storage address of an installation file selected when an installation package of the distributed file system is run, and the preset directory may be a default storage address preset in the installation package of the distributed file system, or may be a self-selected storage address selected by providing a function of selecting a storage address according to the installation package of the distributed file system.
In an alternative embodiment, the installation package of the distributed file system includes: the installation directory configuration module runs an installation package of the distributed file system, and can record the installation files of the distributed file system in the process of installing the installation files to a preset directory of a preset operating system: the storage path of the installation files of the plurality of components, the name of the configuration file, the storage path of the server operation data, the address of the gateway and the like.
S200: and calling the execution file of the deployment module from a preset directory by executing the execution script of the deployment module in the installation file, so that the deployment module calls each configuration module in the installation file to generate the configuration file.
Specifically, the installation files of the distributed file system include an execution script of the deployment module, an execution file of the deployment module, and execution files of the configuration modules called by the execution files of the deployment module, the execution files of the deployment module are called according to a saving path of the installation files of the deployment module recorded in the installation directory configuration module by executing the execution script of the deployment module in a preset operating system, the configuration modules are called by executing the execution files of the deployment module, and the configuration files are generated by executing the execution files of the configuration modules.
In an alternative embodiment, the deployment modules are one-key deployment modules, and each configuration module may include: the system comprises a deployment environment initialization module, an environment variable automatic adding module, a cluster attribute configuration module, a deployment information recording module, a service starting/stopping module and a service self-starting module.
The one-key deployment module can call and execute the functions of each configuration module to generate the configuration file corresponding to each configuration module.
The deployment environment initialization module is used for creating a configuration file of cluster recording information, creating a directory of the configuration file, recording an IP of a server and a node name of the server, wherein the IP of the server is used for cluster deployment verification, and the node name of the server is a unique name of each server in a cluster.
And the automatic environment variable adding module is used for loading the environment variables of the preset operating system, namely the parameters of the running environment of the preset operating system according to the saving paths of the installation files of the plurality of components recorded in the installation directory configuration module.
And the cluster attribute configuration module is used for configuring a cluster log storage path, a cluster log plug-in storage path, a Python module storage path, a monitoring cluster state module (Managers, mgr) dependent path, an ID, an IP (Internet protocol) and a port of a monitoring module (Monitor, mon), an fsid value, a cluster public gateway and other configuration parameters. For example, the mgr module is a component in the distributed file system, the dependent path of the mgr module needs to be configured independently, the mon module is a component in the distributed file system, the ID of the mon module is the unique identifier of the mon module, and an IP and a port need to be configured, and the fsid value is the unique identifier of the cluster and is generated when cluster deployment is performed.
For example, table 1 configures other configuration parameters that the module needs to configure for the cluster attribute.
Table 1 other configuration parameters for the cluster attribute configuration module
The deployment information recording module stores information to be recorded, which is generated in the deployment process of the distributed file system, in a configuration file of cluster recording information created by the deployment environment initialization module, and specifically includes: the ID (Identity document) of the mon module, the IP (Internet Protocol) and port, the number of Object Storage devices (osd), the ID of each osd module, the ID of the Metadata service daemon module (mds), and the ID of the mgr module.
And the service starting/stopping module is used for starting/stopping the service of the server, starting the service of the server by calling the starting instruction and stopping the service of the server by calling the stopping command.
And the service self-starting module is used for realizing the self-starting of the distributed file system when the server starts.
S300: and calling the execution file of the configuration synchronization module from the preset directory by executing the execution script of the configuration synchronization module in the installation file so as to synchronize the configuration file to other nodes in the distributed system cluster.
Specifically, the installation files of the distributed file system further include an execution script for configuring the synchronization module and an execution file for configuring the synchronization module, the execution script push _ config.sh for configuring the synchronization module is executed in a preset operating system, the execution file for configuring the synchronization module is called according to a storage path of the installation files of the configuration synchronization module recorded in the installation directory configuration module, the node name of a target server to be synchronized and the IP of the target server are input, the configuration file of the node and the configuration file of the target node are synchronized, manual copying and pasting of the configuration file of each node are not needed, synchronization efficiency is improved, and error rate is reduced.
The distributed file system deployment method provided by the embodiment of the application is applied to any node in a distributed system cluster, wherein all nodes in the distributed system cluster are in communication connection through a local area network, and installation files of the distributed file system are installed in a preset directory of a preset operating system by operating an installation package of the distributed file system; the deployment module calls the execution files of the deployment modules from the preset directory by executing the execution scripts of the deployment modules in the installation files so that the deployment modules call the configuration modules in the installation files to generate the configuration files, and calls the execution files of the configuration synchronization modules from the preset directory by executing the execution scripts of the configuration synchronization modules in the installation files so as to synchronize the configuration files to other nodes in the distributed system cluster. According to the scheme provided by the application, the installation files of the distributed file system are installed on each node by operating the installation package of the distributed file system, the distributed file system is deployed on each node by executing the execution script of the deployment module to generate the configuration files of each node, the configuration files of each node are synchronized by executing the synchronous configuration script, the rapid deployment of each node in a distributed system cluster can be realized, and the rapid synchronization of each node can be realized through a local area network when each node is in an off-line state.
On the basis of the foregoing embodiments, an embodiment of the present application further provides a distributed document system deployment method, and fig. 3 is a flowchart illustrating a second distributed document system deployment method provided in the embodiment of the present application, and as shown in fig. 3, the method may further include:
s401: and calling the execution file of the monitoring module from a preset directory by executing the adding and deleting script of the monitoring module in the installation file, and adding or removing the monitoring module in the installation file of the distributed file system.
Specifically, the add-delete script of the monitoring module includes an add script of the monitoring module and a delete script of the monitoring module, and the installation file of the distributed file system further includes an add script of the monitoring module, an execution file of the add monitoring module, a remove script of the monitoring module, and an execution file of the remove monitoring module.
The adding script add _ mon.sh of the monitoring module is executed in a preset operating system, so that the execution file of the adding monitoring module is called according to the saving path of the installing file of the adding monitoring module recorded in the installing directory configuration module, and the monitoring module mon is added in the installing file of the distributed file system, so that the monitoring module mon is added on any node of the distributed file system to be deployed; the removal script rm _ mon.sh of the monitoring module is executed in a preset operating system, so that the execution file of the removal monitoring module is called according to the saving path of the installation file of the removal monitoring module recorded in the installation directory configuration module, and the monitoring module mon is deleted in the installation file of the distributed file system, so that the monitoring module mon is removed from any node of the distributed file system to be deployed, and the monitoring module mon is used for maintaining the state of the whole distributed system cluster.
In an optional implementation manner, after the monitoring module mon is added or removed from any node of the distributed file system to be deployed, the method further includes:
and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
Specifically, the node information of the node where the distributed file system is located may be recorded in the configuration file by the deployment information recording module, and the configuration file may be synchronized to other nodes in the distributed system cluster by executing the synchronization configuration script, so as to add or remove the monitoring module mon on the other nodes.
On the basis of any of the foregoing embodiments, an embodiment of the present application further provides a distributed document system deployment method, as shown in fig. 3, the method may further include:
s402: and calling the execution file of the object storage module from a preset directory by executing the adding and deleting script of the object storage module in the installation file, and adding or removing the object storage module in the installation file of the distributed file system.
Specifically, the add-delete script of the object storage module includes an add script of the object storage module and a remove script of the object storage module, and the installation file of the distributed file system further includes an add script of the object storage module, an execution file of the add object storage module, a remove script of the object storage module, and an execution file of the remove object storage module.
Adding an add script add _ osd.sh of an object storage module in a preset operating system, calling an execution file of the added object storage module according to a saving path of an installation file of the added object storage module recorded in an installation directory configuration module, and adding the object storage module osd in the installation file of the distributed file system to realize adding the object storage module osd on any node of the distributed file system to be deployed; the method comprises the steps of executing a removal script rm _ osd.sh of an object storage module in a preset operating system, calling an execution file of the object storage module according to a saving path of an installation file of the object storage module to be removed recorded in an installation directory configuration module, and removing the object storage module osd from the installation file of a distributed file system to remove the object storage module osd from any node of the distributed file system of the file to be deployed, wherein the object storage module osd is used for providing storage resources for a server.
In an optional embodiment, after adding or removing the object storage module osd on any node of the distributed file system to be deployed, the method further comprises:
and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
Specifically, the node information of the node where the distributed file system is located may be recorded in the configuration file by the deployment information recording module, and the configuration file may be synchronized to other nodes in the distributed system cluster by executing the synchronization configuration script, so as to add or remove the object storage module osd on the other nodes.
On the basis of any of the foregoing embodiments, an embodiment of the present application further provides a distributed document system deployment method, as shown in fig. 3, the method may further include:
s403: and calling the execution file of the daemon module of the metadata service from a preset directory by executing the adding and deleting script of the daemon module of the metadata service in the installation file, and adding or removing the daemon module of the metadata service in the installation file of the distributed file system.
Specifically, the add-delete script of the daemon module of the metadata service includes an add script of the daemon module of the metadata service and a remove script of the daemon module of the metadata service, and the installation file of the distributed file system further includes an add script of the daemon module of the metadata service, an execution file of the daemon module of the add metadata service, a remove script of the daemon module of the metadata service, and an execution file of the daemon module of the remove metadata service.
The adding script add _ mds.sh of the daemon module for executing the metadata service in the preset operating system is used for calling the execution file of the daemon module for adding the metadata service according to the saving path of the installation file of the daemon module for adding the metadata service, which is recorded in the installation directory configuration module, and the daemon module mds for adding the metadata service is added in the installation file of the distributed file system so as to realize the addition of the daemon module mds for the metadata service on any node of the distributed file system for the file to be deployed; according to the method, a removal script rm _ mds.sh of a daemon module for executing metadata service in a preset operating system is used for calling an execution file of the daemon module for removing the metadata service according to a saving path of an installation file of the daemon module for removing the metadata service recorded in an installation directory configuration module, and the daemon module mds for removing the metadata service is removed from the installation file of the distributed file system so as to remove the daemon module mds for the metadata service on any node of the distributed file system of the file to be deployed, wherein the daemon module mds for the metadata service is used for caching and synchronizing the distributed metadata.
In an alternative embodiment, after the daemon module mds of the metadata service on any node of the distributed file system is to be deployed, the method further comprises:
and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
Specifically, the node information of the node where the distributed file system is located may be recorded in the configuration file through the deployment information recording module, and the configuration file may be synchronized to other nodes in the distributed system cluster by executing the synchronization configuration script, so as to add or remove the daemon module mds of the metadata service on the other nodes.
On the basis of any of the foregoing embodiments, an embodiment of the present application further provides a distributed document system deployment method, as shown in fig. 3, the method may further include:
s404: and calling the execution file of the monitoring cluster state module from a preset directory by executing the addition and deletion script of the monitoring cluster state module in the installation file, and adding or removing the monitoring cluster state module in the installation file of the distributed file system.
Specifically, the add-delete script of the monitoring cluster state module includes an add script of the monitoring cluster state module and a remove script of the monitoring cluster state module, and the installation file of the distributed file system further includes an add script of the monitoring cluster state module, an execution file of the add monitoring cluster state module, a remove script of the monitoring cluster state module, and an execution file of the remove monitoring cluster state module.
Adding a monitoring cluster state module into an installation file of a distributed file system by executing an adding script add _ mgr.sh of the monitoring cluster state module in a preset operating system, calling the execution file of the adding monitoring cluster state module according to a saving path of the installation file of the adding monitoring cluster state module recorded in an installation directory configuration module, and adding the monitoring cluster state module mgr into the installation file of the distributed file system to realize adding the monitoring cluster state module mgr on any node of the distributed file system to be deployed; the removal script rm _ mgr.sh of the monitoring cluster state module is executed in the preset operating system, so that the execution file of the removal monitoring cluster state module is called according to the saving path of the installation file of the removal monitoring cluster state module recorded in the installation directory configuration module, the monitoring cluster state module mgr is removed from the installation file of the distributed file system, the monitoring cluster state module mgr is removed from any node of the distributed file system to be deployed, and the monitoring cluster state module mgr is responsible for tracking cluster operation indexes and the current state of the cluster.
In an optional implementation manner, after monitoring the cluster status module mgr at any node of the distributed file system to be deployed, the method further includes:
and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
Specifically, the node information of the node where the distributed file system is located may be recorded in the configuration file by the deployment information recording module, and the configuration file may be synchronized to other nodes in the distributed system cluster by executing the synchronization configuration script, so as to add or remove the monitoring cluster state module mgr on the other nodes.
In the method for deploying a distributed file system according to any of the embodiments, the monitoring module is added or removed from the installation file of the distributed file system by executing the add-delete script of the monitoring module, the object storage module is added or removed from the installation file of the distributed file system by executing the add-delete script of the object removal module, the daemon module for adding or removing the metadata service is added or removed from the installation file of the distributed file system by executing the add-delete script of the daemon module for performing the metadata service, and the monitoring cluster state module is added or removed from the installation file of the distributed file system by executing the add-delete script of the monitoring cluster state module. By the method provided by any embodiment, the monitoring module, the object storage module, the metadata service daemon module and the monitoring cluster state module can be rapidly deployed on any node, rapid deployment of the distributed file system is achieved, meanwhile, the monitoring module, the object storage module, the metadata service daemon module and the monitoring cluster state module deployed on any node can be synchronized to other nodes in the distributed system cluster through the synchronous configuration script, manual copying and pasting of configuration files of each node are not needed, synchronous deployment efficiency is high, and errors are not prone to occurring.
On the basis of any of the foregoing embodiments, an embodiment of the present application further provides a method for deploying a distributed document system, where the method may further include:
and configuring the starting and stopping of the cluster state monitoring module in the distributed file system by executing the enabling script of the cluster state monitoring module in the installation file.
Specifically, the installation file of the distributed file system further includes an enabling script of the monitoring cluster state module, and the enabling script enable _ mgr _ module.sh is executed in a preset operating system, so that the monitoring cluster state module starts to operate or stops to operate.
On the basis of any of the foregoing embodiments, an embodiment of the present application further provides a distributed document system deployment method, and fig. 4 is a flowchart illustrating a third distributed document system deployment method provided in the embodiment of the present application, and as shown in fig. 4, the method further includes:
s400: and calling the execution file of the module from a preset directory by executing the adding and deleting script of the module in the installation file, and adding or removing the module in the installation file of the distributed file system.
Specifically, S400 may include at least one of the methods of S401, S402, S403, and S404 described above.
S500: and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
Specifically, the node information of the node where the distributed file system is located may be recorded in the configuration file by the deployment information recording module, and the configuration file may be synchronized to other nodes in the distributed system cluster by executing the synchronization configuration script, so as to add or remove the module on the other nodes.
On the basis of any of the foregoing embodiments, an embodiment of the present application further provides a distributed file system deployment apparatus, which is applied to any node in a distributed system cluster, where nodes in the distributed system cluster are connected through a local area network in a communication manner, and fig. 5 is a schematic structural diagram of the distributed file system deployment apparatus provided in the embodiment of the present application, and as shown in fig. 5, the apparatus includes:
the installation module 200 is configured to install an installation file of the distributed file system into a preset directory of a preset operating system by running an installation package of the distributed file system;
a configuration file generating module 300, configured to invoke the execution file of the deployment module from a preset directory by executing the execution script of the deployment module in the installation file, so that the deployment module invokes each configuration module in the installation file to generate a configuration file;
and the synchronization module 400 is configured to call the execution file of the configuration synchronization module from the preset directory by executing the execution script of the configuration synchronization module in the installation file, so as to synchronize the configuration file to other nodes in the distributed system cluster.
Optionally, the apparatus may further comprise:
and the monitoring module adding and deleting module is used for calling the execution file of the monitoring module from the preset directory by executing the adding and deleting script of the monitoring module in the installation file, and adding or removing the monitoring module in the installation file of the distributed file system.
Optionally, the apparatus may further comprise:
and the object storage module adding and deleting module is used for calling the execution file of the object storage module from the preset directory by executing the adding and deleting script of the object storage module in the installation file, and adding or removing the object storage module in the installation file of the distributed file system.
Optionally, the apparatus may further comprise:
and the daemon module adding and deleting module is used for calling the execution file of the daemon module of the metadata service from the preset directory through the adding and deleting script of the daemon module of the metadata service in the execution installation file, and adding or removing the daemon module of the metadata service in the installation file of the distributed file system.
Optionally, the apparatus may further comprise:
and the monitoring cluster state module adding and deleting module is used for calling the execution file of the monitoring cluster state module from the preset directory by executing the adding and deleting script of the monitoring cluster state module in the installation file, and adding or removing the monitoring cluster state module in the installation file of the distributed file system.
Optionally, the apparatus may further comprise:
and the start-stop module is used for configuring the start-stop of the cluster state monitoring module in the distributed file system by executing the enabling script of the cluster state monitoring module in the installation file.
Optionally, the apparatus may further comprise:
and the node information recording module is used for recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 6 is a schematic diagram of a server provided in an embodiment of the present application, where the server 100 includes: a processor 101, a memory 102, and a bus.
The memory 102 stores program instructions executable by the processor 101, and when the server is running, the processor 101 communicates with the memory through the bus, and the processor 101 executes the program instructions to execute any one of the method embodiments. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the computer program performs any of the above method embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and shall be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A distributed file system deployment method is characterized in that the method is applied to any node in a distributed system cluster, and all nodes in the distributed system cluster are in communication connection through a local area network; the method comprises the following steps:
installing installation files of a distributed file system into a preset directory of a preset operating system by operating an installation package of the distributed file system;
calling the execution files of the deployment modules from the preset directory by executing the execution scripts of the deployment modules in the installation files so that the deployment modules call the configuration modules in the installation files to generate configuration files;
and calling the execution file of the configuration synchronization module from the preset directory by executing the execution script of the configuration synchronization module in the installation file so as to synchronize the configuration file to other nodes in the distributed system cluster.
2. The method of claim 1, wherein the method further comprises:
and calling an execution file of the monitoring module from the preset directory by executing the adding and deleting script of the monitoring module in the installation file, and adding or removing the monitoring module in the installation file of the distributed file system.
3. The method of claim 1, wherein the method further comprises:
and calling an execution file of the object storage module from the preset directory by executing the adding and deleting script of the object storage module in the installation file, and adding or removing the object storage module in the installation file of the distributed file system.
4. The method of claim 1, wherein the method further comprises:
and calling the execution file of the metadata service daemon module from the preset directory by executing the add-delete script of the metadata service daemon module in the installation file, and adding or removing the metadata service daemon module in the installation file of the distributed file system.
5. The method of claim 1, wherein the method further comprises:
and calling an execution file of the monitoring cluster state module from the preset directory by executing an adding and deleting script of the monitoring cluster state module in the installation file, and adding or removing the monitoring cluster state module in the installation file of the distributed file system.
6. The method of claim 5, wherein the method further comprises:
and configuring the starting and stopping of the monitoring cluster state module in the distributed file system by executing the enabling script of the monitoring cluster state module in the installation file.
7. The method of claim 1, wherein the method further comprises:
and recording the node information of the node where the distributed file system is located by executing the deployment information recording module in the installation file.
8. A distributed file system deployment device is applied to any node in a distributed system cluster, and all nodes in the distributed system cluster are in communication connection through a local area network, and the device comprises:
the installation module is used for installing the installation files of the distributed file system into a preset directory of a preset operating system by operating the installation package of the distributed file system;
the configuration file generation module is used for calling an execution file of a preset deployment module in the installation file from the preset directory by executing a deployment script of the preset operating system so that the preset deployment module calls each configuration module in the installation file to generate a configuration file;
and the synchronization module is used for calling an execution file of the configuration synchronization module in the installation file from the preset directory by executing the synchronization configuration script of the preset operating system so as to synchronize the configuration file to other nodes in the distributed system cluster.
9. A server, comprising: a processor, a memory and a bus, wherein the memory stores program instructions executable by the processor, the processor and the memory communicate via the bus when the electronic device is running, and the processor executes the program instructions to perform the steps of the distributed document system deployment method as claimed in any one of claims 1 to 7.
10. A storage medium having stored thereon a computer program for executing the steps of the distributed file system deployment method according to any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110119938.5A CN112860276B (en) | 2021-01-28 | 2021-01-28 | Distributed file system deployment method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110119938.5A CN112860276B (en) | 2021-01-28 | 2021-01-28 | Distributed file system deployment method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112860276A true CN112860276A (en) | 2021-05-28 |
| CN112860276B CN112860276B (en) | 2024-06-28 |
Family
ID=75987732
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110119938.5A Active CN112860276B (en) | 2021-01-28 | 2021-01-28 | Distributed file system deployment method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112860276B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107332936A (en) * | 2017-08-25 | 2017-11-07 | 长沙曙通信息科技有限公司 | A kind of distributed storage distribution and installation network system realization |
| US20180157521A1 (en) * | 2016-12-02 | 2018-06-07 | Nutanix, Inc. | Virtualized server systems and methods including load balancing for virtualized file servers |
| CN111625497A (en) * | 2020-05-22 | 2020-09-04 | 柏科数据技术（深圳）股份有限公司 | Deployment method, device, equipment and storage medium of distributed file system |
| CN111625496A (en) * | 2020-05-22 | 2020-09-04 | 柏科数据技术（深圳）股份有限公司 | Method, device and equipment for deploying distributed file system in virtual machine environment |
| CN112114746A (en) * | 2020-08-27 | 2020-12-22 | 紫光云(南京)数字技术有限公司 | Automatic deployment method of distributed storage cluster |
-
2021
- 2021-01-28 CN CN202110119938.5A patent/CN112860276B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180157521A1 (en) * | 2016-12-02 | 2018-06-07 | Nutanix, Inc. | Virtualized server systems and methods including load balancing for virtualized file servers |
| CN107332936A (en) * | 2017-08-25 | 2017-11-07 | 长沙曙通信息科技有限公司 | A kind of distributed storage distribution and installation network system realization |
| CN111625497A (en) * | 2020-05-22 | 2020-09-04 | 柏科数据技术（深圳）股份有限公司 | Deployment method, device, equipment and storage medium of distributed file system |
| CN111625496A (en) * | 2020-05-22 | 2020-09-04 | 柏科数据技术（深圳）股份有限公司 | Method, device and equipment for deploying distributed file system in virtual machine environment |
| CN112114746A (en) * | 2020-08-27 | 2020-12-22 | 紫光云(南京)数字技术有限公司 | Automatic deployment method of distributed storage cluster |
Non-Patent Citations (1)
| Title |
|---|
| 陈友旭: "分布式文件系统中元数据管理优化", 《中国博士学位论文全文数据库 信息科技辑》, pages 137 - 13 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112860276B (en) | 2024-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107526626B (en) | Docker container thermal migration method and system based on CRIU | |
| CN111614733B (en) | Deployment method, device and storage medium of distributed multi-fragmentation cluster | |
| US11474910B2 (en) | Method, device and computer program product to backup data | |
| WO2012000999A1 (en) | Configuring a computer system for a software package installation | |
| CN106713493B (en) | System and method for constructing distributed file in computer cluster environment | |
| CN107222320A (en) | The method and apparatus that Cloud Server cluster sets up High Availabitity connection | |
| CN113783765B (en) | Method, system, equipment and medium for realizing intercommunication between cloud internal network and cloud external network | |
| CN112202853B (en) | Data synchronization method, system, computer device and storage medium | |
| CN111159105B (en) | System and method for supporting NFS protocol and setting ACL based on mangia sharing mode | |
| CN113656147A (en) | Cluster deployment method, device, equipment and storage medium | |
| CN113626286A (en) | Multi-cluster instance processing method and device, electronic equipment and storage medium | |
| CN111538625A (en) | Ambari cluster deployment and data backup method based on Docker technology and electronic equipment | |
| US10318385B2 (en) | Service recovery using snapshots and interservice messages | |
| CN111371620A (en) | Bare computer service deployment method and system based on distributed cache | |
| CN114510464A (en) | Management method and management system of high-availability database | |
| CN104360839A (en) | Method for automatically migrating LINUX system to WINDOWS system | |
| CN104636441A (en) | Network file system realization method and device | |
| CN107526838B (en) | Method and device for database cluster capacity expansion | |
| CN111741102B (en) | Upgrading method and device for distributed micro-service application | |
| CN111339055B (en) | Big data cluster capacity expansion method and device | |
| CN107623705B (en) | Storage mode upgrading method, device and system based on video cloud storage system | |
| CN112860276B (en) | Distributed file system deployment method, device, server and storage medium | |
| CN112073499A (en) | Dynamic service method of multi-machine type cloud physical server | |
| CN114281600A (en) | Disaster recovery backup and recovery method, device, equipment and storage medium | |
| CN113535476A (en) | Method and device for rapidly recovering cloud assets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |