CN108111566B - Cloud storage system capacity expansion method and device and cloud storage system - Google Patents
Cloud storage system capacity expansion method and device and cloud storage system Download PDFInfo
- Publication number
- CN108111566B CN108111566B CN201611066896.9A CN201611066896A CN108111566B CN 108111566 B CN108111566 B CN 108111566B CN 201611066896 A CN201611066896 A CN 201611066896A CN 108111566 B CN108111566 B CN 108111566B
- Authority
- CN
- China
- Prior art keywords
- server
- capacity expansion
- servers
- auxiliary
- auxiliary server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a cloud storage system capacity expansion method, a cloud storage system capacity expansion device and a cloud storage system, wherein the method is applied to a management server in the cloud storage system and comprises the following steps: determining at least two capacity expansion servers which are registered to the management server within a preset time; selecting an auxiliary server from the at least two capacity expansion servers, and indicating the auxiliary server to manage the at least two capacity expansion servers; when a data writing request is detected, writing data into the auxiliary server so that the auxiliary server writes the received data into at least two capacity expansion servers according to a load balancing principle; monitoring the occupation proportion of the storage space of at least two capacity expansion servers through an auxiliary server; when the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers. By applying the embodiment of the invention, data can be stably written into the capacity expansion server.
Description
Technical Field
The invention relates to the technical field of cloud storage, in particular to a method and a device for expanding a cloud storage system and the cloud storage system.
Background
Currently, users often access data through cloud storage systems. The cloud storage system is a system which is extended and developed on the cloud computing concept, has functions of cluster application, grid technology or distributed file system and the like, is integrated and cooperatively works through application software, and provides data storage and service access functions to the outside. The cloud storage system may include a management server and a storage server. The management server is used for writing the received data into a storage server in the cloud storage system; the storage server is used for storing the received data.
In practical applications, the storage capacity of the cloud storage system varies from a few PB (bytes) to hundreds PB. However, as the cloud storage technology is widely applied, the capacity of the fixed cloud storage system cannot meet the requirement of the user. Therefore, technicians often need to manually add a corresponding number of capacity expansion servers to the cloud storage system according to storage requirements, so as to increase the storage capacity of the cloud storage system.
However, when the cloud storage system is expanded in this way, the following problems may occur: during capacity expansion, the management server can preferentially store data to the capacity expansion server with high storage performance, which causes a surge in storage pressure of the capacity expansion server, even exceeds the data writing capability that the capacity expansion server can bear, and causes data loss to be written into the capacity expansion server.
Disclosure of Invention
The embodiment of the invention aims to provide a cloud storage system capacity expansion method and device and a cloud storage system, so as to stably write data into a capacity expansion server.
In a first aspect, an embodiment of the present invention provides a method for expanding a cloud storage system, where the method is applied to a management server in a cloud storage system, and the method may include:
determining at least two capacity expansion servers which are registered to the management server within a preset time;
selecting an auxiliary server from the at least two capacity expansion servers, and instructing the auxiliary server to manage the at least two capacity expansion servers, wherein the management server does not write data into any capacity expansion server registered to the management server within the preset time before the auxiliary server is determined;
when a data writing request is detected, writing data into the auxiliary server so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
and when the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers.
Optionally, the step of selecting the auxiliary server from the at least two capacity expansion servers may include:
and determining a target capacity expansion server of the at least two capacity expansion servers as an auxiliary server, wherein the target capacity expansion server is a capacity expansion server which is registered to the management server first in the preset time.
Optionally, the step of selecting the auxiliary server from the at least two capacity expansion servers may include:
and selecting the expansion server selected by the at least two expansion servers as an auxiliary server, wherein the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the expansion servers.
Optionally, before the step of writing data to the auxiliary server, the method may further include:
and sending an initialization and/or formatting instruction to the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
Optionally, the step of writing, by the auxiliary server, the received data into the at least two capacity expansion servers according to a load balancing principle may include:
the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
In a second aspect, an embodiment of the present invention further provides a cloud storage system capacity expansion device, which is applied to a management server in a cloud storage system, where the device may include:
the system comprises a determining unit, a judging unit and a processing unit, wherein the determining unit is used for determining at least two capacity expansion servers which are registered to the management server within preset time;
the selecting unit is used for selecting an auxiliary server from the at least two capacity expansion servers;
an indicating unit, configured to instruct the auxiliary server to manage the at least two capacity expansion servers, where the management server does not write data to any capacity expansion server registered in the management server within the preset time before the auxiliary server is determined;
the data writing unit is used for writing data into the auxiliary server when a data writing request is detected, so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
the monitoring unit is used for monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
and the acquisition unit is used for acquiring the management authority for managing the at least two capacity expansion servers when the storage space occupation ratios of the at least two capacity expansion servers reach a first threshold value.
Optionally, in an implementation manner, the selecting unit is specifically configured to:
and determining a target capacity expansion server of the at least two capacity expansion servers as an auxiliary server, wherein the target capacity expansion server is a capacity expansion server which is registered to the management server first in the preset time.
Optionally, in another implementation manner, the selecting unit is specifically configured to:
and selecting the expansion server selected by the at least two expansion servers as an auxiliary server, wherein the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the expansion servers.
Optionally, the apparatus may further include:
and the sending unit is used for sending an initialization and/or formatting instruction to the auxiliary server before the data writing unit writes data into the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
Optionally, after the data writing unit writes data into the auxiliary server, the step of writing the received data into the at least two capacity expansion servers by the auxiliary server according to a load balancing principle may include:
the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
In a third aspect, an embodiment of the present invention further provides a cloud storage system, including a management server and a plurality of capacity expansion servers, where the management server is configured to:
determining at least two capacity expansion servers which are registered to the management server within a preset time;
selecting an auxiliary server from the at least two capacity expansion servers, and instructing the auxiliary server to manage the at least two capacity expansion servers, wherein the management server does not write data into any capacity expansion server registered to the management server within the preset time before the auxiliary server is determined;
when a data writing request is detected, writing data into the auxiliary server so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
and when the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers.
In the embodiment of the invention, the management server in the cloud storage system may select one expansion server as an auxiliary server from expansion servers registered to the management server within a preset time; before the auxiliary server is selected and obtained, data are not written into any capacity expansion server, so that the problem of surge of the instantaneous storage pressure of the capacity expansion server when the capacity expansion server is accessed to a cloud storage system is avoided; after the auxiliary server is selected and obtained, the auxiliary server is indicated to manage the at least two capacity expansion servers which are registered to the management server within the preset time; when a data writing request is detected, writing data into an auxiliary server, and stably writing the data into a capacity expansion server by the auxiliary server; the management server can also monitor the occupation proportion of the storage space of each capacity expansion server through the auxiliary server; when the auxiliary server monitors that the occupation proportion of the storage space of each expansion server reaches a first threshold value, the management server acquires the management authority of each expansion server, and at the moment, the management server can stably write data into the expansion servers, namely, the expansion servers are stably added into the cloud storage system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a capacity expansion method for a cloud storage system according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating a capacity expansion method of a cloud storage system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a capacity expansion device of a cloud storage system according to an embodiment of the present invention;
fig. 4 is a schematic interaction diagram of a management server and an auxiliary server in a cloud storage system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the problem that when a cloud storage system is expanded, a management server in the cloud storage system preferentially stores data to an expansion server with high storage performance, so that the storage pressure of the expansion server is increased suddenly, and even exceeds the data writing capability that the expansion server can bear, the embodiment of the invention provides a cloud storage system expansion method and device.
First, a capacity expansion method of a cloud storage system according to an embodiment of the present invention is described below.
It should be noted that the execution main body of the cloud storage system capacity expansion method is as follows: there is a need for a management server in a cloud storage system that expands storage capacity. The functional software for implementing the capacity expansion method of the cloud storage system according to the embodiment of the present invention may be: the software specially used for capacity expansion, which is arranged in the management server, may also be: and the functional plug-in is arranged in the software for capacity expansion in the management server, but is not limited to this.
Referring to fig. 1, the cloud storage system expansion method may include the following steps:
s101: determining at least two capacity expansion servers which are registered to the management server within a preset time;
it is to be understood that the management server may receive the registration request sent by the capacity expansion server, and further, the management server may determine at least two capacity expansion servers that are registered to itself within a preset time.
For example, the management server may determine that at least two capacity expansion servers are registered to itself within a time period of 9:00 to 9:05, and use the at least two capacity expansion servers as a first capacity expansion device cluster. Of course, it is reasonable that the management server may also determine that at least two capacity expansion servers are registered in the preset time of 10:00 to 15:05, and use the determined at least two capacity expansion servers as the second capacity expansion device cluster.
S102: selecting an auxiliary server from the at least two capacity expansion servers, and instructing the auxiliary server to manage the at least two capacity expansion servers, wherein the management server does not write data into any capacity expansion server registered to the management server within the preset time before the auxiliary server is determined;
after determining at least two capacity expansion servers registered to the management server within a preset time, in an implementation manner, the management server may determine a target capacity expansion server of the determined at least two capacity expansion servers as an auxiliary server. The target expansion server may be the expansion server that is first registered to the management server within the preset time, which is not limited to this.
In another implementation manner, the management server may select the expansion server selected by the determined at least two expansion servers as an auxiliary server, where the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the respective expansion servers. This implementation is described in detail later for clarity of layout.
It should be noted that the storage performance parameters include, but are not limited to, a memory usage rate, a central processing unit usage rate, and a network card usage rate.
It is to be understood that, after selecting the auxiliary server from the determined at least two capacity expansion servers, the management server may instruct the auxiliary server to manage the at least two capacity expansion servers, that is, the auxiliary server obtains a management right for managing the at least two capacity expansion servers, where the management right includes data writing management for the at least two capacity expansion servers.
For example, when the management server selects a capacity expansion server a in a first registered capacity expansion device cluster (including a capacity expansion server a, a capacity expansion server B, and a capacity expansion server C) as an auxiliary server within a preset time period of 9:00 to 9:05, the management server may instruct the capacity expansion server a to manage itself, the capacity expansion server B, and the capacity expansion server C.
Similarly, when the management server selects the capacity expansion server D in the registered second capacity expansion device cluster (including the capacity expansion server D and the capacity expansion server E) as the auxiliary server within the preset time period of 10:00-15:05, the management server may instruct the capacity expansion server D to manage itself and the capacity expansion server E.
It should be noted that, in order to avoid the situation that the storage pressure of the capacity expansion server is instantaneously increased due to the fact that the management server writes data into the capacity expansion server crazy at the instant that the capacity expansion server registers in the management server, the management server does not write data into any capacity expansion server that is registered in the management server within the preset time before the auxiliary server is determined.
S103: when a data writing request is detected, writing data into the auxiliary server so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
it will be appreciated that in one implementation, the management server may write data to the secondary server upon receiving a write data request to write data to the secondary server. Wherein the write data request may be issued by the auxiliary server.
In another implementation manner, when the management server sends a write data request to the auxiliary server and when the auxiliary server serves as a receiver of the write data request, receives the write data request and responds to permission to write data, the management server writes data to the auxiliary server.
In addition, after the management server writes data into the auxiliary server, the auxiliary server receives the data written by the management server, and can write the received data into the at least two capacity expansion servers according to a load balancing principle, so that the data writing management of each capacity expansion server is realized. For clear layout, a detailed description is subsequently given of an implementation manner in which the auxiliary server performs write data management on each capacity expansion server.
S104: monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
s105: and when the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers.
It is to be understood that, during the period when the auxiliary server manages the at least two capacity expansion servers, the management server may monitor the storage space occupation ratio of the at least two capacity expansion servers through the auxiliary server. When it is monitored that the storage space occupation ratios of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers, so that the management server can directly manage the at least two capacity expansion servers, and capacity expansion of the cloud storage system is completed.
Wherein, the mode of the management server obtaining the management authority of the at least two capacity expansion servers includes: and obtaining index data corresponding to the data written by the auxiliary server to the at least two capacity expansion servers, and instructing the auxiliary server to close the function of managing the at least two capacity expansion servers. And after the auxiliary server closes the management function, only storing data as a storage server.
It should be noted that an absolute value of a difference between the first threshold and a storage space occupation ratio of each server in the cloud storage system is smaller than a preset threshold. The preset threshold may be set by a person skilled in the art according to actual conditions, and is not limited herein.
In addition, the storage space occupation ratio is a ratio of the data amount written in the capacity expansion server to the storage capacity of the capacity expansion server.
It should be noted that, in the embodiment of the present invention, the at least two expansion servers are all directed to the expansion server registered to the management server within the same preset time.
In the embodiment of the invention, the management server in the cloud storage system may select one expansion server as an auxiliary server from expansion servers registered to the management server within a preset time; before the auxiliary server is selected and obtained, data are not written into any capacity expansion server, so that the problem of surge of the instantaneous storage pressure of the capacity expansion server when the capacity expansion server is accessed to a cloud storage system is avoided; after the auxiliary server is selected and obtained, the auxiliary server is indicated to manage the at least two capacity expansion servers which are registered to the management server within the preset time; when a data writing request is detected, writing data into an auxiliary server, and stably writing the data into a capacity expansion server by the auxiliary server; the management server can also monitor the occupation proportion of the storage space of each capacity expansion server through the auxiliary server; when the auxiliary server monitors that the occupation proportion of the storage space of each expansion server reaches a first threshold value, the management server acquires the management authority of each expansion server, and at the moment, the management server can stably write data into the expansion servers, namely, the expansion servers are stably added into the cloud storage system.
The following further explains an implementation manner in which the management server selects the expansion server selected by the determined at least two expansion servers as the auxiliary server.
For example, after the management server registers to its own first capacity expansion device cluster (including the capacity expansion server a, the capacity expansion server B, and the capacity expansion server C) within a time period of 9:00 to 9:05, the management server may send information, such as the number of capacity expansion servers included in the first capacity expansion device cluster, and the IP addresses and storage performance parameters of the capacity expansion servers included in the first capacity expansion device cluster, to each capacity expansion server in the first capacity expansion device cluster, respectively.
After receiving the number 3 of the capacity expansion servers included in the first capacity expansion equipment cluster, the IP addresses of the capacity expansion servers a, B, and C, and the storage performance parameters of the capacity expansion servers a, B, and C sent by the management server, the capacity expansion server a may elect the capacity expansion server B as an auxiliary server according to the storage performance parameters of each capacity expansion server, and send a connection request to the capacity expansion server B according to the IP address of the capacity expansion server B, and send information such as its own working state and disk parameters to the capacity expansion server B.
According to the same pushing mode, the capacity expansion server B pushes itself to be an auxiliary server; and the capacity expansion server C elects the capacity expansion server B as an auxiliary server, sends a connection request to the capacity expansion server B according to the IP address of the capacity expansion server B, and sends information such as the working state of the capacity expansion server C and the disk parameters to the capacity expansion server B.
The capacity expansion server B detects that 2 connection requests are received, the capacity expansion server B can determine the capacity expansion servers pushed out by the capacity expansion server A, the capacity expansion server B and the capacity expansion server C, and reports the capacity expansion servers pushed out by the at least two capacity expansion servers to the management service, so that the management server can select the capacity expansion server B as an auxiliary server.
In the prior art, before the capacity expansion server is accessed to the cloud storage system, a technician needs to perform an initialization operation or a formatting operation on the capacity expansion server through a manual operation, and after the initialization operation or the formatting operation is finished, the capacity expansion server is registered in the cloud storage system, which needs to consume a lot of human resources. And would be very time consuming for a technician as the initialization and formatting operations often require a lot of time. Therefore, in an embodiment of the present invention, before the step of writing data to the auxiliary server, the method may further include:
and sending an initialization and/or formatting instruction to the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
It is to be understood that, after the capacity expansion servers are registered to themselves and before data is written to the auxiliary server, the management server may send an initialization instruction to the auxiliary server, so that, after receiving the initialization instruction, the auxiliary server may initialize the at least two capacity expansion servers based on the initialization instruction. For example, the disks of the at least two capacity expansion servers are virtualized into one disk, but is not limited to this.
In addition, before writing data into the auxiliary server, the management server may also send a format instruction to the auxiliary server, so that after receiving the format instruction, the auxiliary server may format the disks of the at least two expansion servers into the disk format of the cloud storage system based on the format instruction. For example: the disks of the at least two capacity expansion servers are divided into preset groups (i.e., preset block areas), which is not limited to this.
It should be noted that, after the initialization and/or formatting of the at least two capacity expansion servers are completed, the auxiliary server may send a write data request to the management server; and the management server writes data into the auxiliary server after receiving the data writing request.
The following describes in detail an implementation manner of the auxiliary server performing write data management on each capacity expansion server in the embodiment of the present invention.
In one implementation manner, the step of writing, by the auxiliary server, the received data into the at least two capacity expansion servers according to a load balancing principle may include:
the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
The auxiliary server may preset a first threshold and a second threshold, and set the second threshold to be smaller than the first threshold. The first threshold is already described above, and is not described herein.
It can be understood that, when the auxiliary server receives the data sent by the management server, the auxiliary server may write data to the auxiliary server itself and other capacity expansion servers; and detecting whether the occupation ratios of the storage spaces of the auxiliary server and the other capacity expansion servers reach a second threshold value or not in a real-time detection mode.
If the data do not reach the second threshold value, continuing to write the data into the capacity expansion servers which do not reach the second threshold value; when the storage pressures of the at least two capacity expansion servers reach the second threshold value, the storage pressures (namely the storage space occupation ratios) of the at least two capacity expansion servers reach the first balance, and at the moment, data are continuously written into the auxiliary server and the other capacity expansion servers; then, the auxiliary server detects whether the storage space occupation proportions of the auxiliary server and other capacity expansion servers reach a first threshold value, if not, the auxiliary server continuously writes data into the capacity expansion servers which do not reach the first threshold value; and when the first threshold value is reached, the storage pressures of the at least two capacity expansion servers reach second equalization, the storage pressures of the at least two capacity expansion servers are smoothly transited from the second threshold value to the first threshold value, and at the moment, the writing of data to the auxiliary server and other capacity expansion servers is stopped.
It should be noted that the other capacity expansion servers are: and all the capacity expansion servers except the auxiliary server in at least two capacity expansion services registered to the management server within a preset time.
It should be noted that, in order to make the storage pressure of the at least two expansion servers transition to the first threshold more gradually, it is reasonable that a third threshold, a fourth threshold, a fifth threshold, and the like may be preset. Wherein the third threshold, the fourth threshold, and the fifth threshold are all less than the first threshold.
In addition, when the at least two capacity expansion servers are abnormal, the occupation ratio of the storage space of the at least two servers is usually very high. Therefore, in order to detect whether the at least two capacity expansion servers are abnormal, a person skilled in the art may set an abnormal threshold according to actual conditions. And when the occupation ratio of the storage space of any one of the at least two capacity expansion servers exceeds the abnormal threshold value, determining that the capacity expansion server is abnormal and notifying a technician.
Wherein the anomaly threshold is greater than the first threshold, as will be understood by those skilled in the art.
The following further describes the capacity expansion method of the cloud storage system according to the embodiment of the present invention with reference to fig. 2.
Referring to fig. 2, a method for capacity expansion of a cloud storage system according to an embodiment of the present invention may include the following steps:
s1: starting at least two capacity expansion servers;
s2: the at least two expansion servers are registered to a management server of the cloud storage system within a preset time;
s3: the management server returns the number of the capacity expansion servers registered to the management server and the storage performance parameters of each capacity expansion server registered to the management server to each capacity expansion server in the at least two capacity expansion servers;
s4: the at least two capacity expansion servers select an auxiliary server according to the storage performance parameters of each capacity expansion server;
s5: reporting information such as the online state of the server and disk parameters to the auxiliary server by other capacity expansion servers except the auxiliary server;
s6: after receiving information such as online states and disk parameters reported by other capacity expansion devices, the auxiliary server initializes each capacity expansion server registered to the management server within the preset time;
s7: after the initialization operation is completed, the auxiliary server formats each capacity expansion server registered to the management server within the preset time;
s8: after the formatting operation is finished, the auxiliary server sends a data writing request to the management server;
s9: the management server writes data into the auxiliary server after receiving the data writing request;
s10: after the auxiliary server receives the data written by the management server, the auxiliary server writes the data into the auxiliary server and other capacity expansion servers;
s11: the auxiliary server detects whether the occupation ratios of the storage spaces of the auxiliary server and other capacity expansion servers reach a second threshold value;
s12: when the auxiliary server detects that the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value, the auxiliary server continues to write data into the auxiliary server and the other capacity expansion servers;
s13: the auxiliary server detects whether the occupation ratios of the storage spaces of the auxiliary server and other capacity expansion servers reach a first threshold value;
s14: and when the auxiliary server detects that the occupation ratios of the storage spaces of the auxiliary server and other capacity expansion servers reach a first threshold value, the auxiliary server transfers the management authority to the management server.
Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides a cloud storage system capacity expansion apparatus, where the apparatus is applied to a management server in a cloud storage system, and referring to fig. 3, the apparatus may include:
a determining unit 301, configured to determine at least two expansion servers registered to the management server within a preset time;
a selecting unit 302, configured to select an auxiliary server from the at least two capacity expansion servers;
an indicating unit 303, configured to instruct the auxiliary server to manage the at least two capacity expansion servers, where the management server does not write data to any capacity expansion server registered in the management server within the preset time before the auxiliary server is determined;
a data writing unit 304, configured to write data to the auxiliary server when a data writing request is detected, so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
a monitoring unit 305, configured to monitor a storage space occupation ratio of the at least two capacity expansion servers through the auxiliary server;
an obtaining unit 306, configured to obtain a management right for managing the at least two capacity expansion servers when it is monitored that the storage space occupation ratios of the at least two capacity expansion servers both reach a first threshold.
In the embodiment of the invention, the management server in the cloud storage system may select one expansion server as an auxiliary server from expansion servers registered to the management server within a preset time; before the auxiliary server is selected and obtained, data are not written into any capacity expansion server, so that the problem of surge of the instantaneous storage pressure of the capacity expansion server when the capacity expansion server is accessed to a cloud storage system is avoided; after the auxiliary server is selected and obtained, the auxiliary server is indicated to manage the at least two capacity expansion servers which are registered to the management server within the preset time; when a data writing request is detected, writing data into an auxiliary server, and stably writing the data into a capacity expansion server by the auxiliary server; the management server can also monitor the occupation proportion of the storage space of each capacity expansion server through the auxiliary server; when the auxiliary server monitors that the occupation proportion of the storage space of each expansion server reaches a first threshold value, the management server acquires the management authority of each expansion server, and at the moment, the management server can stably write data into the expansion servers, namely, the expansion servers are stably added into the cloud storage system.
Optionally, in an implementation manner, the selecting unit 302 is specifically configured to:
and determining a target capacity expansion server of the at least two capacity expansion servers as an auxiliary server, wherein the target capacity expansion server is a capacity expansion server which is registered to the management server first in the preset time.
Optionally, in another implementation manner, the selecting unit 302 is specifically configured to:
and selecting the expansion server selected by the at least two expansion servers as an auxiliary server, wherein the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the expansion servers.
Optionally, in an embodiment of the present invention, the apparatus further includes:
and the sending unit is used for sending an initialization and/or formatting instruction to the auxiliary server before the data writing unit writes data into the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
Optionally, after the data writing unit 304 writes data into the auxiliary server, the step of writing the received data into the at least two capacity expansion servers by the auxiliary server according to a load balancing principle may include:
the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
Corresponding to the above method and apparatus embodiments, an embodiment of the present invention further provides a cloud storage system, where the cloud storage system includes a management server and a plurality of capacity expansion servers, and the management server is configured to:
determining at least two capacity expansion servers which are registered to the management server within a preset time;
selecting an auxiliary server from the at least two capacity expansion servers, and instructing the auxiliary server to manage the at least two capacity expansion servers, wherein the management server does not write data into any capacity expansion server registered to the management server within the preset time before the auxiliary server is determined;
when a data writing request is detected, writing data into the auxiliary server so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
and when the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers.
Optionally, in an implementation manner, the management server is specifically configured to: and determining a target capacity expansion server of the at least two capacity expansion servers as an auxiliary server, wherein the target capacity expansion server is a capacity expansion server which is registered to the management server first in the preset time.
Optionally, in another implementation manner, the management server is specifically configured to: and selecting the expansion server selected by the at least two expansion servers as an auxiliary server, wherein the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the expansion servers.
Optionally, the management server is specifically configured to: before the step of writing data into the auxiliary server, sending an initialization and/or formatting instruction to the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
Optionally, the auxiliary server is specifically configured to: the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
The management server and the auxiliary server in the cloud storage system are further described with reference to fig. 4.
Referring to fig. 4, since a management server in a cloud storage system needs to manage storage servers in the system, the management server may include:
the first data writing module comprises a first receiving unit and a first forwarding unit. The first receiving unit is used for receiving data transmitted by a user, wherein the data comprises file data, picture data and video data; the first forwarding unit is used for forwarding data when receiving a scheduling instruction sent by the first data scheduling module;
a first data scheduling module, configured to schedule the data received by the first data writing module to a storage server in the cloud storage system, where the first data scheduling module includes: a write data unit 304 and a first monitor unit. The data writing unit 304 may write data into the storage server when detecting that the request for writing data into the storage server is detected and the first monitoring unit detects that the operating state of the storage server is an online state; the data writing unit 304 may also write data into the auxiliary server when detecting that the request for writing data into the auxiliary server is detected and the first monitoring unit detects that the operating state of the auxiliary server is an online state;
a first capacity expansion server management module, configured to manage a capacity expansion server registered in the management server within a preset time according to a determining unit 301, a selecting unit 302, an indicating unit 303, a monitoring unit 305, an obtaining unit 306, and a sending unit included in the first capacity expansion management module, where functions of the determining unit 301, the selecting unit 302, the indicating unit 303, the monitoring unit 305, the obtaining unit 306, and the sending unit are described in detail above and are not described herein again.
The auxiliary server may include:
a second data writing module, which includes a second receiving unit and a second forwarding unit; the second receiving unit is configured to: receiving data written by the data writing unit 304 in the first capacity expansion server management module; the second forwarding unit is used for forwarding the received data when receiving a scheduling instruction sent by the second data scheduling module;
a second data scheduling module, configured to schedule the data received by the second data writing module to a corresponding capacity expansion server, where the second data scheduling module includes: a target write data unit and a second monitoring unit; the target data writing unit may write data into the expansion server when detecting a data writing request for writing data into the expansion server and the second monitoring unit detects that the operation state of the expansion server is an online state;
the auxiliary server election module is used for electing an auxiliary server from the at least two capacity expansion servers;
and the second capacity expansion server management module is configured to receive the instruction sent by the first capacity expansion server management module, manage the second data write-in module, the data scheduling module and the auxiliary server election module according to the instruction, monitor a storage space occupation ratio of each capacity expansion server, and send a monitoring result to the monitoring unit 305 in the first capacity expansion server management module.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (11)
1. A cloud storage system capacity expansion method is applied to a management server in a cloud storage system, and comprises the following steps:
determining at least two capacity expansion servers which are registered to the management server within a preset time;
selecting an auxiliary server from the at least two capacity expansion servers, and instructing the auxiliary server to manage the at least two capacity expansion servers, wherein the management server does not write data into any capacity expansion server registered to the management server within the preset time before the auxiliary server is determined;
when a data writing request is detected, sending data to the auxiliary server so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
when it is monitored that the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers;
wherein, the acquiring the management authority of the at least two capacity expansion servers includes: and obtaining index data corresponding to the data written by the auxiliary server to the at least two capacity expansion servers, and instructing the auxiliary server to close the function of managing the at least two capacity expansion servers.
2. The method of claim 1, wherein the step of selecting the secondary server from the at least two capacity expansion servers comprises:
and determining a target capacity expansion server of the at least two capacity expansion servers as an auxiliary server, wherein the target capacity expansion server is a capacity expansion server which is registered to the management server first in the preset time.
3. The method of claim 1, wherein the step of selecting the secondary server from the at least two capacity expansion servers comprises:
and selecting the expansion server selected by the at least two expansion servers as an auxiliary server, wherein the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the expansion servers.
4. The method of any of claims 1-3, further comprising, prior to the step of writing data to the secondary server:
and sending an initialization and/or formatting instruction to the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
5. The method according to any one of claims 1 to 3, wherein the step of writing the received data to the at least two capacity expansion servers by the auxiliary server according to a load balancing principle comprises:
the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
6. The cloud storage system expansion device is applied to a management server in a cloud storage system, and comprises the following components:
the system comprises a determining unit, a judging unit and a processing unit, wherein the determining unit is used for determining at least two capacity expansion servers which are registered to the management server within preset time;
the selecting unit is used for selecting an auxiliary server from the at least two capacity expansion servers;
an indicating unit, configured to instruct the auxiliary server to manage the at least two capacity expansion servers, where the management server does not write data to any capacity expansion server registered in the management server within the preset time before the auxiliary server is determined;
the data writing unit is used for sending data to the auxiliary server when a data writing request is detected, so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
the monitoring unit is used for monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
the acquisition unit is used for acquiring management authority for managing the at least two capacity expansion servers when the storage space occupation proportion of the at least two capacity expansion servers reaches a first threshold value;
the obtaining unit is specifically configured to: and obtaining index data corresponding to the data written by the auxiliary server to the at least two capacity expansion servers, and instructing the auxiliary server to close the function of managing the at least two capacity expansion servers.
7. The apparatus according to claim 6, wherein the selecting unit is specifically configured to:
and determining a target capacity expansion server of the at least two capacity expansion servers as an auxiliary server, wherein the target capacity expansion server is a capacity expansion server which is registered to the management server first in the preset time.
8. The apparatus according to claim 6, wherein the selecting unit is specifically configured to:
and selecting the expansion server selected by the at least two expansion servers as an auxiliary server, wherein the auxiliary server is selected by the at least two expansion servers based on the received storage performance parameters of the expansion servers.
9. The apparatus according to any one of claims 6-8, further comprising:
and the sending unit is used for sending an initialization and/or formatting instruction to the auxiliary server before the data writing unit writes data into the auxiliary server, so that the auxiliary server initializes and/or formats each capacity expansion server according to the received initialization and/or formatting instruction.
10. The apparatus according to any one of claims 6 to 8, wherein after the data writing unit writes data to the secondary server, the step of the secondary server writing the received data to the at least two capacity expansion servers according to a load balancing principle includes:
the auxiliary server writes the received data into the auxiliary server and other capacity expansion servers, and detects whether the storage space occupation proportions of the auxiliary server and the other capacity expansion servers reach a second threshold value;
if not, continuing to write data into the capacity expansion server which does not reach the second threshold value;
if the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, data are written into the auxiliary server and other expansion servers, and data writing into the auxiliary server and other expansion servers is stopped until the storage space occupation proportions of the auxiliary server and other expansion servers reach the first threshold, wherein the second threshold is smaller than the first threshold.
11. A cloud storage system comprises a management server and a plurality of capacity expansion servers, wherein the management server is used for:
determining at least two capacity expansion servers which are registered to the management server within a preset time;
selecting an auxiliary server from the at least two capacity expansion servers, and instructing the auxiliary server to manage the at least two capacity expansion servers, wherein the management server does not write data into any capacity expansion server registered to the management server within the preset time before the auxiliary server is determined;
when a data writing request is detected, sending data to the auxiliary server so that the auxiliary server writes the received data into the at least two capacity expansion servers according to a load balancing principle;
monitoring the occupation proportion of the storage space of the at least two capacity expansion servers through the auxiliary server;
when it is monitored that the occupation ratios of the storage spaces of the at least two capacity expansion servers reach a first threshold value, acquiring management authority for managing the at least two capacity expansion servers;
wherein, the acquiring the management authority of the at least two capacity expansion servers includes: and obtaining index data corresponding to the data written by the auxiliary server to the at least two capacity expansion servers, and instructing the auxiliary server to close the function of managing the at least two capacity expansion servers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611066896.9A CN108111566B (en) | 2016-11-25 | 2016-11-25 | Cloud storage system capacity expansion method and device and cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611066896.9A CN108111566B (en) | 2016-11-25 | 2016-11-25 | Cloud storage system capacity expansion method and device and cloud storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108111566A CN108111566A (en) | 2018-06-01 |
| CN108111566B true CN108111566B (en) | 2020-11-06 |
Family
ID=62204678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611066896.9A Active CN108111566B (en) | 2016-11-25 | 2016-11-25 | Cloud storage system capacity expansion method and device and cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108111566B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112019577B (en) * | 2019-05-29 | 2023-04-07 | 中国移动通信集团重庆有限公司 | Exclusive cloud storage implementation method and device, computing equipment and computer storage medium |
| CN110333953A (en) * | 2019-07-11 | 2019-10-15 | 中国工商银行股份有限公司 | Data storage resource dispatching method and device |
| CN111258710B (en) * | 2020-01-09 | 2023-03-31 | 同盾控股有限公司 | System maintenance method and device |
| CN111208952B (en) * | 2020-02-13 | 2022-07-12 | 河南文正电子数据处理有限公司 | Storage system capacity expansion method, readable storage medium and computing device |
| CN113377781A (en) * | 2021-07-12 | 2021-09-10 | 中国工商银行股份有限公司 | Data storage method and device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102664914A (en) * | 2012-03-22 | 2012-09-12 | 北京英孚斯迈特信息技术有限公司 | IS/DFS-Image distributed file storage query system |
| CN103810244A (en) * | 2013-12-09 | 2014-05-21 | 北京理工大学 | Distributed data storage system expansion method based on data distribution |
| CN104023246A (en) * | 2014-04-28 | 2014-09-03 | 深圳英飞拓科技股份有限公司 | Private video data cloud-storage system and method |
| CN105162859A (en) * | 2015-08-20 | 2015-12-16 | 湖南亿谷科技发展股份有限公司 | Dynamic server dilatation system and method |
| CN105657066A (en) * | 2016-03-23 | 2016-06-08 | 天津书生云科技有限公司 | Load rebalance method and device used for storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070022129A1 (en) * | 2005-07-25 | 2007-01-25 | Parascale, Inc. | Rule driven automation of file placement, replication, and migration |
-
2016
- 2016-11-25 CN CN201611066896.9A patent/CN108111566B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102664914A (en) * | 2012-03-22 | 2012-09-12 | 北京英孚斯迈特信息技术有限公司 | IS/DFS-Image distributed file storage query system |
| CN103810244A (en) * | 2013-12-09 | 2014-05-21 | 北京理工大学 | Distributed data storage system expansion method based on data distribution |
| CN104023246A (en) * | 2014-04-28 | 2014-09-03 | 深圳英飞拓科技股份有限公司 | Private video data cloud-storage system and method |
| CN105162859A (en) * | 2015-08-20 | 2015-12-16 | 湖南亿谷科技发展股份有限公司 | Dynamic server dilatation system and method |
| CN105657066A (en) * | 2016-03-23 | 2016-06-08 | 天津书生云科技有限公司 | Load rebalance method and device used for storage system |
Non-Patent Citations (1)
| Title |
|---|
| 基于轮转部署的 ＲAID6 分布式存储系统扩容方案;高帅，等;《计算机应用与软件》;20160831;第121-125,189页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108111566A (en) | 2018-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108111566B (en) | Cloud storage system capacity expansion method and device and cloud storage system | |
| CN108737132B (en) | Alarm information processing method and device | |
| CN108023967B (en) | Data balancing method and device and management equipment in distributed storage system | |
| CN104079445A (en) | Distributed type pressure testing system and method | |
| CN109802986B (en) | Equipment management method, system, device and server | |
| CN109151028A (en) | A kind of distributed memory system disaster recovery method and device | |
| CA3030504A1 (en) | Blockchain network and task scheduling method therefor | |
| CN110244901B (en) | Task allocation method and device and distributed storage system | |
| WO2014194704A1 (en) | A grouping processing method and system | |
| CN102088490A (en) | Data storage method, device and system | |
| CN112965879A (en) | Data processing method and device, electronic equipment and readable storage medium | |
| CN109101371B (en) | Disaster recovery switching method and device | |
| CN111562884B (en) | Data storage method and device and electronic equipment | |
| CN107613023B (en) | Equipment connection method and device | |
| CN108459910A (en) | A kind of method and apparatus for deleting resource | |
| CN110209548B (en) | Service control method, system, electronic device and computer readable storage medium | |
| CN109558152A (en) | Flow switching method, device, medium and electronic equipment when system gray scale is issued | |
| US9479579B2 (en) | Grouping processing method and system | |
| CN111722783B (en) | Data storage method and device | |
| US11157381B2 (en) | Display control method and display control device | |
| KR20180047079A (en) | Method and apparatus for detrmining event level of monitoring result | |
| CN105337778A (en) | Internet of Things event-driven service management system | |
| CN111897657A (en) | Remote teaching method, scheduling equipment, server and electronic equipment | |
| CN111147556A (en) | Load balancing method, device, equipment and storage medium | |
| CN110502333B (en) | Access request processing method and cloud storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |