CN112905119A - Data writing control method, device and equipment of distributed storage system - Google Patents
Data writing control method, device and equipment of distributed storage system Download PDFInfo
- Publication number
- CN112905119A CN112905119A CN202110189184.0A CN202110189184A CN112905119A CN 112905119 A CN112905119 A CN 112905119A CN 202110189184 A CN202110189184 A CN 202110189184A CN 112905119 A CN112905119 A CN 112905119A
- Authority
- CN
- China
- Prior art keywords
- disk
- utilization rate
- capacity utilization
- data
- weight value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Abstract
The invention discloses a data write-in control method, a device, equipment and a computer readable storage medium of a distributed storage system, based on a mechanism that the probability of storing new data in a disk is positively correlated with the weight value of the disk, acquiring the capacity utilization rate of each disk in the distributed storage system by pre-establishing a corresponding relation that the weight value of the disk is in negative correlation with the capacity utilization rate of the disk, the weight value of the disk is adjusted according to the capacity utilization rate of the disk, the dynamic adjustment of the weight value of the disk is realized, the probability of storing new data is reduced when the capacity utilization rate of the disk is higher, when the utilization rate of the disk capacity is low, the probability of storing new data is improved, so that the storage balance of each disk is realized in the whole distributed storage system, and the problems of data storage interruption and data loss caused by the fact that partial disk capacity is full are avoided.
Description
Technical Field
The present invention relates to the field of distributed storage technologies, and in particular, to a data write control method, apparatus, device, and computer-readable storage medium for a distributed storage system.
Background
With the rapid development of cloud computing and big data technology in the development of modern society, production data accumulated in production and life also grow exponentially, and mass storage technology is becoming an indispensable part in the development of the internet. In the distributed storage system, as mass data requests are stored in the disks according to a certain crush rule, part of the disks in the cluster can reach the capacity level preferentially, and a near full state may occur, which may cause data storage interruption, thereby causing problems such as data loss.
Therefore, how to perform balanced storage in a distributed storage system is a technical problem to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a data writing control method, a data writing control device, data writing control equipment and a computer readable storage medium of a distributed storage system, which are used for realizing balanced storage of the distributed storage system.
To solve the above technical problem, the present invention provides a data write control method for a distributed storage system, including:
acquiring the capacity utilization rate of each disk in the distributed storage system;
adjusting the weight value of the disk according to the capacity utilization rate of the disk;
when new data needs to be stored, determining the storage priority of each disk according to the weight value of each disk, and then determining the storage position of the new data according to the storage priority of each disk;
wherein the weight value of the disk is negatively correlated with the capacity utilization of the disk; the storage priority of the disk is positively correlated with the weight value of the disk.
Optionally, the method further includes:
and when the capacity utilization rate of the magnetic disk reaches a second threshold value, carrying out data isolation on the magnetic disk to prohibit new data from being stored in the magnetic disk.
Optionally, when the capacity utilization of the disk reaches the second threshold, the method further includes:
checking the health state parameters of the disk;
and when the disk is determined to be in a healthy running state according to the health state parameters of the disk, calling a data reconstruction script to reconstruct the data of the disk to other disks.
Optionally, the invoking the data reconstruction script to reconstruct the data of the disk to other disks specifically includes:
determining the disk with the capacity utilization rate smaller than a third threshold as a target disk;
and after the data of the magnetic disk is reconstructed into a plurality of groups of data according to the data continuity, uniformly storing each group of data into the magnetic disk and each target magnetic disk.
Optionally, the method further includes:
updating a historical capacity utilization rate curve of the disk according to the capacity utilization rate of the disk;
predicting a time predicted value when the capacity utilization rate of the disk reaches a preset threshold value according to the historical capacity utilization rate curve;
correspondingly, the adjusting the weight value of the disk according to the capacity utilization rate of the disk specifically includes:
and adjusting the weight value of the disk according to the capacity utilization rate of the disk and the time predicted value.
Optionally, the method further includes:
and when a query instruction is received, displaying the historical capacity utilization rate curve on a query interface.
Optionally, the adjusting the weighted value of the disk according to the preset corresponding relationship between the disk capacity utilization rate and the disk weighted value includes:
and when the capacity utilization rate of the disk reaches a first threshold value, adjusting the weight value of the disk to a first weight value.
In order to solve the above technical problem, the present invention further provides a data write control device for a distributed storage system, including:
the acquisition unit is used for acquiring the capacity utilization rate of each disk in the distributed storage system;
the adjusting unit is used for adjusting the weight value of the disk according to the capacity utilization rate of the disk;
the control unit is used for determining the storage priority of each disk according to the weight value of each disk when new data needs to be stored, and then determining the storage position of the new data according to the storage priority of each disk;
wherein the weight value of the disk is negatively correlated with the capacity utilization of the disk; the storage priority of the disk is positively correlated with the weight value of the disk.
In order to solve the above technical problem, the present invention further provides a data write control device for a distributed storage system, including:
a memory for storing instructions, the instructions including the steps of the data write control method of any one of the above-mentioned distributed storage systems;
a processor to execute the instructions.
To solve the above technical problem, the present invention further provides a computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the steps of the data write control method of the distributed storage system according to any one of the above.
The data writing control method of the distributed storage system provided by the invention is based on a mechanism that the probability of storing new data in a disk is positively correlated with the weight value of the disk, obtains the capacity utilization rate of each disk in the distributed storage system by pre-establishing the corresponding relation that the weight value of the disk is negatively correlated with the capacity utilization rate of the disk, adjusts the weight value of the disk according to the capacity utilization rate of the disk, realizes dynamic adjustment of the weight value of the disk, reduces the probability of storing new data when the capacity utilization rate of the disk is higher, and improves the probability of storing new data when the capacity utilization rate of the disk is lower, thereby realizing storage balance of each disk in the whole distributed storage system, and avoiding the problems of data storage interruption and data loss caused by the full capacity of part of the disk.
The invention also provides a data write-in control device, equipment and a computer readable storage medium of the distributed storage system, which have the beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a flowchart of a data write control method of a distributed storage system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a data write control apparatus of a distributed storage system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a data write control device of a distributed storage system according to an embodiment of the present invention.
Detailed Description
The core of the invention is to provide a data write-in control method, a device, equipment and a computer readable storage medium of a distributed storage system, which are used for realizing the balanced storage of the distributed storage system.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of a data write control method of a distributed storage system according to an embodiment of the present invention.
As shown in fig. 1, a data write control method of a distributed storage system according to an embodiment of the present invention includes:
s101: and acquiring the capacity utilization rate of each disk in the distributed storage system.
S102: and adjusting the weight value of the disk according to the capacity utilization rate of the disk.
S103: when new data needs to be stored, determining the storage priority of each disk according to the weight value of each disk, and then determining the storage position of the new data according to the storage priority of each disk.
Wherein, the weight value of the disk is negatively correlated with the capacity utilization rate of the disk; the storage priority of the disk is positively correlated with the weight value of the disk.
In a specific implementation, for step S101, a certain node in the distributed storage system is selected to start a disk capacity information detection process, and capacity utilization rates of all disks in the cluster are detected. Specifically, the detection may be performed every fixed time, or different detection frequencies may be set in different time periods according to the storage habit of the user. And after the capacity utilization rate of each disk is obtained, storing the capacity utilization rate into a database according to a preset storage rule, regularly cleaning historical capacity utilization rate data, and reserving the historical capacity utilization rate data within a preset time length from the current moment onward.
For step S102, the weight (weight) value of the disk is used by the distributed storage system to determine the priority of the storage node when storing new data, that is, the storage node is preferentially stored in the disk with the higher weight value. In general, the weight value of each disk will not change after factory setting, which cannot adapt to the change of the disk utilization rate, and may cause data to be preferentially stored in the disk to be filled. Therefore, in the embodiment of the invention, the weight value of the disk is adaptively adjusted when the capacity utilization rate of the disk changes according to the regulation principle that the weight value of the disk is in negative correlation with the capacity utilization rate of the disk in advance. The functional relation between the weight value of the disk and the capacity utilization rate of the disk can be pre-established, and the threshold value of the capacity utilization rate of one or more disks can be set so as to adjust the weight value of the disk when the capacity utilization rate of the disk reaches the threshold value. Step S102 may specifically be:
and when the capacity utilization rate of the disk reaches a first threshold value, adjusting the weight value of the disk to the first weight value.
In practical applications, the first threshold may be one, for example, when the capacity utilization rate of the disk reaches 70%, the weight value of the disk is reduced, and then the incoming request data is preferentially stored in other disks in the distributed storage system, so as to reduce the data writing frequency to the disk. The number of the first threshold values may also be multiple, and the first weight values correspond to the first threshold values one to one. It is to be understood that as the first threshold increases, the first weight value decreases.
For step S103, when new data needs to be stored, the weight values of the disks are compared, and the disk with the higher weight value has the higher probability of storing the new data, and the storage location of the new data is determined by the balanced storage calculation based on the probability of storing the new data in each disk.
Further, the data write control method of the distributed storage system provided by the embodiment of the present invention may further include:
and when the capacity utilization rate of the disk reaches an alarm threshold, executing an alarm method corresponding to the alarm threshold.
In practical applications, the alarm threshold may be the same as the first threshold, or may be a different threshold. Multiple alarm thresholds may be set, corresponding to different alarm methods. The warning method may include, but is not limited to, notifying system management personnel of the information of the disk whose capacity utilization rate reaches the warning threshold by means of mail or short message.
The alarm threshold may include an abnormal alarm threshold and a normal alarm threshold, where the abnormal alarm threshold is a threshold when the disk capacity utilization rate is high (e.g., 70%), and the normal alarm threshold is a threshold when the disk capacity utilization rate is recovered to be normal.
The first threshold and the alarm threshold mentioned in the embodiment of the invention can be set by a system manager according to requirements.
The data write-in control method of the distributed storage system provided by the embodiment of the invention is based on a mechanism that the higher the weight value of a disk is, the higher the probability of storing new data is, the higher the weight value of the disk is, the higher the probability of storing new data is, the corresponding relation is established in advance, the capacity utilization rate of each disk in the distributed storage system is obtained at regular time, the weight value of the disk is adjusted according to the capacity utilization rate of the disk according to the corresponding relation, the dynamic adjustment of the weight value of the disk is realized, the probability of storing new data is reduced when the capacity utilization rate of the disk is higher, the probability of storing new data is improved when the capacity utilization rate of the disk is lower, the storage balance of each disk is realized in the whole distributed storage system, and the problems of data storage interruption and data loss caused by.
On the basis of the foregoing embodiment, to further ensure storage security, the data write control method for a distributed storage system according to an embodiment of the present invention further includes:
and when the capacity utilization rate of the disk reaches a second threshold value, carrying out data isolation on the disk to reject new data to be stored in the disk.
In a specific implementation, the second threshold is a limit value of the use of the disk, and when the capacity utilization rate of the disk reaches the second threshold, data isolation is performed on the disk to prohibit new data from being stored in the disk. The second threshold is specifically a threshold close to but less than the maximum capacity of the disk.
Further, when the capacity utilization rate of the disk reaches the second threshold, the data write control method of the distributed storage system provided in the embodiment of the present invention may further include:
checking the health state parameters of the magnetic disk;
and when the disk is determined to be in a healthy running state according to the health state parameters of the disk, calling a data reconstruction script to reconstruct the data of the disk to other disks.
When the capacity utilization rate of the disk reaches a second threshold value, under the condition that the disk is in a healthy running state and the data is not damaged, the data of the disk is reconstructed to other disks by using the data reconstruction function of the distributed storage system so as to ensure storage balance.
The invoking of the data reconstruction script to reconstruct the data of the disk to other disks may specifically include:
determining the disk with the capacity utilization rate smaller than a third threshold as a target disk;
and after the data of the disk is reconstructed into a plurality of groups of data according to the data continuity, uniformly storing each group of data into the disk and each target disk.
In practical application, the disks can be arranged according to the sequence of the capacity utilization rate of the disks from small to large, and the first disks are selected as target disks; the third threshold value can also be used as a reference for selecting the target disk. And on the premise of ensuring the data continuity, reconstructing the data of the disk with the capacity utilization rate reaching the second threshold value and in the healthy running state into multiple groups of data, so that a part of data is left in the original disk, and other groups of data are uniformly stored in each target disk to realize data reconstruction.
On the basis of the foregoing embodiment, the data write control method for a distributed storage system according to an embodiment of the present invention further includes:
updating a historical capacity utilization rate curve of the disk according to the capacity utilization rate of the disk;
and predicting a time predicted value when the capacity utilization rate of the disk reaches a preset threshold value according to the historical capacity utilization rate curve.
Correspondingly, in step S102, the weight value of the disk is adjusted according to the capacity utilization rate of the disk, specifically:
and adjusting the weight value of the disk according to the capacity utilization rate and the time predicted value of the disk.
In order to further ensure storage balance, a historical capacity utilization rate curve of the disk is established according to the capacity utilization rate of the disk in each detection, and the historical capacity utilization rate curve is updated after the capacity utilization rate of the disk is obtained each time. The updating mode may be to extend the historical capacity utilization rate curve, or may include deleting the curve data that is a preset time before the current time advances.
The preset threshold may be the maximum capacity utilization rate of the disk, or may be a value smaller than the maximum capacity utilization rate of the disk, so as to adjust the weight value of the disk in a targeted manner according to the increase rate of the capacity utilization rate of the disk. It can be understood that, under the same capacity utilization rate of the disk, the smaller the time predicted value is, that is, the faster the speed of predicting that the disk reaches the preset threshold value is, the smaller the weight value of the disk is reduced.
Further, the data write control method of the distributed storage system provided by the embodiment of the present invention may further include:
and when a query instruction is received, displaying a historical capacity utilization rate curve on a query interface.
When the distributed storage system normally operates and a query instruction is received, displaying a historical capacity utilization rate curve on a query interface. In addition, a historical capacity utilization rate curve and other parameters related to the disk capacity utilization rate can be automatically displayed on the query interface at regular time.
On the basis of the above detailed description of the embodiments corresponding to the data writing control method of the distributed storage system, the invention also discloses a data writing control device, equipment and a computer readable storage medium of the distributed storage system corresponding to the method.
Fig. 2 is a schematic structural diagram of a data write control device of a distributed storage system according to an embodiment of the present invention.
As shown in fig. 2, a data write control apparatus of a distributed storage system according to an embodiment of the present invention includes:
an obtaining unit 201, configured to obtain a capacity utilization rate of each disk in the distributed storage system;
the adjusting unit 202 is configured to adjust a weight value of the disk according to a capacity utilization rate of the disk;
the control unit 203 is used for determining the storage priority of each disk according to the weight value of each disk when new data needs to be stored, and then determining the storage position of the new data according to the storage priority of each disk;
wherein, the weight value of the disk is negatively correlated with the capacity utilization rate of the disk; the storage priority of the disk is positively correlated with the weight value of the disk.
Further, the data write control apparatus of the distributed storage system provided in the embodiment of the present invention may further include:
and the isolation unit is used for carrying out data isolation on the magnetic disk to prohibit new data from being stored in the magnetic disk when the capacity utilization rate of the magnetic disk reaches a second threshold value.
Further, the data write control apparatus of the distributed storage system provided in the embodiment of the present invention may further include:
the reconstruction unit is used for checking the health state parameters of the disk when the capacity utilization rate of the disk reaches a second threshold value; and when the disk is determined to be in a healthy running state according to the health state parameters of the disk, calling a data reconstruction script to reconstruct the data of the disk to other disks.
Further, the data write control apparatus of the distributed storage system provided in the embodiment of the present invention may further include:
the drawing unit is used for updating the historical capacity utilization rate curve of the disk according to the capacity utilization rate of the disk;
the prediction unit is used for predicting a time prediction value of the disk reaching a full-writing state according to the historical capacity utilization rate curve;
correspondingly, the adjusting unit 202 adjusts the weight value of the disk according to the capacity utilization rate of the disk, specifically:
and adjusting the weight value of the disk according to the capacity utilization rate and the time predicted value of the disk.
Further, the data write control apparatus of the distributed storage system provided in the embodiment of the present invention may further include:
and the display unit is used for displaying the historical capacity utilization rate curve on the query interface when receiving the query instruction.
Further, the data write control apparatus of the distributed storage system provided in the embodiment of the present invention may further include:
and the alarm unit is used for executing an alarm method corresponding to the alarm threshold when the capacity utilization rate of the disk reaches the alarm threshold.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
Fig. 3 is a schematic structural diagram of a data write control device of a distributed storage system according to an embodiment of the present invention.
As shown in fig. 3, a data write control device of a distributed storage system according to an embodiment of the present invention includes:
a memory 310 for storing instructions, the instructions including the steps of the data write control method of the distributed storage system according to any one of the above embodiments;
a processor 320 for executing the instructions.
Processor 320 may include one or more processing cores, such as a 3-core processor, an 8-core processor, and so forth. The processor 320 may be implemented in at least one hardware form of a digital Signal processing (dsp), a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (pla), or a digital Signal processing (dsp). The processor 320 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a central Processing unit (cpu); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 320 may be integrated with an image processor GPU (graphics Processing unit), which is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, processor 320 may also include an Artificial Intelligence (AI) (artificial intelligence) processor for processing computational operations related to machine learning.
Memory 310 may include one or more computer-readable storage media, which may be non-transitory. Memory 310 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 310 is at least used for storing a computer program 311, wherein after the computer program 311 is loaded and executed by the processor 320, the relevant steps in the data writing control method of the distributed storage system disclosed in any one of the foregoing embodiments can be implemented. In addition, the resources stored by the memory 310 may also include an operating system 312, data 313, and the like, and the storage may be transient storage or persistent storage. The operating system 312 may be Windows, among others. Data 313 may include, but is not limited to, data involved in the above-described methods.
In some embodiments, the data write control device of the distributed storage system may further include a display 330, a power supply 340, a communication interface 350, an input output interface 360, a sensor 370, and a communication bus 380.
Those skilled in the art will appreciate that the architecture shown in fig. 3 does not constitute a limitation of the data write control apparatus of the distributed storage system and may include more or fewer components than those shown.
The data writing control device of the distributed storage system provided by the embodiment of the application comprises a memory and a processor, and when the processor executes a program stored in the memory, the data writing control method of the distributed storage system can be realized, and the effect is the same as that of the data writing control device.
It should be noted that the above-described embodiments of the apparatus and device are merely illustrative, for example, the division of modules is only one division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form. Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and performs all or part of the steps of the methods according to the embodiments of the present invention, or all or part of the technical solution.
To this end, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the data write control method, such as a distributed storage system.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory ROM (Read-Only Memory), a random Access Memory ram (random Access Memory), a magnetic disk, or an optical disk.
The computer program contained in the computer-readable storage medium provided in this embodiment can implement the steps of the data write control method of the distributed storage system described above when executed by the processor, and the effects are the same as above.
The data write control method, device, equipment and computer readable storage medium of the distributed storage system provided by the invention are described in detail above. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device, the apparatus and the computer-readable storage medium disclosed in the embodiments correspond to the method disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A data write control method for a distributed storage system, comprising:
acquiring the capacity utilization rate of each disk in the distributed storage system;
adjusting the weight value of the disk according to the capacity utilization rate of the disk;
when new data needs to be stored, determining the storage priority of each disk according to the weight value of each disk, and then determining the storage position of the new data according to the storage priority of each disk;
wherein the weight value of the disk is negatively correlated with the capacity utilization of the disk; the storage priority of the disk is positively correlated with the weight value of the disk.
2. The data write control method according to claim 1, further comprising:
and when the capacity utilization rate of the magnetic disk reaches a second threshold value, carrying out data isolation on the magnetic disk to prohibit new data from being stored in the magnetic disk.
3. The data write control method according to claim 2, when the capacity utilization rate of the disk reaches a second threshold, further comprising:
checking the health state parameters of the disk;
and when the disk is determined to be in a healthy running state according to the health state parameters of the disk, calling a data reconstruction script to reconstruct the data of the disk to other disks.
4. The data write control method according to claim 2, wherein the invoking a data reconstruction script to reconstruct the data of the disk to another disk specifically includes:
determining the disk with the capacity utilization rate smaller than a third threshold as a target disk;
and after the data of the magnetic disk is reconstructed into a plurality of groups of data according to the data continuity, uniformly storing each group of data into the magnetic disk and each target magnetic disk.
5. The data write control method according to claim 1, further comprising:
updating a historical capacity utilization rate curve of the disk according to the capacity utilization rate of the disk;
predicting a time predicted value when the capacity utilization rate of the disk reaches a preset threshold value according to the historical capacity utilization rate curve;
correspondingly, the adjusting the weight value of the disk according to the capacity utilization rate of the disk specifically includes:
and adjusting the weight value of the disk according to the capacity utilization rate of the disk and the time predicted value.
6. The data write control method according to claim 5, characterized by further comprising:
and when a query instruction is received, displaying the historical capacity utilization rate curve on a query interface.
7. The data write-in control method according to claim 1, wherein the adjusting the weight value of the disk according to the preset correspondence between the disk capacity utilization rate and the disk weight value according to the capacity utilization rate of the disk specifically includes:
and when the capacity utilization rate of the disk reaches a first threshold value, adjusting the weight value of the disk to a first weight value.
8. A data write control apparatus of a distributed storage system, comprising:
the acquisition unit is used for acquiring the capacity utilization rate of each disk in the distributed storage system;
the adjusting unit is used for adjusting the weight value of the disk according to the capacity utilization rate of the disk;
the control unit is used for determining the storage priority of each disk according to the weight value of each disk when new data needs to be stored, and then determining the storage position of the new data according to the storage priority of each disk;
wherein the weight value of the disk is negatively correlated with the capacity utilization of the disk; the storage priority of the disk is positively correlated with the weight value of the disk.
9. A data write control apparatus of a distributed storage system, characterized by comprising:
a memory for storing instructions, the instructions comprising the steps of the data write control method of the distributed storage system of any one of claims 1 to 7;
a processor to execute the instructions.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of a data write control method of a distributed storage system according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110189184.0A CN112905119B (en) | 2021-02-19 | 2021-02-19 | Data write-in control method, device and equipment of distributed storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110189184.0A CN112905119B (en) | 2021-02-19 | 2021-02-19 | Data write-in control method, device and equipment of distributed storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112905119A true CN112905119A (en) | 2021-06-04 |
| CN112905119B CN112905119B (en) | 2022-10-28 |
Family
ID=76123832
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110189184.0A Active CN112905119B (en) | 2021-02-19 | 2021-02-19 | Data write-in control method, device and equipment of distributed storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112905119B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114741220A (en) * | 2022-03-31 | 2022-07-12 | 北京志凌海纳科技有限公司 | Disk isolation method, system, device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190196738A1 (en) * | 2016-12-29 | 2019-06-27 | Huawei Technologies Co., Ltd. | Data migration method for a storage system after expansion and storage system |
| CN110018932A (en) * | 2019-03-26 | 2019-07-16 | 中国联合网络通信集团有限公司 | A kind of monitoring method and device of container disk |
| US20200133811A1 (en) * | 2018-10-31 | 2020-04-30 | EMC IP Holding Company LLC | Method, device, and computer program product for rebuilding data |
| CN111324304A (en) * | 2020-02-14 | 2020-06-23 | 西安奥卡云数据科技有限公司 | Data protection method and device based on SSD hard disk life prediction |
| CN111562884A (en) * | 2020-04-28 | 2020-08-21 | 北京奇艺世纪科技有限公司 | Data storage method and device and electronic equipment |
| CN111736772A (en) * | 2020-06-15 | 2020-10-02 | 中国工商银行股份有限公司 | Storage space data processing method and device of distributed file system |
-
2021
- 2021-02-19 CN CN202110189184.0A patent/CN112905119B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190196738A1 (en) * | 2016-12-29 | 2019-06-27 | Huawei Technologies Co., Ltd. | Data migration method for a storage system after expansion and storage system |
| US20200133811A1 (en) * | 2018-10-31 | 2020-04-30 | EMC IP Holding Company LLC | Method, device, and computer program product for rebuilding data |
| CN110018932A (en) * | 2019-03-26 | 2019-07-16 | 中国联合网络通信集团有限公司 | A kind of monitoring method and device of container disk |
| CN111324304A (en) * | 2020-02-14 | 2020-06-23 | 西安奥卡云数据科技有限公司 | Data protection method and device based on SSD hard disk life prediction |
| CN111562884A (en) * | 2020-04-28 | 2020-08-21 | 北京奇艺世纪科技有限公司 | Data storage method and device and electronic equipment |
| CN111736772A (en) * | 2020-06-15 | 2020-10-02 | 中国工商银行股份有限公司 | Storage space data processing method and device of distributed file system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114741220A (en) * | 2022-03-31 | 2022-07-12 | 北京志凌海纳科技有限公司 | Disk isolation method, system, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112905119B (en) | 2022-10-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3044156C (en) | Adaptive service timeouts | |
| CN111143039A (en) | Virtual machine scheduling method and device and computer storage medium | |
| CN112650575A (en) | Resource scheduling method and device and cloud service system | |
| CN110515917B (en) | Method, device and medium for controlling reconstruction speed | |
| CN115576505A (en) | Data storage method, device and equipment and readable storage medium | |
| CN112905119B (en) | Data write-in control method, device and equipment of distributed storage system | |
| CN111953569B (en) | State information reporting method, device, equipment and medium | |
| CN113778644A (en) | Task processing method, device, equipment and storage medium | |
| CN111367471B (en) | Speed limiting method, device, equipment and medium for write-in cache | |
| US10664952B2 (en) | Image processing method, and device, for performing coordinate conversion | |
| CN112817687A (en) | Data synchronization method and device | |
| CN114995770B (en) | Data processing method, device, equipment, system and readable storage medium | |
| CN115718732A (en) | Disk file management method, device, equipment and storage medium | |
| CN115437572A (en) | Data dropping method, device, equipment and medium | |
| CN115309735A (en) | Big data cleaning method and device, computer equipment and storage medium | |
| CN114721876A (en) | Data backup method, device and medium | |
| CN111124754A (en) | Data recovery method, device, equipment and medium | |
| CN111090633A (en) | Small file aggregation method, device and equipment of distributed file system | |
| CN113703678A (en) | Method, device, equipment and medium for re-splitting index of storage bucket | |
| CN113032369A (en) | Data migration method, device and medium | |
| CN110955644A (en) | IO control method, device, equipment and storage medium of storage system | |
| CN114443767B (en) | Method, device, equipment and medium for determining consistency level of distributed system | |
| CN115543209A (en) | Method for monitoring execution speed of SDS storage domain and related components | |
| CN115562815A (en) | Tenant resource processing method, device and medium | |
| CN111143073B (en) | Virtualized resource management method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |