CN114153399B - Data storage method, device, control device and medium of storage system - Google Patents
Data storage method, device, control device and medium of storage system Download PDFInfo
- Publication number
- CN114153399B CN114153399B CN202111484720.6A CN202111484720A CN114153399B CN 114153399 B CN114153399 B CN 114153399B CN 202111484720 A CN202111484720 A CN 202111484720A CN 114153399 B CN114153399 B CN 114153399B
- Authority
- CN
- China
- Prior art keywords
- storage
- coverage period
- pool
- data
- storage space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 60
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 88
- 238000004590 computer program Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0617—Improving the reliability of storage systems in relation to availability
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data storage, in particular to a data storage method, a device, a control device and a medium of a storage system, which aim to solve the problems that the resource utilization rate of the storage system is low or the service of the storage system is unavailable due to improper cyclic coverage period setting of a storage pool. For this purpose, the invention obtains the storage space required by storing the service data in a cyclic coverage mode, and adjusts the preset cyclic coverage period of the storage pool according to the required storage space and the residual storage space of the storage pool, so as to realize that the storage pool stores the service data in a cyclic coverage mode according to the adjusted preset cyclic coverage period. By the configuration mode, the method and the device can comprehensively consider the influence of the factors of the residual storage space of the storage pool and the preset cycle coverage period on the process of storing the service data in the storage pool in the cycle coverage mode, improve the resource utilization rate of the storage pool and ensure the availability of storage service.
Description
Technical Field
The invention relates to the technical field of data storage, and particularly provides a data storage method, device, control device and medium of a storage system.
Background
When a storage system is used for storing the data such as pictures and videos, parameters are generally configured according to the query period of the data, and the cycle coverage period of each storage pool in the storage system is set. When the storage time of the stored data reaches the cycle coverage period, the storage system stores new data and covers the original data, so that the cycle utilization of the storage space in the storage pool is realized, and the new data can be always written into the corresponding storage pool. However, in practical application, if the cycle coverage period of the storage pool is set to be shorter, the storage space cannot be fully utilized, so that the storage resource utilization rate is low, and the storage is wasted. Meanwhile, if the cycle coverage period of the storage pool is set longer, the storage space may be fully occupied, so that some storage services are not available, new storage data cannot be written into the storage pool, and further the situation of loss of the storage data is caused.
Accordingly, there is a need in the art for a new data storage scheme for a storage system that addresses the above-described problems.
Disclosure of Invention
The present invention has been made to overcome the above-mentioned drawbacks, and to provide a solution or at least partially solve the problem of low utilization of resources of a storage system or unavailability of storage system services due to improper cyclic coverage period settings of a storage pool.
In a first aspect, the present invention provides a data storage method of a storage system, the storage system comprising a plurality of storage pools, each storage pool being configured to store traffic data of a different data type, the method comprising:
acquiring a data type designated by a data storage request and a designated cycle coverage period, and determining the storage speed of a storage pool according to the total storage space of the storage pool corresponding to the data type and the designated cycle coverage period;
determining a storage space required when the service data of the data type are stored in a cyclic coverage mode according to the storage speed and a preset cyclic coverage period of the storage pool;
according to the required storage space and the residual storage space of the storage pool, adjusting a preset cycle coverage period of the storage pool;
and controlling the storage pool to store the service data of the data type in a cyclic coverage mode according to the storage speed and the adjusted preset cyclic coverage period.
In one technical scheme of the data storage method of the storage system, the step of adjusting the preset cycle coverage period of the storage pool according to the required storage space and the remaining storage space of the storage pool specifically includes:
and if the required storage space is larger than the residual storage space, acquiring a new cycle coverage period according to the storage speed and the residual storage space, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period.
In one technical scheme of the data storage method of the storage system, the step of acquiring a new cycle coverage period according to the storage speed and the remaining storage space, and adjusting the preset cycle coverage period of the storage pool to the new cycle coverage period specifically includes:
judging whether other storage pools with the same specified cyclic coverage period as the current storage pool exist in all the storage pools with the required storage space larger than the residual storage space aiming at the storage pools with the required storage space larger than the residual storage space;
if so, calculating a new cycle coverage period according to the method shown in the following formula, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period:
wherein x represents a new cyclic coverage period, B 1 Representing the storage speed corresponding to the current storage pool, B i And (3) representing the storage speed corresponding to the ith other storage pool, wherein n represents the total number of the current storage pool and the other storage pools, and A represents the total sum of the current storage pool and the residual storage space of the other storage pools.
In one technical scheme of the data storage method of the storage system, the step of adjusting the preset cycle coverage period of the storage pool according to the required storage space and the remaining storage space of the storage pool specifically includes:
if the required storage space is smaller than or equal to the residual storage space, acquiring an upper limit storage space of the storage pool, wherein the upper limit storage space is larger than or equal to the required storage space;
and acquiring a new cycle coverage period according to the upper limit storage space and the storage speed, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period.
In one technical scheme of the data storage method of the storage system, the storage system comprises at least one storage node, each storage node is respectively provided with at least one storage pool, the storage node is configured to respond to a received data storage request, and control the storage pool corresponding to the data type according to the data type specified in the data storage request to store the service data of the data type in a cyclic coverage mode;
before the step of adjusting the preset cycle coverage period of the storage pool according to the required storage space and the remaining storage space of the storage pool, the method further includes:
and for each storage pool, if the storage space utilization rate of the current storage pool is smaller than a preset utilization rate threshold value and the number of data storage requests responded by the storage nodes corresponding to the current storage pool is larger than a preset number threshold value, not adjusting the preset cycle coverage period of the current storage pool.
In a second aspect, the present invention provides a data storage device of a storage system, the storage system comprising a plurality of storage pools, each storage pool being configured to store traffic data of a different data type, the device comprising:
the storage speed acquisition module is configured to acquire a data type designated by a data storage request and a designated cycle coverage period, and determine the storage speed of the storage pool according to the total storage space of the storage pool corresponding to the data type and the designated cycle coverage period;
a storage space acquisition module configured to determine a storage space required when storing the service data of the data type in a cyclic coverage manner in accordance with the storage speed and a preset cyclic coverage period of the storage pool;
the cycle coverage period adjustment module is configured to adjust a preset cycle coverage period of the storage pool according to the required storage space and the residual storage space of the storage pool;
and the service data storage module is configured to control the storage pool to store the service data of the data type in a cyclic coverage mode according to the storage speed and the adjusted preset cyclic coverage period.
In one aspect of the data storage device of the above storage system, the cycle coverage period adjustment module includes:
and the first cycle coverage period adjustment unit is configured to acquire a new cycle coverage period according to the storage speed and the residual storage space if the required storage space is larger than the residual storage space, and adjust the preset cycle coverage period of the storage pool to the new cycle coverage period.
In one aspect of the data storage device of the above storage system, the first cycle coverage period adjustment unit is further configured to perform the steps of:
judging whether other storage pools with the same specified cyclic coverage period as the current storage pool exist in all the storage pools with the required storage space larger than the residual storage space aiming at the storage pools with the required storage space larger than the residual storage space;
if so, calculating a new cycle coverage period according to the method shown in the following formula, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period:
wherein x represents a new cyclic coverage period, B 1 Representing the storage speed corresponding to the current storage pool, B i Indicating the storage speed corresponding to the ith other storage pool, n indicating the total number of the current storage pool and the other storage pool, and A indicating the surplus of the current storage pool and the other storage poolSum of the remaining storage space.
In one aspect of the data storage device of the above storage system, the cycle coverage period adjustment module includes a second cycle coverage period adjustment unit configured to perform the steps of:
if the required storage space is smaller than or equal to the residual storage space, acquiring an upper limit storage space of the storage pool, wherein the upper limit storage space is larger than or equal to the required storage space;
and acquiring a new cycle coverage period according to the upper limit storage space and the storage speed, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period.
In one technical scheme of the data storage device of the storage system, the storage system comprises at least one storage node, each storage node is respectively provided with at least one storage pool, the storage node is configured to respond to a received data storage request, and control the storage pool corresponding to the data type according to the data type specified in the data storage request to store the service data of the data type in a cyclic coverage mode;
the apparatus further comprises:
the cycle coverage period judging module is configured to not adjust the preset cycle coverage period of the current storage pool if the storage space utilization rate of the current storage pool is smaller than a preset utilization rate threshold value and the number of the data storage requests responded by the storage nodes corresponding to the current storage pool is larger than a preset number threshold value for each storage pool.
In a third aspect, a control device is provided, the control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and executed by the processor to perform the data storage method of the storage system according to any one of the technical aspects of the data storage method of the storage system.
In a fourth aspect, a computer readable storage medium is provided, in which a plurality of program codes are stored, the program codes being adapted to be loaded and executed by a processor to perform the data storage method of the storage system according to any one of the technical aspects of the data storage method of the storage system.
The technical scheme provided by the invention has at least one or more of the following beneficial effects:
in the technical scheme of implementing the invention, the storage space required by storing the service data in a cyclic coverage mode can be obtained according to the storage speed of the storage pool and the preset cyclic coverage period, and the preset cyclic coverage period of the storage pool is adjusted according to the required storage space and the residual storage space of the storage pool, so that the storage pool can store the service data in a cyclic coverage mode according to the adjusted preset cyclic coverage period. By the configuration mode, the influence of the factors of the remaining storage space and the preset cycle coverage period of the storage pool on the process of storing the service data in the storage pool in the cycle coverage mode can be comprehensively considered, the preset cycle coverage period of the storage pool is dynamically adjusted according to the condition of actually storing the service data, the service data is stored according to the adjusted preset cycle coverage period, the resource utilization rate of the storage pool can be improved, and the availability of the storage service of the storage pool is ensured.
Drawings
The present disclosure will become more readily understood with reference to the accompanying drawings. As will be readily appreciated by those skilled in the art: the drawings are for illustrative purposes only and are not intended to limit the scope of the present invention. Wherein:
FIG. 1 is a flow chart of the main steps of a data storage method of a storage system according to one embodiment of the present invention;
FIG. 2 is a flow chart illustrating the main steps of storing service data in a cyclic overlay manner in a storage pool according to an embodiment of the present invention;
fig. 3 is a main structural block diagram of a data storage device of a storage system according to an embodiment of the present invention.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module," "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, or software components, such as program code, or a combination of software and hardware. The processor may be a central processor, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware, or a combination of both. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "a and/or B" means all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and/or B" and may include A alone, B alone or A and B. The singular forms "a", "an" and "the" include plural referents.
Referring to fig. 1, fig. 1 is a schematic flow chart of main steps of a data storage method of a storage system according to an embodiment of the present invention. As shown in fig. 1, in an embodiment of the present invention, a storage system may include a plurality of storage pools, each of which may be configured to store service data of a different data type, and a data storage method of the storage system mainly includes the following steps S101 to S104.
Step S101: and acquiring the data type and the specified cycle coverage period specified by the data storage request, and determining the storage speed of the storage pool according to the total storage space of the storage pool corresponding to the data type and the specified cycle coverage period.
In this embodiment, the storage pool may be configured to store traffic data of one data type, and may obtain the data type specified by the data storage request and the specified cycle coverage period. The storage speed of the storage pool may be further determined according to the total storage space of the storage pool corresponding to the data type and the specified cycle coverage period. The cycle coverage period refers to a preset duration of storing service data when the storage pool stores the service data in a cycle coverage mode, and when the storing duration of the service data exceeds the preset duration, the storage pool stores new service data to cover the service data with the storing duration exceeding the preset duration. The specified cycle coverage period refers to a cycle coverage period of the service data specified by the user when the service data is stored.
Step S102: and determining the storage space required when the service data of the data type is stored in a cyclic coverage mode according to the storage speed and the preset cyclic coverage period of the storage pool.
In this embodiment, the storage space required when storing the service data of the data type corresponding to the storage pool in the cyclic coverage manner may be determined according to the storage speed obtained in step S101 and the preset cyclic coverage period of the storage pool.
Step S103: and adjusting the preset cycle coverage period of the storage pool according to the required storage space and the residual storage space of the storage pool. The preset cycle coverage period refers to a cycle coverage period preset by the storage pool when service data is stored.
In this embodiment, the preset cycle coverage period of the storage pool may be adjusted according to the required storage space and the remaining storage space of the storage pool.
Step S104: and controlling the storage pool to store the service data of the data type in a cyclic coverage mode according to the storage speed and the adjusted preset cyclic coverage period.
In this embodiment, the storage pool may be controlled to store the service data of the corresponding data type in a cyclic coverage manner according to the storage speed obtained in step S101 and the adjusted preset cyclic coverage period obtained in step S103.
Based on the steps S101 to S104, the present invention can obtain a storage space required for storing service data in a cyclic coverage manner according to a storage speed of a storage pool and a preset cyclic coverage period, and adjust the preset cyclic coverage period of the storage pool according to the required storage space and a remaining storage space of the storage pool, so as to implement that the storage pool stores service data in a cyclic coverage manner according to the adjusted preset cyclic coverage period. By the configuration mode, the influence of the factors of the remaining storage space and the preset cycle coverage period of the storage pool on the process of storing the service data in the storage pool in the cycle coverage mode can be comprehensively considered, the preset cycle coverage period of the storage pool is dynamically adjusted according to the condition of actually storing the service data, the service data is stored according to the adjusted preset cycle coverage period, the resource utilization rate of the storage pool can be improved, and the availability of the storage service of the storage pool is ensured.
Step S103 is further described below.
In one implementation of the embodiment of the present invention, step S103 may further include:
step S1031: and if the required storage space is larger than the residual storage space, acquiring a new cycle coverage period according to the storage speed and the residual storage space, and adjusting the preset cycle coverage period of the storage pool into the new cycle coverage period.
In this embodiment, when the storage space required for storing the service data of the corresponding data type in the cyclic coverage manner is larger than the remaining storage space of the storage pool, a new cyclic coverage period may be acquired according to the storage speed of the storage pool and the remaining storage space of the storage pool, and the preset cyclic coverage period of the storage pool may be adjusted to the new cyclic coverage period.
In one embodiment, the new cyclic coverage period may be calculated according to the method shown in the following equation (1):
where x represents the new cycle coverage period, a represents the remaining storage space of the storage pool, and b represents the storage speed of the storage pool.
After the new cycle coverage period is obtained through the calculation of the formula (1), the preset cycle coverage period of the storage pool can be adjusted to be the new cycle coverage period.
In one embodiment, step S1031 may further include:
step S10311: judging whether other storage pools with the same specified cyclic coverage period as the current storage pool exist in all the storage pools with the required storage space larger than the residual storage space aiming at the storage pools with the required storage space larger than the residual storage space; if so, jumping to step S10312;
step S10312: calculating a new cycle coverage period according to a method shown in the following formula (2), and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period:
wherein x represents a new cyclic coverage period, B 1 Representing the storage speed corresponding to the current storage pool, B i Indicating the storage speed corresponding to the ith other storage pool, n indicating the total number of the current storage pool and the other storage pools, and a indicating the total sum of the remaining storage spaces of the current storage pool and the other storage pools.
In this embodiment, it may be determined, for each storage pool where the required storage space is greater than the remaining storage space, whether there are other storage pools having the same specified cycle coverage period as the current storage pool in all storage pools where the required storage space is greater than the remaining storage space, and if there are any, a new cycle coverage period may be calculated according to the method shown in the formula (2), and the preset cycle coverage period of the storage pool may be adjusted to be the new cycle coverage period. By the configuration mode, the storage pool with the specified cycle coverage period can be subjected to global calculation to obtain a new cycle coverage period, so that the cycle coverage period of the storage pool with the same specified cycle coverage period in the storage system can be planned globally.
As an example, there are two types of service data (service data 1 and service data 2) to be stored, service data 1 to be stored in storage pool 1, service data 2 to be stored in storage pool 2, both of the cycle coverage periods specified by the two types of service data to be 30 days, the storage speed of storage pool 1 to be 1 TB/day, the storage speed of storage pool 2 to be 3 TB/day, and the sum of the remaining storage spaces of storage pool 1 and storage pool 2 to be 100TB, then a new cycle coverage period of 25 days can be obtained according to formula (2). It can be seen that if the service data 1 and the service data 2 are stored for 30 days, the space of the storage pool is used up during the storage of the service data, resulting in a situation where the storage service is not available. This will not occur when storing the service data according to the storage speed and the new cycle coverage period obtained by the remaining space.
In one embodiment, step S103 may further include:
step S1032: if the required storage space is smaller than or equal to the residual storage space, acquiring an upper limit storage space of the storage pool, wherein the upper limit storage space is larger than or equal to the required storage space;
step S1033: and acquiring a new cycle coverage period according to the upper limit storage space and the storage speed, and adjusting the preset cycle coverage period of the storage pool into the new cycle coverage period.
In this embodiment, if the storage space required for storing service data of a corresponding data service type in a cyclic coverage manner is smaller than or equal to the remaining storage space of the storage pool, the upper limit storage space of the storage pool may be obtained, a new cyclic coverage period may be obtained according to the upper limit storage space and the storage speed, and further, the preset cyclic coverage period of the storage pool may be adjusted to the new cyclic coverage period. Wherein the upper limit storage space is greater than or equal to the required storage space. When the storage space required for storing the service data of the corresponding data service type in a cyclic coverage mode is smaller than or equal to the remaining storage space of the storage pool, a certain storage space allowance (the remaining storage space of the storage pool-the upper limit storage space) can be reserved for the storage pool, and then a new cyclic coverage period of the storage pool is acquired according to the upper limit storage space, so that the preset cyclic coverage period of the storage pool is adjusted to be longer, and the resource utilization rate of the storage pool is improved.
It should be noted that, a person skilled in the art may flexibly set a specific value of the upper limit storage space according to actual requirements. As an example, the upper limit storage space may be 95% of the storage space of the storage pool.
In one implementation of the embodiment of the present invention, the storage system may include at least one storage node, where each storage node may be respectively provided with at least one storage pool, and the storage node may be configured to, in response to a received data storage request, control, according to a data type specified in the data storage request, a storage pool corresponding to the data type to store service data corresponding to the data type in a cyclic coverage manner. A storage node refers to a device capable of performing a data storage method of a storage system according to an embodiment of the present invention, including but not limited to: computers, servers, mobile computers, etc. After receiving the data storage request, the storage node may control the storage pool corresponding to the data type according to the data type specified in the data storage request to store the service data corresponding to the data type in a cyclic coverage manner by executing the methods described in steps S101 to S104. In addition to the steps S101 to S104, the embodiment of the present invention may further include the following step S105 before step S103:
step S105: and for each storage pool, if the storage space utilization rate of the current storage pool is smaller than a preset utilization rate threshold value and the number of data storage requests responded by the storage nodes corresponding to the current storage pool is larger than a preset number threshold value, not adjusting the preset cycle coverage period of the current storage pool. A data storage request that a storage node is responding to refers to a data storage request that the storage node receives and responds to.
In this embodiment, before step S103 is performed, it may be determined whether to adjust the preset cycle coverage period of the current storage pool according to the storage space usage of the current storage pool and the number of data storage requests that the storage node corresponding to the current storage pool is responding to. The usage threshold and the number threshold may be set by those skilled in the art according to the needs of the actual application.
As an example, when the space usage of the storage pool is less than 90% and the number of data storage requests that the storage node corresponding to the storage pool is responding to is greater than 3, the preset cycle coverage period of the current storage pool is not adjusted.
Referring to fig. 2, fig. 2 is a schematic flow chart of main steps of storing service data in a storage pool in a cyclic coverage manner according to an embodiment of the present invention. As shown in fig. 2, in one embodiment, when the storage pool stores service data in a cyclic coverage manner, a trigger, a collector, a load balancer and an actuator may complete a process of storing service data in a cyclic coverage manner, and main steps of storing service data in a cyclic coverage manner in the storage pool may include:
step S201: a trigger.
In this embodiment, the trigger may periodically detect whether each storage pool in the storage system starts a cyclic coverage policy, i.e. whether the storage pool is allowed to store traffic data in a cyclic coverage manner. For a storage pool for which the cyclic coverage policy is not started, the cyclic coverage mode is not used to store the service data in any case.
Step S202: a collector.
In this embodiment, the collector may be configured to perform the method described in step S101 in the foregoing method embodiment.
Step S203: and a load balancer.
In this embodiment, the load balancer may be configured to perform the methods described in step S102, step S103, and step S105 in the foregoing method embodiments.
Step S204: an actuator.
In this embodiment, the actuator may be configured to perform the method described in step S104 in the foregoing method embodiment.
It should be noted that, although the foregoing embodiments describe the steps in a specific order, it will be understood by those skilled in the art that, in order to achieve the effects of the present invention, the steps are not necessarily performed in such an order, and may be performed simultaneously (in parallel) or in other orders, and these variations are within the scope of the present invention.
Further, the invention also provides a data storage device of the storage system.
Referring to fig. 3, fig. 3 is a main block diagram of a data storage device of a storage system according to an embodiment of the present invention. As shown in fig. 3, a storage system according to an embodiment of the present invention may include a plurality of storage pools, each of which may be configured to store service data of a different data type, respectively; the data storage device of the storage system may include a storage speed acquisition module, a storage space acquisition module, a cycle coverage period adjustment module, and a service data storage module. In this embodiment, the storage speed obtaining module may be configured to obtain a data type and a specified cycle coverage period specified by the data storage request, and determine a storage speed of the storage pool according to a total storage space of the storage pool corresponding to the data type and the specified cycle coverage period. The storage space acquisition module may be configured to determine a storage space required when storing traffic data of a data type in a cyclic coverage manner according to a storage speed and a preset cyclic coverage period of the storage pool. The cycle coverage period adjustment module may be configured to adjust a preset cycle coverage period of the storage pool based on the required storage space and the remaining storage space of the storage pool. The service data storage module may be configured to control the storage pool to store the service data of the data type in a cyclic coverage manner according to the storage speed and the adjusted preset cyclic coverage period.
In one embodiment, the cyclical coverage period adjustment module may include a first cyclical coverage period adjustment unit. The first cycle coverage period adjustment unit may be configured to acquire a new cycle coverage period according to the storage speed and the remaining storage space if the required storage space is greater than the remaining storage space, and adjust the preset cycle coverage period of the storage pool to the new cycle coverage period.
In one embodiment, the first cyclical coverage-period adjustment unit may be further configured to perform the steps of: judging whether other storage pools with the same specified cyclic coverage period as the current storage pool exist in all the storage pools with the required storage space larger than the residual storage space aiming at the storage pools with the required storage space larger than the residual storage space; if so, calculating a new cycle coverage period according to a method shown in the following formula (2), and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period:
wherein x represents a new cyclic coverage period, B 1 Representing the storage speed corresponding to the current storage pool, B i Indicating the storage speed corresponding to the ith other storage pool, n indicating the total number of the current storage pool and the other storage pools, and a indicating the total sum of the remaining storage spaces of the current storage pool and the other storage pools.
In one embodiment, the cyclical coverage period adjustment module may include a second cyclical coverage period adjustment unit. In the present embodiment, the second cyclic coverage period adjustment unit may be configured to perform the steps of: if the required storage space is smaller than or equal to the residual storage space, acquiring an upper limit storage space of the storage pool, wherein the upper limit storage space is larger than or equal to the required storage space; and acquiring a new cycle coverage period according to the upper limit storage space and the storage speed, and adjusting the preset cycle coverage period of the storage pool into the new cycle coverage period.
In one embodiment, the storage system may include at least one storage node, each storage node being respectively provided with at least one storage pool, the storage node being configured to control, in response to a received data storage request, a storage pool corresponding to a data type according to a data type specified in the data storage request to store service data of the data type in a cyclic coverage manner; the data storage device of the storage system may further include a cycle coverage period determination module. In this embodiment, the cycle coverage period determining module may be configured to, for each storage pool, not adjust the preset cycle coverage period of the current storage pool if the storage space usage rate of the current storage pool is less than a preset usage rate threshold and the number of data storage requests that the storage node corresponding to the current storage pool is responding to is greater than a preset number threshold.
The technical principles of the foregoing data storage device of the storage system and the technical principles of the foregoing data storage method embodiment of the storage system shown in fig. 1, the technical problems to be solved and the technical effects to be produced are similar, and those skilled in the art can clearly understand that, for convenience and brevity of description, the specific working process and related description of the data storage device of the storage system may refer to the description of the embodiment of the data storage method of the storage system, which is not repeated herein.
It will be appreciated by those skilled in the art that the present invention may implement all or part of the above-described methods according to the above-described embodiments, or may be implemented by means of a computer program for instructing relevant hardware, where the computer program may be stored in a computer readable storage medium, and where the computer program may implement the steps of the above-described embodiments of the method when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include: any entity or device, medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like capable of carrying the computer program code. It should be noted that the computer readable storage medium may include content that is subject to appropriate increases and decreases as required by jurisdictions and by jurisdictions in which such computer readable storage medium does not include electrical carrier signals and telecommunications signals.
Further, the invention also provides a control device. In one control device embodiment according to the present invention, the control device includes a processor and a storage device, the storage device may be configured to store a program for executing the data storage method of the storage system of the above-described method embodiment, and the processor may be configured to execute the program in the storage device, including, but not limited to, the program for executing the data storage method of the storage system of the above-described method embodiment. For convenience of explanation, only those portions of the embodiments of the present invention that are relevant to the embodiments of the present invention are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present invention. The control device may be a control device formed of various electronic devices.
Further, the invention also provides a computer readable storage medium. In one embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for performing the data storage method of the storage system of the above-described method embodiment, the program being loadable and executable by a processor to implement the data storage method of the above-described storage system. For convenience of explanation, only those portions of the embodiments of the present invention that are relevant to the embodiments of the present invention are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present invention. The computer readable storage medium may be a storage device including various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present invention is a non-transitory computer readable storage medium.
Further, it should be understood that, since the respective modules are merely set to illustrate the functional units of the apparatus of the present invention, the physical devices corresponding to the modules may be the processor itself, or a part of software in the processor, a part of hardware, or a part of a combination of software and hardware. Accordingly, the number of individual modules in the figures is merely illustrative.
Those skilled in the art will appreciate that the various modules in the apparatus may be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principle of the present invention, and therefore, the technical solution after splitting or combining falls within the protection scope of the present invention.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will fall within the scope of the present invention.
Claims (8)
1. A data storage method of a storage system, the storage system comprising a plurality of storage pools, each storage pool being configured to store traffic data of a different data type, the method comprising:
acquiring a data type designated by a data storage request and a designated cycle coverage period, and determining the storage speed of a storage pool according to the total storage space of the storage pool corresponding to the data type and the designated cycle coverage period;
determining a storage space required when the service data of the data type are stored in a cyclic coverage mode according to the storage speed and a preset cyclic coverage period of the storage pool;
according to the required storage space and the residual storage space of the storage pool, adjusting a preset cycle coverage period of the storage pool;
controlling the storage pool to store the service data of the data type in a cyclic coverage mode according to the storage speed and the adjusted preset cyclic coverage period;
the step of adjusting the preset cycle coverage period of the storage pool according to the required storage space and the remaining storage space of the storage pool specifically includes:
if the required storage space is larger than the residual storage space, acquiring a new cycle coverage period according to the storage speed and the residual storage space, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period;
if the required storage space is smaller than or equal to the residual storage space, acquiring an upper limit storage space of the storage pool, wherein the upper limit storage space is larger than or equal to the required storage space;
and acquiring a new cycle coverage period according to the upper limit storage space and the storage speed, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period.
2. The method for storing data in a storage system according to claim 1, wherein the step of acquiring a new cycle coverage period according to the storage speed and the remaining storage space and adjusting the preset cycle coverage period of the storage pool to the new cycle coverage period specifically comprises:
judging whether other storage pools with the same specified cyclic coverage period as the current storage pool exist in all the storage pools with the required storage space larger than the residual storage space aiming at the storage pools with the required storage space larger than the residual storage space;
if so, calculating a new cycle coverage period according to the method shown in the following formula, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period:
wherein x represents a new cyclic coverage period, B 1 Representing the storage speed corresponding to the current storage pool, B i And (3) representing the storage speed corresponding to the ith other storage pool, wherein n represents the total number of the current storage pool and the other storage pools, and A represents the total sum of the current storage pool and the residual storage space of the other storage pools.
3. The data storage method of a storage system according to claim 1, wherein the storage system comprises at least one storage node, each storage node is respectively provided with at least one storage pool, and the storage node is configured to respond to a received data storage request, and control the storage pool corresponding to the data type according to the data type specified in the data storage request to store the service data of the data type in a cyclic coverage manner;
before the step of adjusting the preset cycle coverage period of the storage pool according to the required storage space and the remaining storage space of the storage pool, the method further includes:
and for each storage pool, if the storage space utilization rate of the current storage pool is smaller than a preset utilization rate threshold value and the number of data storage requests responded by the storage nodes corresponding to the current storage pool is larger than a preset number threshold value, not adjusting the preset cycle coverage period of the current storage pool.
4. A data storage device of a storage system, the storage system comprising a plurality of storage pools, each storage pool being configured to store traffic data of a different data type, the device comprising:
the storage speed acquisition module is configured to acquire a data type designated by a data storage request and a designated cycle coverage period, and determine the storage speed of the storage pool according to the total storage space of the storage pool corresponding to the data type and the designated cycle coverage period;
a storage space acquisition module configured to determine a storage space required when storing the service data of the data type in a cyclic coverage manner in accordance with the storage speed and a preset cyclic coverage period of the storage pool;
the cycle coverage period adjustment module is configured to adjust a preset cycle coverage period of the storage pool according to the required storage space and the residual storage space of the storage pool;
the business data storage module is configured to control the storage pool to store the business data of the data type in a cyclic coverage mode according to the storage speed and the adjusted preset cyclic coverage period;
the cyclic coverage period adjustment module includes:
a first cycle coverage period adjustment unit configured to acquire a new cycle coverage period according to the storage speed and the remaining storage space if the required storage space is larger than the remaining storage space, and adjust a preset cycle coverage period of the storage pool to the new cycle coverage period;
the cyclic coverage period adjustment module includes a second cyclic coverage period adjustment unit configured to perform the steps of:
if the required storage space is smaller than or equal to the residual storage space, acquiring an upper limit storage space of the storage pool, wherein the upper limit storage space is larger than or equal to the required storage space;
and acquiring a new cycle coverage period according to the upper limit storage space and the storage speed, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period.
5. The data storage device of the storage system of claim 4, wherein the first cycle coverage period adjustment unit is further configured to perform the steps of:
judging whether other storage pools with the same specified cyclic coverage period as the current storage pool exist in all the storage pools with the required storage space larger than the residual storage space aiming at the storage pools with the required storage space larger than the residual storage space;
if so, calculating a new cycle coverage period according to the method shown in the following formula, and adjusting the preset cycle coverage period of the storage pool to be the new cycle coverage period:
where x represents the new cyclic coverage period,B 1 representing the storage speed corresponding to the current storage pool, B i And (3) representing the storage speed corresponding to the ith other storage pool, wherein n represents the total number of the current storage pool and the other storage pools, and A represents the total sum of the current storage pool and the residual storage space of the other storage pools.
6. The data storage device of claim 4, wherein the storage system comprises at least one storage node, each storage node is respectively provided with at least one storage pool, and the storage node is configured to respond to a received data storage request, and control the storage pool corresponding to the data type to store the service data of the data type in a cyclic coverage manner according to the data type specified in the data storage request;
the apparatus further comprises:
the cycle coverage period judging module is configured to not adjust the preset cycle coverage period of the current storage pool if the storage space utilization rate of the current storage pool is smaller than a preset utilization rate threshold value and the number of the data storage requests responded by the storage nodes corresponding to the current storage pool is larger than a preset number threshold value for each storage pool.
7. A control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program code, characterized in that the program code is adapted to be loaded and executed by the processor to perform the data storage method of the storage system of any one of claims 1 to 3.
8. A computer readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and executed by a processor to perform the data storage method of the storage system of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111484720.6A CN114153399B (en) | 2021-12-07 | 2021-12-07 | Data storage method, device, control device and medium of storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111484720.6A CN114153399B (en) | 2021-12-07 | 2021-12-07 | Data storage method, device, control device and medium of storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114153399A CN114153399A (en) | 2022-03-08 |
| CN114153399B true CN114153399B (en) | 2023-10-20 |
Family
ID=80453286
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111484720.6A Active CN114153399B (en) | 2021-12-07 | 2021-12-07 | Data storage method, device, control device and medium of storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114153399B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508784A (en) * | 2011-11-02 | 2012-06-20 | 杭州海康威视数字技术股份有限公司 | Data storage method of flash memory card in video monitoring equipment, and system thereof |
| CN104123387A (en) * | 2014-08-07 | 2014-10-29 | 浪潮电子信息产业股份有限公司 | Method for managing life cycle of magnetic tape data |
| CN106060442A (en) * | 2016-05-20 | 2016-10-26 | 浙江宇视科技有限公司 | Video storage method, device and system |
| CN106961569A (en) * | 2017-03-21 | 2017-07-18 | 深圳英飞拓科技股份有限公司 | One kind video recording covering method, device and network hard disk video recorder |
| CN106993148A (en) * | 2017-03-21 | 2017-07-28 | 深圳英飞拓科技股份有限公司 | One kind video recording covering method, device and network hard disk video recorder |
| CN108076317A (en) * | 2016-11-17 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | Video data storage method and system, video cut-in method and node |
| CN109391647A (en) * | 2017-08-04 | 2019-02-26 | 杭州海康威视系统技术有限公司 | Storage resource recovery method, apparatus and system |
| CN110300134A (en) * | 2018-03-22 | 2019-10-01 | 杭州海康威视系统技术有限公司 | Memory space method of adjustment, device and the cloud storage system of cloud storage resource pool |
| CN111813322A (en) * | 2019-04-11 | 2020-10-23 | 杭州海康威视系统技术有限公司 | Method, device and equipment for creating storage pool and storage medium |
| CN112734565A (en) * | 2021-01-12 | 2021-04-30 | 中国工商银行股份有限公司 | Method and device for predicting mobile coverage rate |
| CN112765046A (en) * | 2019-10-21 | 2021-05-07 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for reclaiming storage space |
| CN113177033A (en) * | 2021-03-31 | 2021-07-27 | 新华三信息安全技术有限公司 | Log data storage method and device, electronic equipment and medium |
| CN113190175A (en) * | 2021-04-28 | 2021-07-30 | 杭州海康威视系统技术有限公司 | Object data processing method, device and equipment |
| CN113254365A (en) * | 2021-06-09 | 2021-08-13 | 北京字节跳动网络技术有限公司 | Method, apparatus, device, medium, and program product for managing storage space |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10425480B2 (en) * | 2014-06-26 | 2019-09-24 | Hitachi Vantara Corporation | Service plan tiering, protection, and rehydration strategies |
| US9875052B2 (en) * | 2016-03-15 | 2018-01-23 | International Business Machines Corporation | Storage capacity allocation using distributed spare space |
| CN111124254B (en) * | 2018-10-30 | 2023-09-29 | 伊姆西Ip控股有限责任公司 | Method, electronic device and program product for scheduling memory space reclamation requests |
-
2021
- 2021-12-07 CN CN202111484720.6A patent/CN114153399B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508784A (en) * | 2011-11-02 | 2012-06-20 | 杭州海康威视数字技术股份有限公司 | Data storage method of flash memory card in video monitoring equipment, and system thereof |
| CN104123387A (en) * | 2014-08-07 | 2014-10-29 | 浪潮电子信息产业股份有限公司 | Method for managing life cycle of magnetic tape data |
| CN106060442A (en) * | 2016-05-20 | 2016-10-26 | 浙江宇视科技有限公司 | Video storage method, device and system |
| CN108076317A (en) * | 2016-11-17 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | Video data storage method and system, video cut-in method and node |
| CN106961569A (en) * | 2017-03-21 | 2017-07-18 | 深圳英飞拓科技股份有限公司 | One kind video recording covering method, device and network hard disk video recorder |
| CN106993148A (en) * | 2017-03-21 | 2017-07-28 | 深圳英飞拓科技股份有限公司 | One kind video recording covering method, device and network hard disk video recorder |
| CN109391647A (en) * | 2017-08-04 | 2019-02-26 | 杭州海康威视系统技术有限公司 | Storage resource recovery method, apparatus and system |
| CN110300134A (en) * | 2018-03-22 | 2019-10-01 | 杭州海康威视系统技术有限公司 | Memory space method of adjustment, device and the cloud storage system of cloud storage resource pool |
| CN111813322A (en) * | 2019-04-11 | 2020-10-23 | 杭州海康威视系统技术有限公司 | Method, device and equipment for creating storage pool and storage medium |
| CN112765046A (en) * | 2019-10-21 | 2021-05-07 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for reclaiming storage space |
| CN112734565A (en) * | 2021-01-12 | 2021-04-30 | 中国工商银行股份有限公司 | Method and device for predicting mobile coverage rate |
| CN113177033A (en) * | 2021-03-31 | 2021-07-27 | 新华三信息安全技术有限公司 | Log data storage method and device, electronic equipment and medium |
| CN113190175A (en) * | 2021-04-28 | 2021-07-30 | 杭州海康威视系统技术有限公司 | Object data processing method, device and equipment |
| CN113254365A (en) * | 2021-06-09 | 2021-08-13 | 北京字节跳动网络技术有限公司 | Method, apparatus, device, medium, and program product for managing storage space |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114153399A (en) | 2022-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10637916B2 (en) | Method and device for storage resource allocation for video cloud storage | |
| CN110795244B (en) | Task allocation method, device, equipment and medium | |
| US20120278451A1 (en) | Method, system, and scheduling server for content delivery | |
| CN106713495B (en) | The method for uploading and access method in IP geographical position, device and access system | |
| WO2017172820A1 (en) | Time-based adjustable load balancing | |
| WO2014194704A1 (en) | A grouping processing method and system | |
| CN111213343B (en) | Host upgrading method and device | |
| CN111245732A (en) | Flow control method, device and equipment | |
| CN105491117A (en) | Flow chart data processing system and method for real time data analysis | |
| CN104202305A (en) | Transcoding processing method and device, server | |
| CN112084017A (en) | Memory management method and device, electronic equipment and storage medium | |
| CN108270753B (en) | Method and device for logging out user account | |
| US20130007281A1 (en) | Dynamically tuning server placement | |
| CN114153399B (en) | Data storage method, device, control device and medium of storage system | |
| CN112286623B (en) | Information processing method and device and storage medium | |
| US20140215035A1 (en) | Distribution of application components among devices | |
| CN116955271A (en) | Method and device for storing data copy, electronic equipment and storage medium | |
| CN116820324A (en) | Storage capacity expansion method and device, storage medium and electronic equipment | |
| US9860322B2 (en) | Stateless technique for distributing data uploads | |
| US9479579B2 (en) | Grouping processing method and system | |
| CN105763508B (en) | Data access method and application server | |
| CN113468442A (en) | Resource bit flow distribution method, computing device and computer storage medium | |
| CN111405313B (en) | Method and system for storing streaming media data | |
| CN112291287A (en) | Cloud platform-based containerized application network flow control method and device and storage medium | |
| CN115150400B (en) | Service fault processing method and device, cloud service platform and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |