CN110300134B

CN110300134B - Storage space adjusting method and device of cloud storage resource pool and cloud storage system

Info

Publication number: CN110300134B
Application number: CN201810241477.7A
Authority: CN
Inventors: 李照辉; 林鹏; 林起芊; 汪渭春
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2018-03-22
Filing date: 2018-03-22
Publication date: 2022-10-04
Anticipated expiration: 2038-03-22
Also published as: CN110300134A

Abstract

The embodiment of the invention provides a method and a device for adjusting a storage space of a cloud storage resource pool and a cloud storage system, wherein the method for adjusting the storage space of the cloud storage resource pool is applied to a management server and comprises the following steps: acquiring and recording flow statistical information of the cloud storage resource pool counted by each storage server; predicting the storage capacity required by the residual storage period according to the recorded historical flow statistical information; judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity; if the current storage space is smaller than the preset storage space, the residual storage space of the cloud storage resource pool is adjusted through a preset adjustment storage strategy. By the scheme, the data storage efficiency can be improved.

Description

Storage space adjusting method and device of cloud storage resource pool and cloud storage system

Technical Field

The invention relates to the technical field of cloud storage, in particular to a method and a device for adjusting a storage space of a cloud storage resource pool and a cloud storage system.

Background

The cloud storage system is a system which integrates a large number of various different types of storage devices in a network through software to cooperatively work by using a cluster technology, a virtualization technology and a distributed storage technology and provides data storage and service access functions for the outside. The cloud storage system realizes storage and management of a large amount of data, for example, in a security system, the data volume of security videos is large, and the security videos are stored and managed by the cloud storage system in multiple applications.

The cloud storage system provides a plurality of storage servers, and the storage servers are access storage media in the cloud storage system and are servers providing storage functions in the cloud storage system. The logical division of the storage resources in the storage server is realized by a cloud storage resource pool, and the cloud storage resource pool is used as a resource set and has the characteristics of resource sharing, allocation according to needs, dynamic expansion, standard service, automatic management and the like. The cloud storage resource pool is a virtual storage space provided by the cloud storage system.

The cloud storage resource pool has an attribute of a storage period, in order to ensure that the cloud storage resource pool has enough storage space to store new data, when an interval between storage time of data in the cloud storage resource pool and current time exceeds a preset storage period, the management server may trigger a deletion operation on the data, for example, the storage time of a data is 7; in practical application, due to the fact that the data volume is large, the storage space of the cloud storage resource pool may be occupied quickly, if the remaining storage space of the cloud storage resource pool is not enough to store the new data after the new data is acquired, the storage space of the cloud storage resource pool needs to be adjusted, the storage space is released in a capacity covering mode, namely the storage space is released by deleting the historical data with the earliest storage time, and normal storage of the new data is guaranteed.

For the situation that the remaining storage space of the cloud storage resource pool is not enough to store new data, the method needs to passively delete the historical data after receiving the new data, and the new data can be stored only when the cloud storage resource pool releases enough storage space due to the fact that the new data is received, so that the storage efficiency of the data is seriously affected.

Disclosure of Invention

The embodiment of the invention aims to provide a method and a device for adjusting the storage space of a cloud storage resource pool and a cloud storage system, so as to improve the storage efficiency of data. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for adjusting a storage space of a cloud storage resource pool, where the method is applied to a management server, and the method includes:

acquiring and recording flow statistical information of a cloud storage resource pool counted by each storage server;

predicting the storage capacity required by the residual storage period according to the recorded historical flow statistical information;

judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity or not;

and if the current storage space is smaller than the preset storage space, adjusting the residual storage space of the cloud storage resource pool through a preset adjustment storage strategy.

Optionally, the obtaining of the traffic statistical information of the cloud storage resource pool counted by each storage server includes:

issuing an information acquisition instruction to each storage server so that each storage server sends statistical flow statistical information of the cloud storage resource pool to the management server after receiving the information acquisition instruction;

and receiving the flow statistic information sent by each storage server.

periodically acquiring traffic statistical information of the cloud storage resource pool, which is counted by each storage server, according to a preset acquisition period;

after the obtaining and recording the traffic statistic information of the cloud storage resource pool counted by each storage server, the method further includes:

and summarizing the flow statistical information of all the cloud storage resource pools in the preset acquisition period.

Optionally, the predicting, according to the recorded historical traffic statistical information, a storage capacity required by a remaining storage period includes:

predicting data flow of the cloud storage resource pool in the next preset acquisition period according to recorded first flow statistical information in the current preset acquisition period, a first average value of each flow statistical information acquired at a plurality of preset time points, a second average value of each flow statistical information acquired in a preset time period, and weights pre-distributed for the first flow statistical information, the first average value and the second average value respectively;

calculating to obtain a residual storage period of the cloud storage resource pool according to the recorded historical flow statistical information and the total storage period of the cloud storage resource pool;

and calculating to obtain the storage capacity required by the residual storage period through the product operation of the proportion of the residual storage period in the preset acquisition period and the data flow of the cloud storage resource pool in the next preset acquisition period.

Optionally, the adjusting the remaining storage space of the cloud storage resource pool by presetting the adjustment storage policy includes:

judging whether the unused storage space in the cloud storage system is larger than or equal to a first capacity difference value, wherein the first capacity difference value is the difference value between the storage capacity and the residual storage space;

if so, allocating a temporary storage space to the cloud storage resource pool, wherein the temporary storage space is a storage space with unused storage capacity in the cloud storage system as a first capacity difference value;

if not, allocating unused storage space in the cloud storage system to the cloud storage resource pool; generating a capacity alarm instruction; and sending the capacity alarm instruction to a user side.

Optionally, after allocating the unused storage space in the cloud storage system to the cloud storage resource pool, the method further includes:

the storage capacity, the residual storage space and unused storage space in the cloud storage system are subjected to subtraction to obtain the capacity of the space to be deleted;

accumulating all historical traffic statistical information from the recorded historical traffic statistical information with the earliest storage time point until the accumulated value is greater than or equal to the capacity of the space to be deleted;

if the accumulated value is larger than the capacity of the space to be deleted, determining that the storage time point in the last-but-one historical flow statistical information when accumulation is stopped is a deletion time point; if the accumulated value is equal to the capacity of the space to be deleted, determining that the storage time point in the last historical traffic statistic information when accumulation is stopped is a deletion time point;

and issuing the deletion time point to each storage server so that each storage server deletes all data with the storage time before the deletion time point.

if the video data are stored in each storage server, issuing a frame extraction and transfer instruction to each storage server so that each storage server can reserve the key frames of the stored historical video data and delete the non-key frames of the historical video data;

recording the frequency of issuing the frame-extracting and unloading instruction;

if the residual storage space of the cloud storage resource pool after the frame extraction and unloading is carried out is smaller than the storage capacity, judging whether the frequency of issuing the frame extraction and unloading instruction reaches a first preset frequency or not;

if the number of times of issuing the frame extracting and unloading instruction reaches the first preset number of times, issuing the frame extracting and unloading instruction and a non-key frame filtering instruction to each storage server so that each storage server can reserve the key frames of the stored historical video data, delete the non-key frames of the historical video data and filter the appointed non-key frames of the newly received video data; recording the frequency of issuing the non-key frame filtering instruction;

if the residual storage space of the cloud storage resource pool after frame extraction and unloading and non-key frame filtering is smaller than the storage capacity, judging whether the frequency of issuing the non-key frame filtering instruction reaches a second preset frequency;

if the frequency of issuing the non-key frame filtering instruction does not reach the second preset frequency, issuing the frame extracting and unloading instruction and the non-key frame filtering instruction to each storage server, and increasing the frequency of issuing the non-key frame filtering instruction;

and if the frequency of issuing the frame extracting and unloading instruction does not reach the first preset frequency, issuing the frame extracting and unloading instruction to each storage server, and increasing the frequency of issuing the frame extracting and unloading instruction.

Optionally, after determining whether the number of times that the non-key frame filtering instruction has been issued reaches a second preset number of times, the method further includes:

if the number of times of issuing the non-key frame filtering instruction reaches the second preset number of times, subtracting the storage capacity, the residual storage space of the cloud storage resource pool subjected to frame extraction and unloading and non-key frame filtering and the unused storage space in the cloud storage system to obtain the capacity of the space to be deleted;

accumulating the historical traffic statistical information from the recorded historical traffic statistical information with the earliest storage time point until the accumulated value is greater than or equal to the capacity of the space to be deleted;

and transmitting the deletion time point to each storage server so that each storage server deletes all video data with the storage time before the deletion time point.

In a second aspect, an embodiment of the present invention provides a storage space adjusting apparatus for a cloud storage resource pool, where the apparatus is applied to a management server, and the apparatus includes:

the acquisition module is used for acquiring and recording the flow statistical information of the cloud storage resource pool counted by each storage server;

the prediction module is used for predicting the storage capacity required by the residual storage period according to the recorded historical flow statistical information;

the judging module is used for judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity;

and the adjusting module is used for adjusting the residual storage space of the cloud storage resource pool through a preset adjusting storage strategy if the judging result of the judging module is that the residual storage space of the cloud storage resource pool is smaller than the storage capacity.

Optionally, the obtaining module is specifically configured to:

and receiving the flow statistic information sent by each storage server.

Optionally, the obtaining module is specifically configured to:

Optionally, the prediction module is specifically configured to:

predicting data flow of the cloud storage resource pool in the next preset acquisition period according to recorded first flow statistical information in the current preset acquisition period, a first average value of each flow statistical information acquired at preset time points, a second average value of each flow statistical information acquired in a preset time period, and weights pre-distributed for the first flow statistical information, the first average value and the second average value respectively;

calculating to obtain the remaining storage period of the cloud storage resource pool according to the recorded historical flow statistical information and the total storage period of the cloud storage resource pool;

Optionally, the adjusting module is specifically configured to:

Optionally, the adjusting module is further configured to:

differencing the storage capacity, the residual storage space and unused storage space in the cloud storage system to obtain the capacity of the space to be deleted;

Optionally, the adjusting module is further configured to:

recording the frequency of issuing the frame extraction and unloading instruction;

if the residual storage space of the cloud storage resource pool after the frame extraction and unloading is smaller than the storage capacity, judging whether the frequency of issuing the frame extraction and unloading instruction reaches a first preset frequency;

if the frequency of issuing the non-key frame filtering instruction does not reach the second preset frequency, issuing the frame extracting unloading instruction and the non-key frame filtering instruction to each storage server, and increasing the frequency of issuing the non-key frame filtering instruction;

Optionally, the adjusting module is further configured to:

if the number of times of issuing the non-key frame filtering instruction reaches the second preset number of times, subtracting the storage capacity, the residual storage space of the cloud storage resource pool subjected to frame extraction and non-key frame filtering and the unused storage space in the cloud storage system to obtain the capacity of a space to be deleted;

if the accumulated value is larger than the capacity of the space to be deleted, determining that the storage time point in the last-but-one historical traffic statistical information when accumulation is stopped is a deletion time point; if the accumulated value is equal to the capacity of the space to be deleted, determining that the storage time point in the last historical traffic statistic information when accumulation is stopped is a deletion time point;

and issuing the deletion time point to each storage server so that each storage server deletes all video data with the storage time before the deletion time point.

In a third aspect, an embodiment of the present invention provides a cloud storage system, where the system includes: a management server and a plurality of storage servers;

the management server is used for acquiring and recording the flow statistical information of the cloud storage resource pool counted by each storage server; predicting the storage capacity required by the residual storage period according to the recorded historical flow statistical information; judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity; if the current storage space is smaller than the preset storage space, adjusting the residual storage space of the cloud storage resource pool through a preset adjustment storage strategy;

the storage server is used for counting the traffic statistical information of the cloud storage resource pool and sending the counted traffic statistical information to the management server; the data is stored.

In a fourth aspect, an embodiment of the present invention provides a management server, including a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to implement the method steps of the first aspect of the embodiment of the present invention when executing the program stored in the memory.

In a fifth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of the first aspect of the present invention.

According to the method, the device and the cloud storage system for adjusting the storage space of the cloud storage resource pool, provided by the embodiment of the invention, the management server predicts the storage capacity required by the residual storage period by acquiring and recording the flow statistical information of the cloud storage resource pool counted by each storage server according to the recorded historical flow statistical information, and adjusts the residual storage space of the cloud storage resource pool by presetting an adjustment storage strategy to ensure the storage of data by judging the size relationship between the prediction result and the residual storage space of the cloud storage resource pool and if the residual storage space is smaller than the storage capacity and the predicted data flow is not enough to be stored in the residual storage space. Before receiving new data needing to be stored, the storage capacity needed by the residual storage period is predicted, and when the residual storage space cannot meet the prediction result, an operation of adjusting the residual storage space is performed, that is, before receiving the new data needing to be stored, the residual storage space of the cloud storage resource pool is adjusted to be capable of storing the new data, so that the storage efficiency of the data is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a cloud storage system in the prior art;

FIG. 2 is a schematic diagram of a data storage process of the prior art;

FIG. 3 is a flowchart illustrating a prior art cycle coverage process;

fig. 4 is a schematic flowchart of a method for adjusting a storage space of a cloud storage resource pool according to an embodiment of the present invention;

fig. 5 is a schematic diagram illustrating a flow of acquiring traffic statistics information according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating the operation flow of historical data deletion according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a storage space adjusting apparatus of a cloud storage resource pool according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a cloud storage system according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a management server according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to improve the storage efficiency of data in a cloud storage system, the embodiment of the invention provides a method and a device for adjusting a storage space of a cloud storage resource pool, the cloud storage system and a management server.

Next, a method for adjusting a storage space of a cloud storage resource pool provided in an embodiment of the present invention is first described.

An executing subject of the method for adjusting the storage space of the cloud storage resource pool provided by the embodiment of the present invention may be a management server in a cloud storage system, and the cloud storage system includes, as shown in fig. 1, a management server 101, a virtual storage space (including a plurality of cloud storage resource pools 102), and a real storage space (including a plurality of storage servers 103). The management server 101 creates a cloud storage resource pool for a server in the cloud storage system that manages the storage server 103, and provides a storage service to the outside. The management server at least comprises a core processing chip capable of realizing intelligent control logic, and the mode for realizing the method for adjusting the storage space of the cloud storage resource pool provided by the embodiment of the invention can be at least one of software, a hardware circuit and a logic circuit arranged in the management server.

The cloud storage resource pool and the storage servers are in a many-to-many corresponding relationship, and data of one cloud storage resource pool may be distributed on a plurality of storage servers. The cloud storage resource pool has two capacity attributes, wherein one is a fixed capacity attribute, namely a storage space when the cloud storage resource pool is created; and the other is a temporary capacity attribute, namely temporary storage space applied to the management server when the fixed capacity of the cloud storage resource pool is insufficient. Therefore, the total capacity of the cloud storage resource pool is the sum of the fixed capacity and the temporary capacity.

After receiving a data storage request of a user, the cloud storage system starts to execute a data storage process, where the data storage process is shown in fig. 2 and includes the following steps:

s201, the management server allocates a storage server to respond to a data storage request;

s202, the storage server stores the received data sent by the user;

and S203, the storage server records the data into the data traffic of the corresponding cloud storage resource pool.

Because the data stored in the cloud storage system is many and the actual storage space of the cloud storage system is limited, the data is stored all the time, the overflow phenomenon will be caused inevitably, and the probability of the actual use of the data stored at an earlier time point is relatively small, so that the data stored at the earlier time point can be subjected to periodic coverage processing, and the storage space is released, so that the new data can be normally stored. The flow of the cycle coverage processing is shown in fig. 3, and includes the following steps:

s301, the management server detects the storage time of the data in the cloud storage resource pool according to a preset detection period;

s302, when the time interval between the earliest storage time of the cloud storage resource pool and the current time is detected to exceed a preset storage period of the cloud storage resource pool, calculating a coverage time point covered by the period by the management server;

s303, the management server sends the coverage time point to each storage server;

s304, each storage server deletes all data with the storage time before the coverage time point according to the received coverage time point.

The coverage time point can be obtained by subtracting a preset storage period of the cloud storage resource pool from the current time point. The preset detection period may be configured in advance, for example, 10 hours, 1 day, 3 days, and the like, and may be determined according to parameters such as a total storage capacity of the cloud storage system, an average size of data traffic, and the like. When performing periodic coverage, the basic unit of periodic coverage is one statistical period, and partial data in one statistical period is not deleted, for example, when there is a group of data whose storage time is 9 to 9, the coverage time point obtained by calculation is 9.

In practical application, a time interval between the earliest storage time in stored data and the current time may not exceed a preset storage period of a cloud storage resource pool, while an actual storage space of a cloud storage system is insufficient for storing new data.

As shown in fig. 4, an embodiment of the present invention provides a method for adjusting a storage space of a cloud storage resource pool, where the method includes the following steps:

s401, obtaining and recording the flow statistic information of the cloud storage resource pool counted by each storage server.

As shown in fig. 2, after storing data, the storage servers will count the data into the data traffic of the corresponding cloud storage resource pool, that is, each storage server will count the traffic statistics information of the cloud storage resource pool, and the traffic statistics information may include the storage time of the data, the size of the data traffic, and the like. The management server may actively or passively obtain traffic statistical information of the cloud storage resource pool, which is counted by each storage server, and the management server needs to record the traffic statistical information because historical traffic statistical information needs to be used in the subsequent prediction of data traffic.

Optionally, S401 may be specifically implemented by the following steps.

and receiving the flow statistic information sent by each storage server.

The management server may actively acquire the traffic statistic information from each storage server, that is, when traffic prediction is required, actively issue an information acquisition instruction to each storage server, where the information acquisition instruction is equivalent to a trigger instruction, and after receiving the information acquisition instruction, each storage server sends the statistical traffic statistic information of the cloud storage resource pool to the management server. It should be noted that the information obtaining instruction issued by the management server may be issued periodically or based on a requirement, and is not limited in this respect.

Of course, the management server may also passively obtain the traffic statistic information from each storage server, that is, after each storage server counts the traffic statistic information, the management server directly sends the counted traffic statistic information to the management server; or each storage server directly sends all the traffic statistical information counted in one sending period to the management server according to a preset sending period.

In summary, whether the management server actively acquires the traffic statistic information from each storage server or passively acquires the traffic statistic information from each storage server, the traffic statistic information may be acquired periodically, that is, optionally, S401 may specifically be:

and periodically acquiring the traffic statistical information of the cloud storage resource pool counted by each storage server according to a preset acquisition period.

Moreover, because there is a many-to-many correspondence between the cloud storage resource pools and the storage servers, when the management server actually obtains the traffic statistical information, the obtained traffic statistical information is not only the traffic statistical information of a certain cloud storage resource pool, but also the traffic statistical information of each cloud storage resource pool, and therefore, after S401, the method for adjusting the storage space of the cloud storage resource pools may further include:

and summarizing the flow statistic information of all the cloud storage resource pools in a preset acquisition period.

After receiving the traffic statistical information of each cloud storage resource pool counted by each storage server, the management server summarizes the traffic statistical information of all the cloud storage resource pools acquired in a preset period, records the traffic statistical information of each cloud storage resource pool respectively, and summarizes and counts the traffic statistical information of each cloud storage resource pool in a database or a file, which is not limited herein.

Specifically, with reference to the above, the management server periodically and actively acquires the traffic statistic information, as shown in fig. 5, the flow of acquiring the traffic statistic information includes:

s501, the management server sends information acquisition instructions to each storage server according to a preset acquisition period;

s502, after receiving the information acquisition instruction, each storage server reports the flow statistical information of all the cloud storage resource pools to the management server, and clears the current flow statistical information;

and S503, the management server collects and records the flow statistical information of all the cloud storage resource pools in the preset acquisition period.

The management server can periodically and actively acquire the flow statistical information counted by each storage server, and because many-to-many correspondence exists between each storage server and each cloud storage resource pool, each storage server counts the flow statistical information of a plurality of corresponding cloud storage resource pools, the management server acquires the flow statistical information of all the cloud storage resource pools counted by each storage server, and through summarizing and recording, the management server is ensured to record the historical flow statistical information of each cloud storage resource pool. In order to ensure the space utilization rate of the storage server, the storage server can empty the current traffic statistical information after reporting the statistical traffic statistical information to the management server.

S402, predicting the storage capacity required by the residual storage period according to the recorded historical flow statistical information.

The recorded historical traffic statistical information reflects the traffic trend and the state of data storage, and generally, the data traffic with similar storage time has small difference and the data traffic stored at the same time in different days has similar size, so that the storage capacity required by the residual storage period can be predicted according to the recorded historical traffic statistical information. For example, a preset value may be added on the basis of the recorded traffic statistic information in the current preset acquisition period, and the added value is used as a data traffic prediction result of the cloud storage resource pool in the next preset acquisition period; the calculation result can also be used as a data traffic prediction result of the cloud storage resource pool in the next preset acquisition period by calculating an average value according to traffic statistical information (such as traffic statistical information acquired in 10. Of course, the data traffic of the cloud storage resource pool in the next preset acquisition period may also be predicted according to the traffic statistical information acquired in the last period of time, which is not described herein again. By analogy, the data flow in each preset acquisition period in the remaining storage period can be predicted, so that the storage capacity required by the remaining storage period can be obtained. The remaining storage period is the remaining time during which data can be stored in one storage period.

Optionally, S402 may be specifically implemented by the following steps.

The method comprises the steps of firstly, predicting data flow of a cloud storage resource pool in the next preset acquisition period according to recorded first flow statistic information in the current preset acquisition period, a first average value of each flow statistic information acquired at a plurality of preset time points, a second average value of each flow statistic information acquired in a preset time period, and weights pre-distributed for the first flow statistic information, the first average value and the second average value.

Based on the analysis, in order to improve the accuracy of predicting the data flow of the cloud storage resource pool in the next preset acquisition period, the recorded first flow statistical information in the current preset acquisition period, the first average value of each flow statistical information acquired at preset multiple time points, and the second average value of each flow statistical information acquired in the preset time period may be combined, and the prediction result of the data flow of the cloud storage resource pool in the next preset acquisition period may be obtained through the weighted calculation of the first flow statistical information, the first average value, and the second average value.

For example, if the predicted data traffic is 5 months, 28 days and 10 days, and the preset acquisition period is 1 hour, the first traffic statistic information is traffic statistic information acquired at 5 months, 28 days and 9 days; the first average value may be an average value of the latest 3 times at the same time, and if the predicted data traffic is 5 months, 28 days and 10 days, 00, the average value may be calculated for the traffic statistics obtained by 5 months, 27 days, 5 months, 26 days, 5 months, 25 days and 10 days, 00; the second average may be an average of traffic statistics obtained on the last day, taking a statistical period of 1 hour as an example, if data traffic of 5 months, 28 days, 10. And distributing 50% weight to the first flow statistical information, distributing 30% weight to the first average value, distributing 20% weight to the second average value, and obtaining a data flow prediction result of the cloud storage resource pool in the next preset acquisition period through weighting operation.

And secondly, calculating to obtain the remaining storage period of the cloud storage resource pool according to the recorded historical flow statistical information and the total storage period of the cloud storage resource pool.

Based on the recorded historical traffic statistical information, the earliest storage time point in the stored data can be determined, and the used storage period in the total storage period can be obtained by subtracting the used storage period from the current time point. The remaining storage period may also be obtained by subtracting the current time point from the next cyclic coverage time point, which is the coverage time point calculated in fig. 3 plus the storage period. For example, if the earliest storage time point in the historical traffic statistic information is 7 hours, the storage period is 5 hours, and the current time point is 9.

And thirdly, calculating the storage capacity required by the residual storage period by the product operation of the proportion of the residual storage period in the preset acquisition period and the data flow of the cloud storage resource pool in the next preset acquisition period.

After the data traffic of the cloud storage resource pool in the next preset acquisition period is obtained through calculation, the ratio of the remaining storage period to the preset acquisition period and the data traffic of the cloud storage resource pool in the next preset acquisition period may be multiplied on the assumption that the data traffic in each preset acquisition period in the remaining storage period is the same, so as to obtain a prediction result of the storage capacity required by the remaining storage period. For example, if the remaining storage period is 3 hours and the preset acquisition period is 1 hour, the remaining storage period is 3 times of the preset acquisition period, that is, the data traffic of the cloud storage resource pool in the next preset acquisition period is multiplied by 3 to obtain a prediction result of the storage capacity required by the remaining storage period.

And S403, judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity.

After the storage capacity is obtained, the remaining storage space of the cloud storage resource pool needs to be compared with the storage capacity, if the remaining storage space is smaller than the storage capacity, it is indicated that the remaining storage space cannot meet the requirement of the storage capacity, the remaining storage space needs to be adjusted, and if the remaining storage space is larger than or equal to the storage capacity, it is indicated that the remaining storage space can meet the requirement of the storage capacity, and the remaining storage space does not need to be adjusted.

And S404, if the current storage space is smaller than the preset storage space, adjusting the residual storage space of the cloud storage resource pool through a preset adjustment storage strategy.

If the residual storage space cannot meet the requirement of the storage capacity, the residual storage space needs to be adjusted, the preset adjustment storage strategy can be to allocate a virtual storage space to the cloud storage resource pool and delete historical data, or for the stored video data, adjustment operations such as data frame extraction and unloading, non-key frame filtering and the like can be performed.

Since the storage capacity required by the remaining storage period is a predicted value and may be different from the actual value, in order to ensure the integrity of data storage, a method of allocating a temporary storage space may be considered preferentially to adjust the remaining storage space.

Alternatively, S404 may be implemented by the following steps.

if so, allocating a temporary storage space for the cloud storage resource pool, wherein the temporary storage space is a storage space with unused storage capacity in the cloud storage system as a first capacity difference value;

if not, allocating unused storage space in the cloud storage system to a cloud storage resource pool; generating a capacity alarm instruction; and sending a capacity alarm instruction to the user side.

The management server preferentially allocates a temporary storage space to the cloud storage resource pool, if the unused storage space in the cloud storage system is greater than or equal to the difference value between the storage capacity and the residual storage space, it indicates that there is enough unused storage space in the cloud storage system for storing the predicted data traffic, the temporary storage space can be allocated to the cloud storage resource pool, and the storage capacity of the temporary storage space is equal to the difference value between the storage capacity and the residual storage space. If the unused storage space in the cloud storage system is smaller than the difference between the storage capacity and the remaining storage space, it indicates that the storage space is still not enough even if the unused storage space is allocated to the cloud storage resource pool, so that the unused storage space can be allocated to the cloud storage resource pool, and a capacity warning instruction is generated and sent to the user side to remind the user side of increasing the storage capacity.

When the unused storage space in the cloud storage system is smaller than the difference between the storage capacity and the remaining storage space, the storage space is still insufficient even if the unused storage space is allocated to the cloud storage resource pool, so that the occupied storage space can be released by adopting a historical data deleting mode besides sending a capacity alarm instruction to a user.

Optionally, after the step of allocating unused storage space in the cloud storage system to the cloud storage resource pool, an operation of deleting historical data may be further performed to release the occupied storage space. The operation flow of deleting the historical data is shown in fig. 6, and may include the following steps:

s601, subtracting the storage capacity, the residual storage space and unused storage space in the cloud storage system to obtain the capacity of the space to be deleted;

s602, accumulating all historical traffic statistical information from the recorded historical traffic statistical information with the earliest storage time point until the accumulated value is greater than or equal to the capacity of the space to be deleted;

s603, if the accumulated value is larger than the capacity of the space to be deleted, determining that the storage time point in the last-but-one historical traffic statistical information when the accumulation is stopped is a deletion time point; if the accumulated value is equal to the capacity of the space to be deleted, determining that the storage time point in the last historical traffic statistic information when accumulation is stopped is a deletion time point;

s604, sending the deletion time point to each storage server, so that each storage server deletes all data whose storage time is before the deletion time point.

The total deleted space may be a storage space that needs to be released, which is obtained by subtracting the storage capacity, the remaining storage space, and an unused storage space in the cloud storage system, that is, how much storage space is needed if all the predicted data traffic is to be completely stored. After the storage space needing to be released is calculated, according to the recorded historical flow statistical information of the cloud storage resource pool, the deletion time point can be estimated, and the deletion time point is issued to all the storage servers, so that all the data of which the storage time is before the deletion time point are deleted by all the storage servers.

The method of estimating the deletion time point is as follows:

and accumulating the historical traffic statistical information from the recorded historical traffic statistical information with the earliest storage time point until the accumulated value is greater than or equal to the capacity of the space to be deleted. If the accumulated value is equal to the space capacity to be deleted, selecting the time point of the statistic value (namely the storage time point in the last historical flow statistic information when accumulation is stopped); and if the accumulated value is larger than the capacity of the space to be deleted, selecting the time point of the last statistical value (namely, the storage time point in the last historical traffic statistical information when accumulation is stopped is the deletion time point).

Because the data stored in the cloud storage system may be video data, and for the video data, the video data may be specially processed to achieve the purpose of releasing the storage space without performing an operation of deleting the historical data, optionally, after the step of allocating the unused storage space in the cloud storage system to the cloud storage resource pool, the management server may further perform:

step one, if video data are stored in each storage server, issuing a frame extraction and unloading instruction to each storage server to enable each storage server to reserve key frames of stored historical video data and delete non-key frames of the historical video data;

secondly, recording the frequency of issuing a frame extraction and unloading instruction;

thirdly, if the residual storage space of the cloud storage resource pool after the frame extraction and unloading is carried out is smaller than the storage capacity, judging whether the frequency of issuing the frame extraction and unloading instruction reaches a first preset frequency or not;

step four, if the number of times of the issued frame-extracting and unloading instruction reaches a first preset number of times, issuing a frame-extracting and unloading instruction and a non-key frame filtering instruction to each storage server so that each storage server can reserve the key frames of the stored historical video data, delete the non-key frames of the historical video data and filter the appointed non-key frames of the newly received video data; recording the frequency of issuing non-key frame filtering instructions;

fifthly, if the residual storage space of the cloud storage resource pool after frame extraction and unloading and non-key frame filtering is smaller than the storage capacity, judging whether the frequency of the issued non-key frame filtering instruction reaches a second preset frequency;

sixthly, if the frequency of issuing the non-key frame filtering instruction does not reach a second preset frequency, issuing a frame extracting and transferring instruction and a non-key frame filtering instruction to each storage server, and increasing the frequency of issuing the non-key frame filtering instruction;

and seventhly, if the frequency of the issued frame-extracting and memory-transferring instruction does not reach the first preset frequency, issuing the frame-extracting and memory-transferring instruction to each storage server, and increasing the frequency of issuing the frame-extracting and memory-transferring instruction.

The first preset times and the second preset times may be the same or different, and as for a video frame, the key frame often carries the most important information of the video, if the remaining storage space is not enough, it may be considered to process the video first, that is, to retain the key frame (I frame) and delete the non-key frame, if the requirement of the storage capacity cannot be met after a plurality of times of frame extraction and unloading operations, the non-key frame filtering processing may be performed on new video data, and the non-key frame filtering processing policy may include: only B frames, B frames and part of P frames are filtered, all P frames cannot be filtered in principle due to new video data, there is an upper limit to the filtering percentage of P frames, e.g. 30%, and the specific filtering upper limit and non-key frame filtering processing strategy are configurable.

If the residual storage space of the cloud storage resource pool after the frame extraction and unloading is larger than or equal to the storage capacity, namely the storage capacity prediction result of the cloud storage resource pool in the new round is that the residual storage space meets the predicted storage capacity, and the frequency of the issued frame extraction and unloading instructions does not reach a first preset frequency, stopping the frame extraction and unloading operation, and clearing the frequency of the frame extraction and unloading instructions; if the residual storage space of the cloud storage resource pool after the frame extraction and unloading and the non-key frame filtering are carried out is larger than or equal to the storage capacity, namely the storage capacity prediction result of the new cloud storage resource pool is that the residual storage space meets the predicted storage capacity, and the frequency of the issued non-key frame filtering instructions does not reach a second preset frequency, stopping filtering the appointed non-key frame of the newly received video data, clearing the frequency of the non-key frame filtering instructions, and only issuing the frame extraction and unloading instructions.

If the number of times of the issued non-key frame filtering instruction reaches a second preset number of times, the storage space is still insufficient, and the operation of deleting the historical data can be executed to release the occupied storage space. Optionally, after determining whether the number of times that the non-key frame filtering instruction has been issued reaches a second preset number of times, the method may further include the following steps:

if the frequency of issuing the non-key frame filtering instruction reaches a second preset frequency, differentiating the storage capacity, the residual storage space of the cloud storage resource pool after frame extraction and unloading and non-key frame filtering and the unused storage space in the cloud storage system to obtain the capacity of the space to be deleted;

if the accumulated value is larger than the capacity of the space to be deleted, determining that the storage time point in the last-but-one historical traffic statistical information when the accumulation is stopped is a deletion time point; if the accumulated value is equal to the capacity of the space to be deleted, determining that the storage time point in the last historical traffic statistic information when accumulation is stopped is a deletion time point;

and issuing the deletion time point to each storage server so that each storage server deletes all the video data with the storage time before the deletion time point.

The operation of deleting the historical data is the same as the operation of deleting the historical data performed after the step of allocating the unused storage space in the cloud storage system to the cloud storage resource pool, and is not described herein again.

By applying the embodiment, the management server obtains and records the traffic statistical information of the cloud storage resource pool counted by each storage server, predicts the storage capacity required by the remaining storage period according to the recorded historical traffic statistical information, and judges the size relationship between the prediction result and the remaining storage space of the cloud storage resource pool, if the remaining storage space is smaller than the storage capacity, it indicates that the remaining storage space is not enough to store the predicted data traffic, the remaining storage space of the cloud storage resource pool can be adjusted by presetting an adjustment storage strategy, so as to ensure the storage of data. Before receiving new data needing to be stored, the storage capacity needed by the residual storage period is predicted, and when the residual storage space cannot meet the prediction result, an operation of adjusting the residual storage space is performed, that is, before receiving the new data needing to be stored, the residual storage space of the cloud storage resource pool is adjusted to be capable of storing the new data, so that the storage efficiency of the data is improved.

Corresponding to the foregoing method embodiment, an embodiment of the present invention further provides a device for adjusting a storage space of a cloud storage resource pool, and as shown in fig. 7, the device for adjusting a storage space of a cloud storage resource pool may include:

an obtaining module 710, configured to obtain and record traffic statistics information of the cloud storage resource pool counted by each storage server;

a prediction module 720, configured to predict, according to the recorded historical traffic statistics information, a storage capacity required by the remaining storage period;

a determining module 730, configured to determine whether a remaining storage space of the cloud storage resource pool is smaller than the storage capacity;

an adjusting module 740, configured to adjust the remaining storage space of the cloud storage resource pool by presetting an adjustment storage policy if the determination result of the determining module 730 is that the remaining storage space of the cloud storage resource pool is smaller than the storage capacity.

Optionally, the obtaining module 710 may be specifically configured to:

and receiving the flow statistical information sent by each storage server.

Optionally, the obtaining module 710 may be specifically configured to:

Optionally, the prediction module 720 may be specifically configured to:

Optionally, the adjusting module 740 may be specifically configured to:

Optionally, the adjusting module 740 may be further configured to:

if the accumulated value is larger than the capacity of the space to be deleted, determining that the storage time point in the last-but-one historical traffic statistical information when accumulation is stopped is a deletion time point; if the accumulated value is equal to the space capacity to be deleted, determining that the storage time point in the last historical flow statistical information when accumulation is stopped is a deletion time point;

By applying the embodiment, the management server obtains and records the traffic statistical information of the cloud storage resource pool counted by each storage server, predicts the storage capacity required by the remaining storage period according to the recorded historical traffic statistical information, and judges the size relationship between the prediction result and the remaining storage space of the cloud storage resource pool, if the remaining storage space is smaller than the storage capacity, it indicates that the remaining storage space is not enough to store the predicted data traffic, the remaining storage space of the cloud storage resource pool can be adjusted by presetting an adjustment storage strategy, so as to ensure the storage of data. Before receiving new data needing to be stored, the storage capacity needed by the residual storage period is predicted, and when the residual storage space cannot meet the prediction result, operation of adjusting the residual storage space is performed, that is, before receiving the new data needing to be stored, the residual storage space of the cloud storage resource pool is adjusted to be capable of storing the new data, so that the storage efficiency of the data is improved.

Corresponding to the above embodiments, an embodiment of the present invention further provides a cloud storage system, as shown in fig. 8, which may include a management server 801 and a plurality of storage servers 802;

a management server 801, configured to acquire and record traffic statistics information of the cloud storage resource pool counted by each storage server 802; predicting the storage capacity required by the residual storage period according to the recorded historical flow statistical information; judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity; if the current storage space is smaller than the preset storage space, adjusting the residual storage space of the cloud storage resource pool through a preset adjustment storage strategy;

the storage server 802 is configured to count traffic statistics information of a cloud storage resource pool, and send the counted traffic statistics information to the management server 801; the data is stored.

Optionally, the management server 801 is specifically configured to:

issuing information acquisition instructions to each storage server 802;

receiving the traffic statistics sent by each storage server 802;

accordingly, each storage server 802 is specifically configured to:

after receiving the information acquisition instruction, sending the statistical traffic statistical information of the cloud storage resource pool to the management server 801.

Optionally, the management server 801 is specifically configured to:

periodically acquiring traffic statistical information of the cloud storage resource pool, which is counted by each storage server 802, according to a preset acquisition period;

the management server 801 is further configured to:

Optionally, the management server 801 is specifically configured to:

if not, allocating the unused storage space in the cloud storage system to the cloud storage resource pool; generating a capacity alarm instruction; and sending the capacity alarm instruction to a user side.

Optionally, the management server 801 is further configured to:

issuing the deletion time point to each storage server 802;

correspondingly, each storage server 802 is specifically configured to:

deleting all data whose storage time is before the deletion time point.

Optionally, the management server 801 is further configured to:

if the video data is stored in each storage server 802, a frame extraction and transfer instruction is issued to each storage server 802;

if the number of times of issuing the frame extracting and unloading instruction reaches the first preset number of times, issuing the frame extracting and unloading instruction and a non-key frame filtering instruction to each storage server 802; recording the frequency of issuing the non-key frame filtering instruction;

if the number of times of issuing the non-key frame filtering instruction does not reach the second preset number of times, issuing the frame extracting and unloading instruction and the non-key frame filtering instruction to each storage server 802, and increasing the number of times of issuing the non-key frame filtering instruction;

if the number of times of issuing the frame-extracting and memory-transferring instruction does not reach the first preset number of times, issuing the frame-extracting and memory-transferring instruction to each storage server 802, and increasing the number of times of issuing the frame-extracting and memory-transferring instruction;

accordingly, each storage server 802 is specifically configured to:

after the frame extraction and unloading instruction is received, reserving key frames of the stored historical video data, and deleting non-key frames of the historical video data;

accordingly, each storage server 802 is specifically configured to:

after the frame extraction and storage instruction and the non-key frame filtering instruction are received, the key frames of the stored historical video data are reserved, the non-key frames of the historical video data are deleted, and the appointed non-key frames of the newly received video data are filtered.

Optionally, the management server 801 is further configured to:

issuing the deletion time point to each storage server 802;

correspondingly, each storage server 802 is specifically configured to:

deleting all video data whose storage time is before the deletion time point.

By applying the embodiment, the management server obtains and records the traffic statistic information of the cloud storage resource pool counted by each storage server, predicts the storage capacity required by the residual storage period according to the recorded historical traffic statistic information, judges the size relationship between the prediction result and the residual storage space of the cloud storage resource pool, and adjusts the residual storage space of the cloud storage resource pool by presetting an adjustment storage strategy to ensure the storage of data if the residual storage space is smaller than the storage capacity and indicates that the residual storage space is not enough to store the predicted data traffic. Before receiving new data needing to be stored, the storage capacity needed by the residual storage period is predicted, and when the residual storage space cannot meet the prediction result, operation of adjusting the residual storage space is performed, that is, before receiving the new data needing to be stored, the residual storage space of the cloud storage resource pool is adjusted to be capable of storing the new data, so that the storage efficiency of the data is improved.

An embodiment of the present invention further provides a management server, as shown in fig. 9, which may include a processor 901 and a memory 902;

the memory 902 is used for storing computer programs;

the processor 901 is configured to implement all the steps of the method for adjusting the storage space of the cloud storage resource pool provided by the embodiment of the present invention when executing the program stored in the memory 902.

The Memory may include a RAM (Random Access Memory) or an NVM (Non-Volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor including a CPU (Central Processing Unit), an NP (Network Processor), and the like; but also a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

Through above-mentioned management server, can realize: the management server predicts the storage capacity required by the residual storage period according to the recorded historical flow statistical information by acquiring and recording the flow statistical information of the cloud storage resource pool counted by each storage server, and adjusts the residual storage space of the cloud storage resource pool by presetting an adjustment storage strategy to ensure the storage of data if the residual storage space is smaller than the storage capacity and indicates that the residual storage space is not enough to store the predicted data flow by judging the size relationship between the prediction result and the residual storage space of the cloud storage resource pool. Before receiving new data needing to be stored, the storage capacity needed by the residual storage period is predicted, and when the residual storage space cannot meet the prediction result, operation of adjusting the residual storage space is performed, that is, before receiving the new data needing to be stored, the residual storage space of the cloud storage resource pool is adjusted to be capable of storing the new data, so that the storage efficiency of the data is improved.

In addition, corresponding to the method for adjusting the storage space of the cloud storage resource pool provided in the foregoing embodiment, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, all the steps of the method for adjusting the storage space of the cloud storage resource pool provided in the embodiment of the present invention are implemented.

The computer-readable storage medium stores an application program that executes the method for adjusting the storage space of the cloud storage resource pool provided by the embodiment of the invention when the application program runs, so that the method can be implemented as follows: the management server predicts the storage capacity required by the residual storage period according to the recorded historical flow statistical information by acquiring and recording the flow statistical information of the cloud storage resource pool counted by each storage server, and adjusts the residual storage space of the cloud storage resource pool by presetting an adjustment storage strategy to ensure the storage of data if the residual storage space is smaller than the storage capacity and indicates that the residual storage space is not enough to store the predicted data flow by judging the size relationship between the prediction result and the residual storage space of the cloud storage resource pool. Before receiving new data needing to be stored, the storage capacity needed by the residual storage period is predicted, and when the residual storage space cannot meet the prediction result, an operation of adjusting the residual storage space is performed, that is, before receiving the new data needing to be stored, the residual storage space of the cloud storage resource pool is adjusted to be capable of storing the new data, so that the storage efficiency of the data is improved.

For the embodiments of the management server and the computer-readable storage medium, the contents of the related methods are substantially similar to those of the foregoing embodiments of the methods, so that the description is relatively simple, and for the relevant points, reference may be made to the partial description of the embodiments of the methods.

It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus, the management server and the computer-readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and in relation to the description, reference may be made to part of the description of the method embodiments.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A storage space adjusting method of a cloud storage resource pool is applied to a management server, and the method comprises the following steps:

acquiring and recording flow statistical information of the cloud storage resource pool counted by each storage server;

if the current storage space is smaller than the preset storage space, adjusting the residual storage space of the cloud storage resource pool through a preset adjustment storage strategy;

adjusting the remaining storage space of the cloud storage resource pool by presetting an adjustment storage strategy comprises:

2. The method according to claim 1, wherein the obtaining traffic statistics information of the cloud storage resource pool counted by each storage server comprises:

and receiving the flow statistic information sent by each storage server.

3. The method according to claim 1, wherein the obtaining traffic statistics information of the cloud storage resource pool counted by each storage server comprises:

4. The method of claim 3, wherein predicting the storage capacity required for the remaining storage period based on the recorded historical traffic statistics comprises:

5. The method of claim 1, wherein after the allocating unused storage space in the cloud storage system to the cloud storage resource pool, the method further comprises:

6. The method of claim 1, wherein after the allocating unused storage space in the cloud storage system to the cloud storage resource pool, the method further comprises:

7. The method according to claim 6, wherein after said determining whether the number of times the non-key frame filtering instruction has been issued reaches a second preset number, the method further comprises:

8. A storage space adjusting device of a cloud storage resource pool is applied to a management server, and the device comprises:

the judging module is used for judging whether the residual storage space of the cloud storage resource pool is smaller than the storage capacity or not;

the adjusting module is used for adjusting the residual storage space of the cloud storage resource pool through a preset adjusting storage strategy if the judging result of the judging module is that the residual storage space of the cloud storage resource pool is smaller than the storage capacity;

wherein, the adjusting module is specifically configured to:

9. The apparatus of claim 8, wherein the obtaining module is specifically configured to:

and receiving the flow statistic information sent by each storage server.

10. The apparatus of claim 8, wherein the obtaining module is specifically configured to:

periodically acquiring flow statistical information of the cloud storage resource pool counted by each storage server according to a preset acquisition period;

11. The apparatus of claim 10, wherein the prediction module is specifically configured to:

12. The apparatus of claim 8, wherein the adjustment module is further configured to:

13. The apparatus of claim 8, wherein the adjustment module is further configured to:

if the number of times of issuing the frame extracting and unloading instruction reaches the first preset number of times, issuing the frame extracting and unloading instruction and a non-key frame filtering instruction to each storage server so that each storage server can reserve key frames of stored historical video data, delete non-key frames of the historical video data and filter appointed non-key frames of newly received video data; recording the frequency of issuing the non-key frame filtering instruction;

14. The apparatus of claim 13, wherein the adjustment module is further configured to:

15. A cloud storage system, the system comprising: a management server and a plurality of storage servers;

adjusting the remaining storage space of the cloud storage resource pool by presetting an adjustment storage strategy comprises: judging whether the unused storage space in the cloud storage system is larger than or equal to a first capacity difference value, wherein the first capacity difference value is the difference value between the storage capacity and the residual storage space; if so, allocating a temporary storage space to the cloud storage resource pool, wherein the temporary storage space is a storage space with unused storage capacity in the cloud storage system as a first capacity difference value; if not, allocating unused storage space in the cloud storage system to the cloud storage resource pool; generating a capacity alarm instruction; sending the capacity alarm instruction to a user side;