CN112130767A - Storage pool use capacity determination method, device, equipment and medium - Google Patents
Storage pool use capacity determination method, device, equipment and medium Download PDFInfo
- Publication number
- CN112130767A CN112130767A CN202010986565.7A CN202010986565A CN112130767A CN 112130767 A CN112130767 A CN 112130767A CN 202010986565 A CN202010986565 A CN 202010986565A CN 112130767 A CN112130767 A CN 112130767A
- Authority
- CN
- China
- Prior art keywords
- capacity
- historical
- volume
- storage pool
- predicted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000004422 calculation algorithm Methods 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
Abstract
The application discloses a method, a device, equipment and a computer readable storage medium for determining the use capacity of a storage pool, wherein the method comprises the following steps: acquiring the historical use capacity of each volume in the storage pool in each historical time period; training the corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model; inputting the target historical use capacity of each volume into a corresponding post-training prediction model, and calculating the predicted use capacity of each volume in a prediction time period; and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity. According to the technical scheme, the trained prediction model is used for calculating the predicted use capacity of each volume, the capacity use result of the storage pool is obtained, and the capacity use result of the storage pool is obtained, so that a user can timely know the use condition of the capacity of the storage pool according to the capacity use result.
Description
Technical Field
The present application relates to the field of storage system technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for determining a usage capacity of a storage pool.
Background
When a storage pool is established on a storage system and a RAID (Redundant Arrays of Independent Disks) is added, the capacity of the storage pool decreases with the use of the storage system by a user.
When a user service is operated, in order to ensure normal operation of the user service, sufficient storage capacity needs to be ensured, but at present, generally, a user determines to perform processing such as capacity expansion on a storage system when finding that the storage capacity is insufficient, and this may cause that the user service cannot enjoy sufficient storage capacity, so that normal operation of the user service may be affected, that is, at present, a situation that the user service cannot normally operate due to the fact that the user cannot timely find that the capacity is insufficient and cannot timely perform processing such as capacity expansion on the storage pool may occur because the user cannot timely know the use condition of the storage pool capacity.
In summary, how to timely obtain the usage of the storage pool capacity is a technical problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of the above, an object of the present application is to provide a storage pool usage capacity determination method, apparatus, device and computer-readable storage medium, which are used for timely learning usage of storage pool capacity.
In order to achieve the above purpose, the present application provides the following technical solutions:
a storage pool usage capacity determination method, comprising:
acquiring the historical use capacity of each volume in the storage pool in each historical time period;
training a corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model;
inputting the target historical use capacity of each volume into the corresponding post-training prediction model, and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period;
and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the remaining total capacity.
Preferably, the training of the corresponding prediction model by using the historical usage capacity of each volume to obtain a trained prediction model includes:
initializing a weighting coefficient in the prediction model, and calculating the predicted use capacity of the volume in the (M + 1) th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model; wherein M is an integer greater than 0;
calculating a prediction error between the historical use capacity and the predicted use capacity of the volume in the (M + 1) th historical time period, and updating the weighting coefficient in the prediction model according to the prediction error by using an RLS algorithm;
and removing the historical use capacity of the volume in the 1 st historical time period, and executing the step of calculating the predicted use capacity of the volume in the M +1 th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model until the weighting coefficients in the prediction model are updated according to the prediction error between the historical use capacity and the predicted use capacity of the volume in the last historical time period by using the RLS algorithm so as to obtain the trained prediction model.
Preferably, calculating a prediction error between the historical usage capacity and the predicted usage capacity of the volume in the (M + 1) th historical time period, and updating the weighting coefficients in the prediction model according to the prediction error by using an RLS algorithm, includes:
using e (n) ═ d (n) — w (n-1)Tx (n) calculating the volume's historical usage capacity d (n) and predicted usage capacity w (n-1) for the M +1 historical time periodTPrediction error e (n) between x (n); w (n-1) is a vector formed by weighting coefficients obtained by updating the weighting coefficients in the prediction model at the previous time, x (n) is a vector formed by historical use capacities of the volume in the previous M historical time periods, and the initial value of w (n-1) is 0;
by using
Calculating a Kalman gain vector k (n), updating the weighting coefficients w (n) in the prediction model by using w (n) ═ w (n-1) + k (n) e (n), and utilizingBy using
Updating the inverse matrix P (n); wherein, the lambda is a forgetting factor,
the initial value of P (n) is-1I is a regularization parameter, and I is an identity matrix.
Preferably, the method further comprises the following steps:
by using
Correcting the forgetting factor lambda (n) and utilizing
Update and utilize M
Adjusting the inverse matrix P (n); wherein, (n) is the posterior error, (n) ═ w (n)Tx (n), gamma is a sensitive factor, round () represents rounding to get an integer, and a, b, c and m are constants.
Preferably, determining the capacity usage result of the storage pool according to the total predicted usage capacity and the total remaining capacity includes:
and dividing the residual total capacity by the predicted total using capacity to obtain the residual using time of the storage pool.
Preferably, determining the capacity usage result of the storage pool according to the total predicted usage capacity and the total remaining capacity includes:
and subtracting the total residual capacity from the total predicted used capacity to obtain the total residual capacity of the storage pool in the prediction time period.
Preferably, the method further comprises the following steps:
acquiring the actual use capacity of each volume in the prediction time period;
calculating a prediction error between an actual usage capacity of the volume and a predicted usage capacity over the prediction time period;
and updating the weighting coefficients in the trained prediction model according to the prediction error by using an RLS algorithm.
A storage pool usage capacity determination apparatus, comprising:
the first acquisition module is used for acquiring the historical use capacity of each volume in the storage pool in each historical time period;
the training module is used for training the corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model;
the first calculation module is used for inputting the target historical use capacity of each volume into the corresponding trained prediction model and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period;
and the determining module is used for obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity.
A storage pool usage capacity determination device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the storage pool usage capacity determination method as described in any one of the above when executing the computer program.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the storage pool usage capacity determination method as claimed in any one of the preceding claims.
The application provides a method, a device, equipment and a computer readable storage medium for determining the usage capacity of a storage pool, wherein the method comprises the following steps: acquiring the historical use capacity of each volume in the storage pool in each historical time period; training the corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model; inputting the target historical use capacity of each volume into a corresponding post-training prediction model, and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period; and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity.
According to the technical scheme disclosed by the application, the prediction model is trained by using the historical use capacity of each volume in the storage pool in each historical time period to obtain a trained prediction model, then the historical use capacity (namely the target historical use capacity) of each volume in a preset number of historical time periods before the prediction time period is input into the trained prediction model corresponding to the volume to respectively calculate the predicted use capacity of each volume in the prediction time period, the total predicted use capacity of the storage pool in the prediction time period is calculated according to the predicted use capacity of each volume in the prediction time period, and the capacity use result of the storage pool is obtained according to the total predicted use capacity and the remaining total capacity, so that a user can timely know the use condition of the storage pool according to the capacity use result of the storage pool, and can determine whether capacity expansion and other treatment are needed or not in advance according to the capacity use result, and further, when the storage pool needs to be subjected to processing such as capacity expansion and the like, the storage pool can be subjected to processing such as capacity expansion in advance, so that the user service can normally run.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a storage pool usage capacity determination method according to an embodiment of the present application;
FIG. 2 is a block diagram of a structure for training a prediction model according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a storage pool usage capacity determination apparatus according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a storage pool usage capacity determination device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, which shows a flowchart of a storage pool usage capacity determination method provided in an embodiment of the present application, a storage pool usage capacity determination method provided in an embodiment of the present application may include:
s11: historical usage capacity of each volume in the storage pool over historical time periods is obtained.
In view of the fact that the capacity expansion and other processing of the storage pool is generally performed only when a user finds that the storage capacity of the storage pool is insufficient, and the normal operation of the user service is affected, the present application provides a storage pool use capacity determination method, which is used for timely obtaining the use condition of the storage pool capacity, so as to provide a reference basis for the user to perform the capacity expansion and other processing of the storage pool according to the obtained use condition of the storage pool capacity, thereby enabling the user service to operate normally.
Specifically, the number of volumes included in the storage pool may be identified first, and the historical usage capacity of each volume in the storage pool in each historical time period may be obtained, where the historical time period may be in units of days, the number of the historical time periods may be d (d is a positive integer), and the d days may be consecutive, that is, the historical usage capacity of each volume in the storage pool per day in the past consecutive d days may be obtained.
Of course, the above mentioned historical time period may also be in units of weeks, etc., and the present application does not limit this.
S12: and training the corresponding prediction model by using each historical use capacity of each volume to obtain the trained prediction model.
After step S11 is performed, a prediction model (a one-to-one relationship between a prediction model and a volume) corresponding to each volume may be trained using the historical usage capacity of each volume in the storage pool in each historical time period, and the trained prediction model may be obtained through training so as to predict the usage capacity of the volume corresponding to the trained prediction model.
S13: and inputting the target historical use capacity of each volume into the corresponding post-training prediction model, and calculating the predicted use capacity of each volume in the prediction time period.
Wherein the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period.
After step S12 is completed, the historical usage capacity of each volume in the preset number of historical time periods before the predicted time period may be determined, that is, the target historical usage capacity of each volume may be determined, where the target historical usage capacity may be obtained from the historical usage capacity of the volume obtained in step S11 in each historical time period, specifically, the historical time periods in step S11 may be arranged in order from front to back in time to obtain a historical time period queue, and the preset number of historical time periods may be selected from the back of the historical time period queue, and the historical usage capacity in the preset number of historical time periods may be used as the target historical usage capacity, for example: assuming that the historical usage capacity of each volume per day for the past continuous 100 days (wherein, the historical usage capacity is recorded as day 1, day 2 … …, day 100 in order of time from front to back) is acquired in step S11, and the preset number is 10, the above-mentioned target historical usage capacity is the historical usage capacity of each day for the past continuous 10 days.
After the target historical usage capacity of each volume is obtained, the obtained target historical usage capacity of each volume may be input into a post-training prediction model corresponding to the volume, so as to respectively and correspondingly calculate the predicted usage capacity of each volume in the prediction time period. For example: in addition to the above example, after the historical usage capacity of each volume per day in the past continuous 10 days is taken as the target historical usage capacity, the target historical usage capacity corresponding to each volume may be input into the post-training prediction model corresponding to the volume to predict the predicted usage capacity of each volume in the storage pool in the prediction time period of tomorrow.
S14: and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity.
After step S13 is performed, the predicted usage capacity of each volume in the storage pool for the predicted time period may be accumulated to obtain the total predicted usage capacity of the storage pool for the predicted time period. Because the usage rules of each volume in the storage pool are different, the statistics and prediction of the usage rule of each volume can be realized by taking each volume in the storage pool as an object to perform historical usage capacity acquisition, prediction model training and calculation of predicted usage capacity in a prediction time period, so that the accuracy of obtaining the total predicted usage capacity of the storage pool in the prediction time period is improved conveniently.
After the predicted total usage capacity of the storage pool in the prediction time period is obtained, the capacity usage result of the storage pool can be determined according to the predicted total usage capacity of the storage pool in the prediction time period and the current remaining total capacity of the storage pool, so that a user can timely know the usage condition of the storage pool capacity according to the capacity usage result, and can conveniently predetermine whether the storage pool capacity meets the user service operation requirement or not according to the usage condition of the storage pool capacity, and the user can timely purchase a new storage medium when the predetermined storage pool capacity does not meet the user service operation requirement and determine how large the storage medium needs to be purchased according to the capacity usage result, so that the user service can normally operate.
According to the technical scheme disclosed by the application, the prediction model is trained by using the historical use capacity of each volume in the storage pool in each historical time period to obtain a trained prediction model, then the historical use capacity (namely the target historical use capacity) of each volume in a preset number of historical time periods before the prediction time period is input into the trained prediction model corresponding to the volume to respectively calculate the predicted use capacity of each volume in the prediction time period, the total predicted use capacity of the storage pool in the prediction time period is calculated according to the predicted use capacity of each volume in the prediction time period, and the capacity use result of the storage pool is obtained according to the total predicted use capacity and the remaining total capacity, so that a user can timely know the use condition of the storage pool according to the capacity use result of the storage pool, and can determine whether capacity expansion and other treatment are needed or not in advance according to the capacity use result, and further, when the storage pool needs to be subjected to processing such as capacity expansion and the like, the storage pool can be subjected to processing such as capacity expansion in advance, so that the user service can normally run.
Referring to fig. 2, a structural block diagram for training a prediction model provided in an embodiment of the present application is shown. In the storage pool usage capacity determining method provided in the embodiment of the present application, the method for training the corresponding prediction model by using each historical usage capacity of each volume to obtain a trained prediction model may include:
initializing a weighting coefficient in the prediction model, and calculating the predicted use capacity of the volume in the (M + 1) th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model; wherein M is an integer greater than 0;
calculating a prediction error between the historical use capacity and the predicted use capacity of the volume in the (M + 1) th historical time period, and updating a weighting coefficient in the prediction model according to the prediction error by using an RLS (recursive least squares) algorithm;
and removing the historical use capacity of the volume in the 1 st historical time period, and executing the step of calculating the predicted use capacity of the volume in the M +1 th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model until the weighting coefficient in the prediction model is updated according to the prediction error between the historical use capacity and the predicted use capacity of the volume in the last historical time period by using the RLS algorithm so as to obtain the trained prediction model.
When the corresponding prediction model is trained by using the historical use capacity of each volume, the weighting coefficients in the prediction model (which are coefficients of the filter) may be initialized, so that the prediction model after the weighting coefficient initialization may be put into training, thereby facilitating the correction of the weighting coefficients in the prediction model.
Then, the predicted usage capacity of each volume in the (M + 1) th history period can be calculated correspondingly according to the history usage capacity of the volume in the previous M history periods and the prediction model after the weighting coefficient initialization, specifically, the predicted usage capacity of each volume in the M +1 th history period can be used
And calculating the predicted usage capacity y (n) of the volume in the M +1 th historical time period, wherein M is an integer larger than 0.
Then, according to the historical usage capacity d (n) of the acquired volume in the (M + 1) th historical time period and the calculated predicted usage capacity y (n) of the volume in the (M + 1) th historical time period, a prediction error e (n) between the historical usage capacity and the predicted usage capacity of the volume in the (M + 1) th historical time period is calculated by using e (n) ═ d (n) -y (n), and a weighting coefficient w (n) in a prediction model is subjected to the prediction error e (n) by using an RLS (Recursive Least squares) algorithm1(n),w2(n)…wM(n) are calculated and updated so that the filter can operate in an optimal state, i.e. so that the cost function in the RLS algorithm is minimized.
After updating the weighting coefficients in the prediction model, the historical usage capacity of the volume in the 1 st historical time period may be removed, and the step of calculating the predicted usage capacity of the volume in the M +1 th historical time period according to the historical usage capacity of the volume in the previous M historical time periods and the prediction model may be performed, that is, the predicted usage capacity of the volume in the M +1 th historical time period (the current M historical time periods are from the 2 nd historical time period to the M +1 th historical time period obtained in step S11) may be calculated according to the historical usage capacity of the previous M historical time periods and the prediction model after updating the weighting coefficients in the M +1 th historical time period (the current M +1 th historical time period is the M +2 th historical time period obtained in step S11), and the step of calculating the prediction error may be performed continuously, and the weighting coefficients in the prediction model may be updated according to the prediction error by using the RLS algorithm, and the like, and updating the weighting coefficients in the prediction model according to the prediction error between the historical use capacity and the predicted use capacity of the volume in the last historical time period by using an RLS algorithm until the predicted use capacity of the last historical time period is calculated, so that a trained prediction model is obtained according to the weighting coefficients updated at the last time, and the predicted use capacity of the corresponding volume in the prediction time period is calculated by using the trained prediction model.
It should be noted that LMS (Least Mean Square) may also be used to update the weighting coefficients in the prediction model, but considering that the convergence rate of the RLS algorithm is one order of magnitude faster than that of the LMS algorithm, the RLS algorithm may be preferably used to perform update calculation so as to improve the training rate of the prediction model.
The storage pool usage capacity determining method provided by the embodiment of the present application, which calculates a prediction error between a historical usage capacity and a predicted usage capacity of a volume in an M +1 th historical time period, and updates a weighting coefficient in a prediction model according to the prediction error by using an RLS algorithm, may include:
using e (n) ═ d (n) — w (n-1)Tx (n) calculating the historical usage capacity d (n) and predicted usage capacity w (n-1) of the volume in the M +1 historical time periodTPrediction error e (n) between x (n); w (n-1) is a vector formed by weighting coefficients obtained by updating the weighting coefficients in the prediction model at the previous time, x (n) is a vector formed by historical use capacities of volumes in the previous M historical time periods, and w (n)-1) has an initial value of 0;
by using
Calculating a Kalman gain vector k (n), updating a weighting coefficient w (n) in the prediction model by using w (n) ═ w (n-1) + k (n) e (n), and using
Updating the inverse matrix P (n); wherein, the lambda is a forgetting factor,
the initial value of P (n) is-1I is a regularization parameter, and I is an identity matrix.
When calculating the prediction error between the historical use capacity and the predicted use capacity of the volume in the (M + 1) th historical time period and updating the weighting coefficient in the prediction model according to the prediction error by using the RLS algorithm, e (n) ═ d (n) — w (n-1) can be used firstlyTx (n) calculating the historical usage capacity d (n) and predicted usage capacity w (n-1) of the volume in the M +1 historical time periodTAnd x (n), wherein w (n-1) is a vector formed by weighting coefficients obtained by updating the weighting coefficients in the prediction model last time, and when initialization is performed, the initial value of w (n-1) is 0, namely w (0) is 0, and x (n) is a vector formed by historical use capacities of volumes in the previous M historical time periods.
At the same time, can utilize
Calculating a Kalman gain vector k (n), updating a weighting coefficient w (n) in the prediction model by using w (n) ═ w (n-1) + k (n) e (n), and using
Updating an inverse matrix P (n), wherein the inverse matrix P (n) is calculated to be involved in the next updating of the Kalman gain vector k (n) and the weighting coefficient w (n), k (n) e (n) is a correction amount for correcting the weighting coefficient,and the obtained w (n) can be put into the calculation of the predicted use capacity and the updating calculation of the weighting coefficient in the M +1 th historical time period of the next round, wherein lambda is a forgetting factor,
the initial value of P (n) is-1I (i.e. P (0) ═ I-1I) And I is an identity matrix and is a regularization parameter, wherein the setting is related to the signal-to-noise ratio, a small value is taken when the signal-to-noise ratio is high, and a larger value is taken when the signal-to-noise ratio is low.
It should be noted that the preset number mentioned above can also be specifically defined by
A determination is made.
The method for determining the usage capacity of the storage pool provided by the embodiment of the application can further comprise the following steps:
by using
Correcting the forgetting factor lambda (n) and utilizing
Update and utilize M
Adjusting the inverse matrix P (n); wherein, (n) is the posterior error, (n) ═ w (n)Tx (n), gamma is a sensitive factor, round () represents rounding to get an integer, and a, b, c and m are constants.
In the application, considering that the forgetting factor lambda is small, the tracking capability is strong, the change rule of the use capacity can be quickly converged when being switched, and when the forgetting factor is large, the noise resistance capability is strong, therefore, before the predicted use capacity of the volume in the (M + 1) th historical time period is calculated according to the historical use capacity of the volume in the previous M historical time periods and the prediction model, the predicted use capacity of the volume in the (M + 1) th historical time period can be utilized firstly
Correcting the forgetting factor lambda (n) and utilizing
Update M and utilize
Adjusting the inverse matrix P (n) to select the historical time period according to the updated M, and calculating the weighting coefficient according to the corrected lambda (n) and the adjusted P (n), wherein,
the method is obtained by using two S functions (Sigmoid function and hyperbolic tangent function) commonly used in a neural network, wherein a, b, c and m are constants, a constant b and a constant c control the value range of the function, the constant a and the constant m control the convergence speed of the hyperbolic tangent function and improve the shape of the top of the inverted curved surface, the constant a is preferably equal to 20, the constant b is preferably equal to 0.6, the constant c is preferably equal to 0.4, the constant m is preferably equal to 150, the (n) is the posterior error (the prediction error e (n) is the prior error), and the (n) is w (n)Tx (n), gamma is a sensitive factor, and round () represents rounding to an integer.
The prediction model can have higher accuracy or noise resistance by modifying and adjusting the forgetting factor and the inverse matrix. In addition, the above-mentioned preset number can also be set by
And updating and determining so that a small number of target historical use capacities can be selected to calculate the predicted use capacity when the historical use capacity changes greatly to improve flexibility and reduce the influence of the historical use capacity on the calculation of the predicted use capacity, thereby improving the accuracy of the calculation of the predicted use capacity, and a large number of target use capacities can be selected to calculate the predicted use capacity when the historical use capacity changes greatly to improve the noise resistance.
The method for determining the usage capacity of the storage pool according to the embodiment of the application, which determines the capacity usage result of the storage pool according to the predicted total usage capacity and the remaining total capacity, may include:
and dividing the residual total capacity by the predicted total using capacity to obtain the residual using time of the storage pool.
When the capacity utilization result of the storage pool is determined according to the predicted total utilization capacity and the remaining total utilization capacity, the remaining total capacity and the predicted total utilization capacity can be divided to obtain the remaining utilization time of the storage pool, so that a user can know the remaining utilization time of the storage pool in time, and the user can determine whether to perform capacity expansion and other processing on the storage pool according to the remaining utilization time of the storage pool.
In addition, after the remaining usage time of the storage pool is obtained, the remaining usage time of the storage pool may be compared with a preset remaining time (which may be set by a user according to experience or a user service operation requirement), and if the remaining usage time of the storage pool is less than the preset remaining time, a prompt may be sent, so that the user may perform processing such as capacity expansion in time according to the prompt, thereby increasing the remaining total capacity of the storage pool.
The method for determining the usage capacity of the storage pool according to the embodiment of the application, which determines the capacity usage result of the storage pool according to the predicted total usage capacity and the remaining total capacity, may include:
and subtracting the total capacity used in the prediction from the residual total capacity to obtain the residual total capacity of the storage pool in the prediction time period.
When the capacity utilization result of the storage pool is determined according to the total predicted utilization capacity and the total remaining utilization capacity, the total remaining capacity and the total predicted utilization capacity are divided to obtain the remaining utilization time of the storage pool, and the total remaining capacity and the total predicted utilization capacity are subtracted to obtain the total remaining capacity of the storage pool in the prediction time period, so that a user can know the utilization condition of the storage pool according to the total remaining capacity of the storage pool in the prediction time period, and can decide whether to carry out capacity expansion and other processing on the storage pool in advance according to the total remaining capacity of the storage pool in the prediction time period.
In addition, after the remaining total capacity of the storage pool in the prediction time period is obtained through calculation, the remaining total capacity of the storage pool in the prediction time period may be compared with a preset remaining total capacity (which may be set by a user according to experience or a user service operation requirement), and if the remaining total capacity of the storage pool in the prediction time period is smaller than the preset remaining total capacity, a prompt may be sent, so that the user may perform processing such as capacity expansion in time according to the prompt, and the remaining total capacity of the storage pool may be increased.
The method for determining the usage capacity of the storage pool provided by the embodiment of the application can further comprise the following steps:
acquiring the actual use capacity of each volume in a prediction time period;
calculating a prediction error between an actual usage capacity of the volume and a predicted usage capacity within a prediction time period;
and updating the weighting coefficients in the trained prediction model according to the prediction error by using an RLS algorithm.
In the application, after the time reaches the prediction time period, the actual use capacity of each volume in the prediction time period can be obtained, the prediction error between the actual use capacity of each volume and the predicted use capacity in the prediction time period is calculated, and the weighting coefficient in the trained prediction model is updated according to the prediction error by using the RLS algorithm, so that the updated trained prediction model can be better put into the subsequent use capacity prediction and calculation, and the accuracy of use capacity prediction is improved.
An embodiment of the present application further provides a storage pool usage capacity determining apparatus, and referring to fig. 3, it shows a schematic structural diagram of a storage pool usage capacity determining apparatus provided in an embodiment of the present application, and the storage pool usage capacity determining apparatus may include:
a first obtaining module 31, configured to obtain a historical usage capacity of each volume in the storage pool in each historical time period;
the training module 32 is configured to train the corresponding prediction model by using each historical usage capacity of each volume to obtain a trained prediction model;
the first calculation module 33 is configured to input the target historical usage capacity of each volume into the corresponding post-training prediction model, and calculate the predicted usage capacity of each volume in the prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period;
and the determining module 34 is configured to obtain a total predicted usage capacity of the storage pool in the predicted time period according to the predicted usage capacity of each volume, and determine a capacity usage result of the storage pool according to the total predicted usage capacity and the remaining total capacity.
In an apparatus for determining a storage pool usage capacity provided by an embodiment of the present application, the training module 32 may include:
the first calculation unit is used for initializing the weighting coefficient in the prediction model and calculating the predicted use capacity of the volume in the (M + 1) th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model; wherein M is an integer greater than 0;
the second calculation unit is used for calculating the prediction error between the historical use capacity and the predicted use capacity of the volume in the M +1 th historical time period and updating the weighting coefficient in the prediction model according to the prediction error by using an RLS algorithm;
and the execution unit is used for removing the historical use capacity of the volume in the 1 st historical time period, and executing the step of calculating the predicted use capacity of the volume in the M +1 th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model until the weighting coefficient in the prediction model is updated by using an RLS algorithm according to the prediction error between the historical use capacity and the predicted use capacity of the volume in the last historical time period so as to obtain the trained prediction model.
In an apparatus for determining a storage pool usage capacity provided by an embodiment of the present application, the second computing unit may include:
a first calculating subunit for using e (n) ═ d (n) — w (n-1)Tx (n) calculating the historical usage capacity d (n) and predicted usage capacity w (n-1) of the volume in the M +1 historical time periodTPrediction error between x (n)e (n); w (n-1) is a vector formed by weighting coefficients obtained by updating the weighting coefficients in the prediction model at the previous time, x (n) is a vector formed by historical use capacities of volumes in the previous M historical time periods, and the initial value of w (n-1) is 0;
a second calculation subunit for utilizing
Calculating a Kalman gain vector k (n), updating a weighting coefficient w (n) in the prediction model by using w (n) ═ w (n-1) + k (n) e (n), and using
Updating the inverse matrix P (n); wherein, the lambda is a forgetting factor,
the initial value of P (n) is-1I is a regularization parameter, and I is an identity matrix.
In an embodiment of the storage pool usage capacity determination apparatus, the second computing unit may further include:
correction unit for using
Correcting the forgetting factor lambda (n) and utilizing
Update and utilize M
Adjusting the inverse matrix P (n); wherein, (n) is the posterior error, (n) ═ w (n)Tx (n), gamma is a sensitive factor, round () represents rounding to get an integer, and a, b, c and m are constants.
In an apparatus for determining a storage pool usage capacity provided by an embodiment of the present application, the determining module 34 may include:
and the dividing unit is used for dividing the residual total capacity and the predicted using total capacity to obtain the residual using time of the storage pool.
In an apparatus for determining a storage pool usage capacity provided by an embodiment of the present application, the determining module 34 may include:
and the subtracting unit is used for subtracting the residual total capacity from the predicted used total capacity to obtain the residual total capacity of the storage pool in the prediction time period.
The storage pool usage capacity determination apparatus provided in the embodiment of the present application may further include:
the second acquisition module is used for acquiring the actual use capacity of each volume in the prediction time period;
a second calculation module for calculating a prediction error between an actual usage capacity of the volume and a predicted usage capacity within a prediction time period;
and the updating module is used for updating the weighting coefficient in the trained prediction model according to the prediction error by using the RLS algorithm.
An embodiment of the present application further provides a storage pool usage capacity determination device, and referring to fig. 4, it shows a schematic structural diagram of a storage pool usage capacity determination device provided in an embodiment of the present application, and the storage pool usage capacity determination device may include:
a memory 41 for storing a computer program;
the processor 42, when executing the computer program stored in the memory 41, may implement the following steps:
acquiring the historical use capacity of each volume in the storage pool in each historical time period; training the corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model; inputting the target historical use capacity of each volume into a corresponding post-training prediction model, and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period; and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity.
An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the following steps may be implemented:
acquiring the historical use capacity of each volume in the storage pool in each historical time period; training the corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model; inputting the target historical use capacity of each volume into a corresponding post-training prediction model, and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period; and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For a description of a relevant part in a storage pool usage capacity determination device, a device, and a computer-readable storage medium provided in the embodiments of the present application, reference may be made to the detailed description of a corresponding part in a storage pool usage capacity determination method provided in the embodiments of the present application, and details are not described here again.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include elements inherent in the list. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for determining storage pool usage capacity, comprising:
acquiring the historical use capacity of each volume in the storage pool in each historical time period;
training a corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model;
inputting the target historical use capacity of each volume into the corresponding post-training prediction model, and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period;
and obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the remaining total capacity.
2. The storage pool usage capacity determination method of claim 1, wherein training a corresponding prediction model using each of said historical usage capacities for each of said volumes to obtain a trained prediction model comprises:
initializing a weighting coefficient in the prediction model, and calculating the predicted use capacity of the volume in the (M + 1) th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model; wherein M is an integer greater than 0;
calculating a prediction error between the historical use capacity and the predicted use capacity of the volume in the (M + 1) th historical time period, and updating the weighting coefficient in the prediction model according to the prediction error by using an RLS algorithm;
and removing the historical use capacity of the volume in the 1 st historical time period, and executing the step of calculating the predicted use capacity of the volume in the M +1 th historical time period according to the historical use capacity of the volume in the previous M historical time periods and the prediction model until the weighting coefficients in the prediction model are updated according to the prediction error between the historical use capacity and the predicted use capacity of the volume in the last historical time period by using the RLS algorithm so as to obtain the trained prediction model.
3. The storage pool usage capacity determination method of claim 2, wherein calculating a prediction error between historical usage capacity and predicted usage capacity of said volume for the M +1 th historical time period, and updating the weighting coefficients in said prediction model based on said prediction error using an RLS algorithm comprises:
using e (n) ═ d (n) — w (n-1)Tx (n) calculating the volume's historical usage capacity d (n) and predicted usage capacity w (n-1) for the M +1 historical time periodTPrediction error e (n) between x (n); w (n-1) is a vector formed by weighting coefficients obtained by updating the weighting coefficients in the prediction model at the previous time, x (n) is a vector formed by historical use capacities of the volume in the previous M historical time periods, and the initial value of w (n-1) is 0;
by using
Calculating a Kalman gain vector k (n) and using w (n) ═ w (n-1) + k (n) e (n) to weight coefficients in the prediction modelw (n) are updated and utilized
Updating the inverse matrix P (n); wherein, the lambda is a forgetting factor,
the initial value of P (n) is-1I is a regularization parameter, and I is an identity matrix.
4. The storage pool usage capacity determination method of claim 3, further comprising:
by using
Correcting the forgetting factor lambda (n) and utilizing
Update and utilize M
Adjusting the inverse matrix P (n); wherein, (n) is the posterior error, (n) ═ w (n)Tx (n), gamma is a sensitive factor, round () represents rounding to get an integer, and a, b, c and m are constants.
5. The storage pool usage capacity determination method of claim 1, wherein determining a capacity usage result for the storage pool based on the total predicted usage capacity and total remaining capacity comprises:
and dividing the residual total capacity by the predicted total using capacity to obtain the residual using time of the storage pool.
6. The storage pool usage capacity determination method of claim 1, wherein determining a capacity usage result for the storage pool based on the total predicted usage capacity and total remaining capacity comprises:
and subtracting the total residual capacity from the total predicted used capacity to obtain the total residual capacity of the storage pool in the prediction time period.
7. The storage pool usage capacity determination method of claim 5 or 6, further comprising:
acquiring the actual use capacity of each volume in the prediction time period;
calculating a prediction error between an actual usage capacity of the volume and a predicted usage capacity over the prediction time period;
and updating the weighting coefficients in the trained prediction model according to the prediction error by using an RLS algorithm.
8. A storage pool usage capacity determination apparatus, comprising:
the first acquisition module is used for acquiring the historical use capacity of each volume in the storage pool in each historical time period;
the training module is used for training the corresponding prediction model by using each historical use capacity of each volume to obtain a trained prediction model;
the first calculation module is used for inputting the target historical use capacity of each volume into the corresponding trained prediction model and calculating the predicted use capacity of each volume in a prediction time period; the target historical use capacity is the historical use capacity in a preset number of historical time periods before the prediction time period;
and the determining module is used for obtaining the total predicted used capacity of the storage pool in the predicted time period according to the predicted used capacity of each volume, and determining the capacity use result of the storage pool according to the total predicted used capacity and the residual total capacity.
9. A storage pool usage capacity determination device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the storage pool usage capacity determination method as claimed in any one of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the storage pool usage capacity determination method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010986565.7A CN112130767B (en) | 2020-09-18 | 2020-09-18 | Storage pool use capacity determination method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010986565.7A CN112130767B (en) | 2020-09-18 | 2020-09-18 | Storage pool use capacity determination method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112130767A true CN112130767A (en) | 2020-12-25 |
| CN112130767B CN112130767B (en) | 2023-01-10 |
Family
ID=73841368
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010986565.7A Active CN112130767B (en) | 2020-09-18 | 2020-09-18 | Storage pool use capacity determination method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112130767B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112799596A (en) * | 2021-02-03 | 2021-05-14 | 联想(北京)有限公司 | Capacity expansion control method and device for storage resources and electronic equipment |
| CN112988071A (en) * | 2021-03-15 | 2021-06-18 | 中国建设银行股份有限公司 | Distributed storage capacity expansion method and device, storage medium and equipment |
| CN114266422A (en) * | 2022-03-02 | 2022-04-01 | 北京唯捷创芯精测科技有限责任公司 | Reel quantity prediction method and device for integrated circuit sealing measurement |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130297905A1 (en) * | 2012-03-29 | 2013-11-07 | International Business Machines Corporation | Dynamic reconfiguration of storage system |
| CN109885469A (en) * | 2019-02-27 | 2019-06-14 | 深信服科技股份有限公司 | A kind of expansion method, prediction model creation method, device, equipment and medium |
-
2020
- 2020-09-18 CN CN202010986565.7A patent/CN112130767B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130297905A1 (en) * | 2012-03-29 | 2013-11-07 | International Business Machines Corporation | Dynamic reconfiguration of storage system |
| CN109885469A (en) * | 2019-02-27 | 2019-06-14 | 深信服科技股份有限公司 | A kind of expansion method, prediction model creation method, device, equipment and medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112799596A (en) * | 2021-02-03 | 2021-05-14 | 联想(北京)有限公司 | Capacity expansion control method and device for storage resources and electronic equipment |
| CN112988071A (en) * | 2021-03-15 | 2021-06-18 | 中国建设银行股份有限公司 | Distributed storage capacity expansion method and device, storage medium and equipment |
| CN114266422A (en) * | 2022-03-02 | 2022-04-01 | 北京唯捷创芯精测科技有限责任公司 | Reel quantity prediction method and device for integrated circuit sealing measurement |
| CN114266422B (en) * | 2022-03-02 | 2022-06-21 | 北京唯捷创芯精测科技有限责任公司 | Reel quantity prediction method and device for integrated circuit sealing measurement |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112130767B (en) | 2023-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112130767B (en) | Storage pool use capacity determination method, device, equipment and medium | |
| CN107808122B (en) | Target tracking method and device | |
| Ljung et al. | Stochastic approximation and optimization of random systems | |
| CN111758104B (en) | Neural network parameter optimization method and neural network calculation method and device suitable for hardware implementation | |
| JPWO2020159568A5 (en) | ||
| JP7044873B2 (en) | Methods, devices and computer programs for creating deep neural networks | |
| CN107457780B (en) | Method and device for controlling mechanical arm movement, storage medium and terminal equipment | |
| JP7468540B2 (en) | Learning system, learning device, and learning method | |
| CN102629105A (en) | Self-adaptive control method and self-adaptive control device | |
| CN110852425A (en) | Optimization-based neural network processing method and device and electronic system | |
| CN103578274B (en) | A kind of traffic flow forecasting method and device | |
| JP7294017B2 (en) | Information processing device, information processing method and information processing program | |
| WO2017198168A2 (en) | Reduction of parameters in fully connected layers of neural networks by low rank factorizations | |
| CN110866608B (en) | Self-adaptive learning rate calculation method | |
| CN114549739A (en) | Control system and method based on three-dimensional data model | |
| CN110650187B (en) | Node type determination method for edge node and target network | |
| JPH06180691A (en) | Adaptive input/output device | |
| CN117872754A (en) | Multi-target tracking iterative learning method | |
| JPH11501747A (en) | Design device for neuron network and neuron network | |
| CN111416595A (en) | Big data filtering method based on multi-core fusion | |
| CN116136752B (en) | Method and system for determining array input strategy | |
| WO2022013909A1 (en) | Graph calculation device, graph calculation method, and graph calculation program | |
| CN112508165A (en) | Apparatus, method, and non-transitory computer-readable storage medium for information processing | |
| CN115987248A (en) | Method and device for determining filter parameters of equalizer, electronic equipment and storage medium | |
| CN114969148A (en) | System access amount prediction method, medium and equipment based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |