CN111400019B - Method, device and computer readable storage medium for distributing business load - Google Patents
Method, device and computer readable storage medium for distributing business load Download PDFInfo
- Publication number
- CN111400019B CN111400019B CN201910000435.9A CN201910000435A CN111400019B CN 111400019 B CN111400019 B CN 111400019B CN 201910000435 A CN201910000435 A CN 201910000435A CN 111400019 B CN111400019 B CN 111400019B
- Authority
- CN
- China
- Prior art keywords
- service
- services
- storage pool
- target storage
- businesses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000004590 computer program Methods 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000015654 memory Effects 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 6
- 239000013598 vector Substances 0.000 description 5
- 238000013507 mapping Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The embodiment of the invention provides a method, a device and a computer readable storage medium for distributing service loads, wherein the method comprises the following steps: determining relatedness between storage load characteristics of a plurality of services; and allocating storage resources for the plurality of services in a target storage pool based on the determined affinity.
Description
Technical Field
The present invention relates to the field of mobile communications technologies, and in particular, to a method, an apparatus, and a computer readable storage medium for distributing a traffic load.
Background
With the breakthrough of new fields such as cloud computing, big data, mobile and social media, the big explosion of the data is promoted, and the storage requirement is rapidly increased. In addition to having to meet the capacity requirements of the upper layer services, the storage system needs to meet the performance requirements of these services.
The better hardware equipment configured by the storage system can provide better performance for upper-layer services, but the cost is increased; critical business exclusive storage systems can guarantee performance requirements, but often result in lower storage system utilization.
Different types of storage media (such as SATA SSD, PCIe SSD, SATA HDD, SAS HDD and the like) are likely to exist in a single storage system, and the different types of storage media form storage pools with different performances; meanwhile, the number of hard disks in a single storage pool is not too large, so that the storage reliability of each storage pool is ensured; there are typically multiple storage pools in a single storage system. Public cloud or private cloud resource pools typically include multiple storage pools for multiple storage systems.
The multiple services share the same storage resource, the different services have different demands on the storage resource, and how to effectively process the mapping relation between the multiple services and the multiple storage pools is the subject of personal study. The most commonly used method is to estimate C1 according to the capacity requirement of a service, and when the remaining space F1 in a certain storage pool is greater than the capacity requirement of the service (F1 > C1), a volume with the capacity of C1 can be created in the storage pool to be allocated to the service for use, and the remaining space of the storage pool is adjusted to be (F1-C1). In addition, in addition to taking into account the capacity requirements of the service, in some scenarios, the storage manager may also take into account the performance requirements of the service when allocating resources. In order to meet the capacity and performance requirements of the service, the service estimated resources have a large margin, which can lead to low utilization rate of the storage system.
The existing related art has the following disadvantages: the service capacity and the performance requirement belong to the predicted value, the real requirement of the service cannot be truly reflected, and the problems that the storage resource can meet the service performance requirement to ensure certain, the utilization rate of a storage system is low, the performance influence on the front-end service is caused by readjusting the storage resource of the service after the resource allocation are solved.
Disclosure of Invention
It is therefore desirable for embodiments of the present invention to provide a method, apparatus, and computer-readable storage medium for distributing traffic load.
In order to achieve the above purpose, the technical solution of the embodiment of the present invention is as follows:
the embodiment of the invention provides a method for distributing service loads, which comprises the following steps:
determining relatedness between storage load characteristics of a plurality of services;
and allocating storage resources for the plurality of services in a target storage pool based on the determined affinity.
Optionally, before determining the affinity between the storage load characteristics of the plurality of services, the method further includes:
the actual performance of the plurality of services is collected from the source storage system while the storage load characteristics of the plurality of services are collected.
Wherein the storage load feature comprises: capacity demand and load characteristics; the load characteristics include one or more of the following:
data block size distribution, read-write times per second IOPS performance distribution with time, bandwidth performance distribution with time, read-write request proportion and IO access interval distribution.
Optionally, after the allocating storage resources for the plurality of services, the method further includes:
determining the service performance satisfaction degree of the target storage pool to the allocated service;
and adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree.
Wherein said allocating storage resources for said plurality of services in a target storage pool based on said affinity determined comprises:
gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses;
and allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool and the capacity requirement of the service load.
Wherein the allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirement of the service load comprises:
determining loads of two businesses with highest weighted operation results to be stored in a target storage pool;
taking the two businesses with the highest weighted operation results as one business, carrying out the weighted operation of load characteristics with the other businesses, and storing the loads of the two businesses with the highest weighted operation results in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
Wherein said determining the satisfaction of the target storage pool to the service performance of the allocated service comprises:
and playing back the storage load characteristics of the allocated service in the target storage pool, and judging the service performance satisfaction degree of the target storage pool to the allocated service.
The adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree comprises the following steps:
determining that the target storage pool meets the service performance requirement of the allocated service, and ending the allocation of the service; otherwise, deleting the corresponding business from the target storage pool.
The embodiment of the invention also provides a device for distributing the service load, which comprises:
a determining module for determining relatedness between storage load characteristics of a plurality of services;
and the allocation module is used for allocating storage resources for the plurality of services in the target storage pool based on the determined affinity.
The embodiment of the invention also provides a device for distributing the service load, which comprises: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to execute the steps of the above method when the computer program is run.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the above method.
The method, the device and the computer-readable storage medium for distributing the service load provided by the embodiment of the invention determine the relatedness among the storage load characteristics of a plurality of services; and allocating storage resources for the plurality of services in a target storage pool based on the determined affinity. The embodiment of the invention fully considers the service storage load characteristic, adopts the real performance and capacity requirement of the service, considers the mapping relation between the service and the storage pool when planning the storage resource in advance, and ensures that the service has enough resource usage; in addition, the satisfaction degree of the target storage pool to the service performance is evaluated (determined), and meanwhile, different resource requirements of a plurality of services are adjusted in the same storage pool, so that the resource utilization rate is improved.
Drawings
Fig. 1 is a schematic flow chart of a method for distributing traffic load according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a second implementation flow of a method for distributing traffic load according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a third implementation flow of a method for distributing traffic load according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an apparatus for distributing traffic load according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a second device for distributing traffic load according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a device for distributing traffic load according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a method for distributing traffic load according to an embodiment of the present invention.
Detailed Description
The invention is described below with reference to the drawings and examples.
The embodiment of the invention provides a method for distributing service loads, as shown in fig. 1, comprising the following steps:
step 101: determining relatedness between storage load characteristics of a plurality of services;
step 102: and allocating storage resources for the plurality of services in a target storage pool based on the determined affinity.
The embodiment of the invention fully considers the service storage load characteristic, adopts the real performance and capacity requirement of the service, considers the mapping relation between the service and the storage pool when planning the storage resource in advance, and ensures that the service has enough resource usage.
In one embodiment, before determining the relatedness between the storage load characteristics of the plurality of services, as shown in fig. 2, the method further comprises:
step 100: the actual performance of the plurality of services is collected from the source storage system while the storage load characteristics of the plurality of services are collected.
In an embodiment of the present invention, the storage load feature includes: capacity demand and load characteristics; the load characteristics include one or more of the following:
data block size distribution, read-write times per second IOPS performance distribution with time, bandwidth performance distribution with time, read-write request proportion and IO access interval distribution.
In one embodiment, as shown in fig. 3, after the storage resources are allocated to the plurality of services, the method further includes:
step 103: determining the service performance satisfaction degree of the target storage pool to the allocated service;
and adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree.
In an embodiment of the present invention, the allocating storage resources for the plurality of services in the target storage pool based on the determined affinity includes:
gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses;
and allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool and the capacity requirement of the service load.
In an embodiment of the present invention, the allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirement of the service load includes:
determining loads of two businesses with highest weighted operation results to be stored in a target storage pool;
taking the two businesses with the highest weighted operation results as one business, carrying out the weighted operation of load characteristics with the other businesses, and storing the loads of the two businesses with the highest weighted operation results in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
In an embodiment of the present invention, the determining the satisfaction of the service performance of the target storage pool to the allocated service includes:
and playing back the storage load characteristics of the allocated service in the target storage pool, and judging the service performance satisfaction degree of the target storage pool to the allocated service.
In an embodiment of the present invention, the adjusting the service allocation relationship based on the determination result of the service performance satisfaction includes:
determining that the target storage pool meets the service performance requirement of the allocated service, and ending the allocation of the service; otherwise, deleting the corresponding business from the target storage pool.
In order to implement the above method embodiment, the embodiment of the present invention further provides an apparatus for distributing a service load, as shown in fig. 4, where the apparatus includes:
a determining module 401, configured to determine affinities between storage load characteristics of a plurality of services;
an allocation module 402 is configured to allocate storage resources for the plurality of services in a target storage pool based on the determined affinity.
In one embodiment, as shown in fig. 5, the apparatus further comprises: a collection module 400;
before the determining module 401 determines affinities between storage load characteristics of a plurality of services, the collecting module 400 is configured to collect real performances of the plurality of services from a source storage system, and collect storage load characteristics of the plurality of services.
In an embodiment of the present invention, the storage load feature includes: capacity demand and load characteristics; the load characteristics include one or more of the following:
data block size distribution, read-write times per second IOPS performance distribution with time, bandwidth performance distribution with time, read-write request proportion and IO access interval distribution.
In one embodiment, as shown in fig. 6, the apparatus further comprises: an adjustment module 403;
after the allocation module 402 allocates storage resources for the plurality of services, the adjustment module 403 is configured to determine a service performance satisfaction degree of the target storage pool for the allocated services; and adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree.
In an embodiment of the present invention, the allocating module 402 allocates storage resources for the plurality of services in the target storage pool based on the determined affinity, including:
gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses;
and allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool and the capacity requirement of the service load.
In an embodiment of the present invention, the allocating module 402 allocates storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirement of the service load, including:
determining loads of two businesses with highest weighted operation results to be stored in a target storage pool;
taking the two businesses with the highest weighted operation results as one business, carrying out the weighted operation of load characteristics with the other businesses, and storing the loads of the two businesses with the highest weighted operation results in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
In an embodiment of the present invention, the determining, by the adjustment module 403, a satisfaction degree of the service performance of the target storage pool on the allocated service includes:
and playing back the storage load characteristics of the allocated service in the target storage pool, and judging the service performance satisfaction degree of the target storage pool to the allocated service.
In an embodiment of the present invention, the adjusting module 403 adjusts the distribution relationship of the service based on the determination result of the satisfaction degree of the service performance, including:
determining that the target storage pool meets the service performance requirement of the allocated service, and ending the allocation of the service; otherwise, deleting the corresponding business from the target storage pool.
The embodiment of the invention also provides a device for distributing the service load, which comprises: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor, when executing the computer program, performs:
determining relatedness between storage load characteristics of a plurality of services;
and allocating storage resources for the plurality of services in a target storage pool based on the determined affinity.
Before said determining the affinity between the stored load characteristics of the plurality of services, said processor is further configured to, when executing said computer program, perform:
the actual performance of the plurality of services is collected from the source storage system while the storage load characteristics of the plurality of services are collected.
Wherein the storage load feature comprises: capacity demand and load characteristics; the load characteristics include one or more of the following:
data block size distribution, read-write times per second IOPS performance distribution with time, bandwidth performance distribution with time, read-write request proportion and IO access interval distribution.
After said allocating storage resources for said plurality of services, said processor is further configured to, when executing said computer program, perform:
determining the service performance satisfaction degree of the target storage pool to the allocated service;
and adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree.
The processor is further configured to, when executing the computer program, perform:
gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses;
and allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool and the capacity requirement of the service load.
The processor is further configured to, when executing the computer program, perform:
determining loads of two businesses with highest weighted operation results to be stored in a target storage pool;
taking the two businesses with the highest weighted operation results as one business, carrying out the weighted operation of load characteristics with the other businesses, and storing the loads of the two businesses with the highest weighted operation results in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
The processor is further configured to, when executing the computer program, perform:
and playing back the storage load characteristics of the allocated service in the target storage pool, and judging the service performance satisfaction degree of the target storage pool to the allocated service.
When the distribution relation of the service is adjusted based on the determined result of the service performance satisfaction degree, the processor is further configured to execute:
determining that the target storage pool meets the service performance requirement of the allocated service, and ending the allocation of the service; otherwise, deleting the corresponding business from the target storage pool.
It should be noted that: in the apparatus provided in the above embodiment, when performing traffic load allocation, only the division of each program module is used as an example, in practical application, the processing allocation may be performed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules, so as to complete all or part of the processing described above. In addition, the apparatus provided in the foregoing embodiments and the corresponding method embodiments belong to the same concept, and a detailed implementation process of the apparatus is shown in the method embodiments, which are not described herein.
In an exemplary embodiment, the present invention also provides a computer-readable storage medium, which may be a Memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash Memory, magnetic surface Memory, optical disk, or CD-ROM; but may be various devices including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.
The embodiment of the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs:
determining relatedness between storage load characteristics of a plurality of services;
and allocating storage resources for the plurality of services in a target storage pool based on the determined affinity.
Before said determining the affinity between the stored load characteristics of the plurality of services, said computer program, when executed by the processor, further performs:
the actual performance of the plurality of services is collected from the source storage system while the storage load characteristics of the plurality of services are collected.
Wherein the storage load feature comprises: capacity demand and load characteristics; the load characteristics include one or more of the following:
data block size distribution, read-write times per second IOPS performance distribution with time, bandwidth performance distribution with time, read-write request proportion and IO access interval distribution.
After said allocating storage resources for said plurality of services, said computer program, when executed by the processor, further performs:
determining the service performance satisfaction degree of the target storage pool to the allocated service;
and adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree.
The computer program, when executed by the processor, further performs:
gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses;
and allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool and the capacity requirement of the service load.
The computer program, when executed by the processor, further performs:
determining loads of two businesses with highest weighted operation results to be stored in a target storage pool;
taking the two businesses with the highest weighted operation results as one business, carrying out the weighted operation of load characteristics with the other businesses, and storing the loads of the two businesses with the highest weighted operation results in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
The computer program, when executed by the processor, further performs:
and playing back the storage load characteristics of the allocated service in the target storage pool, and judging the service performance satisfaction degree of the target storage pool to the allocated service.
When the distribution relation of the service is adjusted based on the determined result of the service performance satisfaction degree, the computer program is executed by the processor and further executes:
determining that the target storage pool meets the service performance requirement of the allocated service, and ending the allocation of the service; otherwise, deleting the corresponding business from the target storage pool.
The invention is described below in connection with scene embodiments.
The embodiment provides a method for distributing service loads in a plurality of storage pools based on service storage load characteristics, fully considers the service storage load characteristics when planning storage resource in advance, adopts real performance and capacity requirements of services, and distributes multiple services for the storage pools based on the capacity of each storage pool. As shown in fig. 7, the method includes:
step 701: determining the real capacity requirement of the service from a source storage system, and collecting the real storage load characteristics of the service; such as capacity requirements C1-CN and load characteristics WL 1-WLN for N services.
Step 702: analyzing the service storage load characteristics to obtain the load characteristics of the service system, and calculating the relativity of the load characteristics among a plurality of services;
the load characteristics of the service may be data block size distribution, IOPS performance distribution over time, bandwidth performance distribution over time, read-write request proportion, IO access interval distribution, and the like. By performing weighted calculation of the load characteristics of the services in pairs step by step to perform affinity analysis (the two services with the highest weighted calculation values are considered to be the highest affinity), it is recommended that a plurality of services with high affinity are stored in the same storage pool. Wherein the calculation of relatedness can be performed on any computer.
Taking the load feature vector of the selected service as { Size1, IOPS1} as an example (Size 1 is the data block Size distribution of the service, and IOPS1 is the IOPS performance distribution of the service over time); the formula R (WLi, WLj) =p×x (Sizei, sizej) + (1-P) ×y (IOPSi, IOPSj) is used to calculate the affinity of the load feature vectors of two different services step by step.
When allocating traffic for storage pool1, the capacity of storage pool1 is F1. Assuming that the affinity of WLm and WLn is the highest, the capacity requirements of WLm and WLn are Cm and Cn respectively; if (F1-Cm-Cn) > 0: taking WLm and WLn as an overall service WLmn, and continuously gradually calculating the relatedness degree of the load feature vectors of different services of the two-to-two services together with other (N-2) services. Supposing that WLmn and WLp relatedness is highest in the second calculation; when the service is continuously allocated for the storage pool1, the residual capacity of the storage pool1 is (F1-Cm-Cn), and the capacity requirement of WLp is Cp; if (F1-Cm-Cn-Cp) > 0: taking WLmn and WLp as an overall service WLmnp, and continuously calculating the relatedness degree of the load characteristic vectors of different services of every two services gradually together with other (N-3) services. If calculated to (F1-Cm-Cn-Cp- … -Cx-Cy-Cz) < 0, WLm, WLn, WLp, …, WLx, WLy are assigned to storage pool 1.
Then, the service is allocated for the storage pool2, and the unallocated remaining services { WL 1-WLN } - { WLm, WLn, WLp, …, WLx, WLy } are selected by the above calculation.
Step 703: based on the rough distribution result of the last step, the storage load characteristics of the services are directly played (played back) repeatedly in the target storage pool, the satisfaction degree of the target storage pool on the service performance is further evaluated, and the service distribution relation is further adjusted according to the evaluation result.
Directly repeatedly playing the business storage loads of WLm, WLn, WLp, …, WLx and WLy in the target storage pool 1; if the target storage pool1 can simultaneously meet the requirements of WLm, WLn, WLp, …, WLx and WLy, ending the allocation; if the target storage pool1 can not meet the requirements of the services at the same time, the service storage loads of WLm, WLn, WLp, … and WLx (i.e. compared with the last repeated play, the service WLy is removed) are directly repeated play in the target storage pool 1; if the target storage pool1 can simultaneously meet the requirements of WLm, WLn, WLp, … and WLx, the allocation is ended.
Example 1
Step one: the capacity requirements of 4 services WL 1-WL 4 collected from several source storage systems are 20TB respectively; and collecting service storage load characteristics of the services.
Step two: analyzing the service storage load characteristics of the 4 services; and obtaining the affinity of the load characteristics of the 4 service systems.
The load feature vector of the service is { Size1, IOPS1}; where Size1 is the data block Size distribution of the service, and IOPS1 is the IOPS performance distribution of the service over time.
Gradually calculating affinity of load characteristics between every two businesses, wherein the formula is as follows: r (WLi, WLj) =p×x (Sizei, sizej) +1-P) Y (IOPSi, IOPSj). The weight P may be 1/3.
Assuming that traffic data block sizes are { (0,8KB ], (8 kb,64kb ], (64 kb,1MB ], (1 MB, -) }, X (Sizei, sizej) =Σ (size1×sizej1+size2+size3×sizej3+size4×sizej 4),. The Size1 of traffic WL1 is {1,0,0,0}, the Size2 of traffic WL2 is {1,0,0,0}, the Size3 of traffic WL3 is {0,0,0.5,0.5}, the Size4 of traffic WL4 is {1,0,0,0}, X (Size 1, size 2) =1,X (Size 1, size 3) 8652zxft 8652 (Size 1, size 4) =3265 zft (Size 3) =352) =3535 (Size 3) =352).
Assuming that the time distribution interval of the performance IOPS of the service is { (00:00, 04:00], (04:00, 08:00], (08:00, 12:00], (12:00, 16:00], (16:00, 20:00], (20:00, 00:00] }, Z (IOPSj) =1/mean square error (((iops1+iopsj 1)/2, (iops2+iops2)/2, (iops3+iops3)/2, (iops4+iopsj 4)/2, (iops5+iops5)/2, (iops6+iops6)/2) if the mean square error is not 0, and the value is 1 if the formula value exceeds 1, the value of Z is 1). The service is 1, iops1= (45 zxft 3245), ps2= (3732) = (3+iops4)/2, (iops5+iops5)/2, (iops6+iops6)/2) if the mean square error is not 0, and (3×39 ps 35, 34) =3×35, 33, 35, 33) =3×3×3×18, 35, 35×18×3×2.
Therefore, the affinity between every two businesses is calculated as follows: r (WL 1, WL 2) =1/3+2/3=1, R (WL 1, WL 3) =0+2/3×0.09=0.06, R (WL 1, WL 4) =1/3+2/3×0.18=0.45, R (WL 2, WL 3) =0+2/3×1=0.67, R (WL 2, WL 4) =0+2/3×0.18=0.12, R (WL 3, WL 4) =0+2/3×0.18=0.12.
Step three: for simplicity of description, it is assumed that all storage pools have substantially identical hardware performance, only their capacity capabilities are considered. The storage pool is assigned traffic based on affinity of traffic storage load characteristics.
Assuming that the capacities of the storage POOLs POOL1 and POOL2 are 60TB, and the performance IOPS capacities are 15; after WL1 is assigned to POOL1, WL2 is also assigned to POOL1 due to the higher affinity of WL1 to WL 2. POOL1 remains 20TB after storing WL1, WL2, available for storing one of the loads WL3 or WL4. By using WL1 and WL2 as services WL12 and calculating the affinity with other services { WL3 and WL4}, and repeating the above calculation process, it can be found that the affinity between WL12 and WL4 is higher, and WL4 can be stored on POOL1 at this time. At this time (60T-20 TB-20T) =0; traffic WL1, WL2, WL4 is allocated on POOL 1.
Step four: after the storage POOL is roughly allocated with a plurality of services WL1, WL2 and WL4, the service storage load characteristics are directly played back in the target storage POOL POOL1, and the satisfaction degree of the target storage POOL on the service performance is further evaluated.
If the target POOL1 can simultaneously meet the demands of the traffic WL1, WL2, WL4 at this time, the allocation ends (traffic WL1, WL2, WL4 is allocated on POOL 1). If the target storage POOL1 cannot meet the requirements of the services at the same time, the service storage loads WL1, WL2 are further replayed in the target storage POOL, the target POOL is found to be satisfied with the traffic performance, then the allocation ends (traffic WL1, WL2 is allocated on POOL 1).
Step five: the select storage pool2 resumes computing unallocated remaining traffic WL3. As can be seen from the analysis of the above graph, by adopting the method for distributing the service load among the plurality of storage POOLs, the resources of the storage POOL POOL1 can be fully utilized, and the utilization efficiency of the resources of the POOL1 can be improved; the POOL2 will be free of more resources for the allocation of the subsequent traffic load.
In this embodiment, according to affinities among the load characteristics of the multiple service storage, storage resources are allocated to multiple services in the target storage pool, in addition, satisfaction of the target storage pool to service performance is also estimated, and the service allocation relation is further adjusted according to the estimation result. The embodiment fully considers the service storage load characteristics, adopts the real performance and capacity requirements of the service, considers the mapping relation between the service and the storage pool when planning the storage resource in advance, ensures that the service has enough resource usage, adjusts the different resource requirements of a plurality of services in the same storage pool, and improves the resource utilization rate.
The foregoing description is only of the preferred embodiments of the invention and is not intended to limit the scope of the invention.
Claims (9)
1. A method of distributing traffic load, the method comprising:
determining relatedness between storage load characteristics of a plurality of services;
allocating storage resources for the plurality of services in a target storage pool based on the determined affinity; wherein:
the determining affinity between storage load characteristics of a plurality of services includes:
gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses to obtain a weighting operation result;
the allocating storage resources for the plurality of services in a target storage pool based on the determined affinity comprises:
allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirements of the traffic load;
the allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirement of the traffic load comprises:
determining to store the loads of the two businesses with the highest affinity in a target storage pool; taking the two businesses with the highest affinity as one business, carrying out weighted calculation of load characteristics with the other businesses to obtain affinity analysis, and storing the loads of the two businesses with the highest affinity in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
2. The method of claim 1, wherein prior to determining affinity between storage load characteristics of the plurality of services, the method further comprises:
the actual performance of the plurality of services is collected from the source storage system while the storage load characteristics of the plurality of services are collected.
3. The method of claim 1 or 2, wherein the storing load characteristics comprises: capacity demand and load characteristics; the load characteristics include one or more of the following:
data block size distribution, read-write times per second IOPS performance distribution with time, bandwidth performance distribution with time, read-write request proportion and IO access interval distribution.
4. The method according to claim 1 or 2, wherein after said allocating storage resources for said plurality of services, the method further comprises:
determining the service performance satisfaction degree of the target storage pool to the allocated service;
and adjusting the distribution relation of the service based on the determined result of the service performance satisfaction degree.
5. The method of claim 4, wherein said determining the service performance satisfaction of the target storage pool for the allocated service comprises:
and playing back the storage load characteristics of the allocated service in the target storage pool, and judging the service performance satisfaction degree of the target storage pool to the allocated service.
6. The method of claim 4, wherein adjusting the distribution relationship of the service based on the determination of the satisfaction of the service performance comprises:
determining that the target storage pool meets the service performance requirement of the allocated service, and ending the allocation of the service; otherwise, deleting the corresponding business from the target storage pool.
7. An apparatus for distributing traffic load, the apparatus comprising:
a determining module for determining relatedness between storage load characteristics of a plurality of services;
an allocation module for allocating storage resources for the plurality of services in a target storage pool based on the determined affinity; wherein:
the determining module is specifically configured to: gradually carrying out weighting operation on the load characteristics of every two businesses in the plurality of businesses to obtain a weighting operation result;
the distribution module is specifically configured to: allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirements of the traffic load;
the allocating storage resources for the plurality of services based on the result of the weighting operation, the capacity of the target storage pool, and the capacity requirement of the traffic load comprises:
determining to store the loads of the two businesses with the highest affinity in a target storage pool; taking the two businesses with the highest affinity as one business, carrying out weighted calculation of load characteristics with the other businesses to obtain affinity analysis, and storing the loads of the two businesses with the highest affinity in the target storage pool; and so on until the capacity requirements of the traffic load stored in the target storage pool is no greater than the capacity of the target storage pool.
8. An apparatus for distributing traffic load, the apparatus comprising: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is adapted to perform the steps of the method of any of claims 1-6 when the computer program is run.
9. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910000435.9A CN111400019B (en) | 2019-01-02 | 2019-01-02 | Method, device and computer readable storage medium for distributing business load |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910000435.9A CN111400019B (en) | 2019-01-02 | 2019-01-02 | Method, device and computer readable storage medium for distributing business load |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111400019A CN111400019A (en) | 2020-07-10 |
| CN111400019B true CN111400019B (en) | 2023-10-27 |
Family
ID=71433988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910000435.9A Active CN111400019B (en) | 2019-01-02 | 2019-01-02 | Method, device and computer readable storage medium for distributing business load |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111400019B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103685054A (en) * | 2013-12-18 | 2014-03-26 | 武汉烽火网络有限责任公司 | Multipath load balancing method based on service awareness |
| CN106203639A (en) * | 2016-06-30 | 2016-12-07 | 乐视控股（北京）有限公司 | The distribution method of business operation and device |
| CN106341893A (en) * | 2016-10-08 | 2017-01-18 | 北京邮电大学 | Resource distribution method and apparatus |
| CN106657226A (en) * | 2016-09-27 | 2017-05-10 | 国网辽宁省电力有限公司沈阳供电公司 | Communication system resource cooperative management method based on performance orientation |
| CN107392512A (en) * | 2016-11-25 | 2017-11-24 | 北京小度信息科技有限公司 | Grouped task method and apparatus |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9753767B2 (en) * | 2014-03-11 | 2017-09-05 | Sas Institute Inc. | Distributed data set task selection |
-
2019
- 2019-01-02 CN CN201910000435.9A patent/CN111400019B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103685054A (en) * | 2013-12-18 | 2014-03-26 | 武汉烽火网络有限责任公司 | Multipath load balancing method based on service awareness |
| CN106203639A (en) * | 2016-06-30 | 2016-12-07 | 乐视控股（北京）有限公司 | The distribution method of business operation and device |
| CN106657226A (en) * | 2016-09-27 | 2017-05-10 | 国网辽宁省电力有限公司沈阳供电公司 | Communication system resource cooperative management method based on performance orientation |
| CN106341893A (en) * | 2016-10-08 | 2017-01-18 | 北京邮电大学 | Resource distribution method and apparatus |
| CN107392512A (en) * | 2016-11-25 | 2017-11-24 | 北京小度信息科技有限公司 | Grouped task method and apparatus |
Non-Patent Citations (2)
| Title |
|---|
| Jatin Chhugani.Billion-particle SIMD-friendly two-point correlation on large-scale HPC cluster systems.《 SC '12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis》.2013,1-11. * |
| 刘柏林 ; 刘胜 ; 包北方 ; .基于复杂网络理论的多项目实施资源优化配置研究.机械.2012,(第08期),43-48. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111400019A (en) | 2020-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10175909B2 (en) | Extent migration in multi-tier storage systems | |
| US11093304B2 (en) | Using response time objectives in a storage system | |
| US10339455B1 (en) | Techniques for determining workload skew | |
| US9665630B1 (en) | Techniques for providing storage hints for use in connection with data movement optimizations | |
| US8949483B1 (en) | Techniques using I/O classifications in connection with determining data movements | |
| US8799916B2 (en) | Determining an allocation of resources for a job | |
| CN106469018B (en) | Load monitoring method and device for distributed storage system | |
| US20080172672A1 (en) | Recommending Moving Resources in a Partitioned Computer | |
| US9612746B1 (en) | Allocation method for meeting system performance and application service level objective (SLO) | |
| CN106407207B (en) | Real-time newly-added data updating method and device | |
| US20120221730A1 (en) | Resource control system and resource control method | |
| US11704022B2 (en) | Operational metric computation for workload type | |
| US10474383B1 (en) | Using overload correlations between units of managed storage objects to apply performance controls in a data storage system | |
| CN110245091B (en) | Memory management method and device and computer storage medium | |
| US10310884B2 (en) | Virtual machine placement in a heterogeneous data center | |
| CN112988066B (en) | Data processing method and device | |
| CN110636388A (en) | Service request distribution method, system, electronic equipment and storage medium | |
| US20200236169A1 (en) | Cloud platform or cloud provider selection | |
| CN112948279A (en) | Method, apparatus and program product for managing access requests in a storage system | |
| CN115421924A (en) | Memory allocation method, device and equipment | |
| US10019195B1 (en) | Storage array health score | |
| CN111400019B (en) | Method, device and computer readable storage medium for distributing business load | |
| US10268714B2 (en) | Data processing in distributed computing | |
| TW202215248A (en) | Method of operating storage system, and method of partitioning tier of storage resources | |
| CN116382569A (en) | Data processing method, device, hard disk and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |