CN116880752A - Method and system for improving utilization efficiency of storage system resources - Google Patents
Method and system for improving utilization efficiency of storage system resources Download PDFInfo
- Publication number
- CN116880752A CN116880752A CN202310840847.XA CN202310840847A CN116880752A CN 116880752 A CN116880752 A CN 116880752A CN 202310840847 A CN202310840847 A CN 202310840847A CN 116880752 A CN116880752 A CN 116880752A
- Authority
- CN
- China
- Prior art keywords
- data channel
- lock
- storage system
- lockless
- free data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 108091006146 Channels Proteins 0.000 claims description 172
- 238000012360 testing method Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000013500 data storage Methods 0.000 abstract description 7
- 238000007726 management method Methods 0.000 description 22
- 238000012545 processing Methods 0.000 description 12
- 239000003795 chemical substances by application Substances 0.000 description 10
- 230000003247 decreasing effect Effects 0.000 description 8
- 230000015654 memory Effects 0.000 description 8
- 238000009434 installation Methods 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0658—Controller construction arrangements
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention belongs to the technical field of data storage, and particularly relates to a method and a system for improving the utilization efficiency of storage system resources. By calculating how much system resources are needed to occupy in order to reach the specified performance index and deploying the lock-free data channel based on how much system resources are needed to occupy, the occupation of the storage system to the system resources is optimized, the resource utilization efficiency of the storage system is improved, namely, data storage service with better performance and larger capacity is provided to the outside with fewer system resources occupation.
Description
Technical Field
The invention belongs to the technical field of data storage, and particularly relates to a method and a system for improving the utilization efficiency of storage system resources.
Background
The storage system can be divided into two parts, namely system hardware and storage system software, wherein the storage system software needs to manage the system hardware, reasonably schedule, allocate and utilize system hardware resources, respond to read-write access requests of clients of the storage system, and efficiently process storage IO, and in the process, the storage system software needs to consume key system resources including CPU, memory, network bandwidth, disk medium bandwidth and the like. The utilization efficiency of the storage system software to the key system resources determines the overall performance of the storage system (the overall performance is usually measured by indexes such as IOPS (input/output), time delay, throughput & bandwidth, storage space utilization rate and the like), determines the investment of the storage system on the system hardware, and further determines the overall cost of the storage system.
Conventional storage system software generally runs on an operating system of a computer, relies on the operating system to manage and schedule system hardware resources, and relies on external management mechanisms such as a CPU scheduling mechanism, a multithreading scheduling mechanism, a memory management mechanism, an interrupt management mechanism, an operating system kernel IO processing mechanism, a network card, a disk and the like of the operating system to complete IO processing, network transmission, data storage and the like. In this process, the system resources required to be consumed by the storage system depend on various resource scheduling mechanisms of the general-purpose computer operating system, accurate estimation and measurement are difficult to achieve, and intelligent allocation and flexible scheduling of the system resources are more difficult to achieve, especially when the storage system software needs to share hardware system resources with other system software and application software (such as a typical super-fusion system), the general-purpose computer operating system needs to uniformly coordinate and manage and schedule the system resources in a more universal and fair manner, and a relationship capable of quantifying measurement is more difficult to be established between the system resource consumption of the storage system and various performance indexes provided by the storage system. Therefore, the traditional storage system software adopts a software and hardware integrated closed mode, the full use of system hardware resources for the storage system software is ensured, the storage system software adopting a software definition mode adopts pre-allocation sufficient system resources to ensure the resource consumption of the storage system, and the two modes lack the capability of dynamic planning and intelligent scheduling of the system resources, so that the overall resource utilization efficiency of the storage system is low.
Disclosure of Invention
The invention aims to provide a method and a system for improving the resource utilization efficiency of a storage system, which are used for solving the problem that the resource utilization rate of the existing storage system is not high.
In order to solve the technical problems, the invention provides a method for improving the utilization efficiency of storage system resources, which comprises the following steps:
1) According to the preset storage system performance requirement and the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information, computing the deployment information of the lock-free data channel required to be deployed when the preset storage system performance requirement is met; the deployment information includes specification and quantity;
2) And deploying the specifications and the number of the lock-free data channels according to the deployment information of the lock-free data channels to be deployed, and binding corresponding system resources for each lock-free data channel.
The beneficial effects are as follows: because each lock-free data channel binds the appointed system resource, the invention has no problems of resource contention, resource rotation waiting, context switching, mutual influence and the like in the lock-free data channels, can accurately estimate how much system resource is needed to occupy in order to reach the appointed performance index (IOPS, time delay, bandwidth and the like), deploys the lock-free data channels based on how much system resource is needed to occupy, can more accurately and intelligently optimize the occupation of the system resource by a storage system, and can provide data storage service with better performance and larger capacity with less occupation of the system resource (refer to the consumption of the storage system to CPU, memory, network bandwidth, disk medium and other key system resources).
Further, in step 2), when the storage system corresponding to the lock-free data channel binding the corresponding system resource is applied, according to the dynamic performance requirement feedback of the corresponding storage system user and the actual performance of the storage system, the preset storage system performance requirement is corrected in real time, and according to the corrected storage system performance requirement, the lock-free data channel binding the corresponding system resource is dynamically adjusted, so that the actual performance index of the dynamically adjusted lock-free data channel is closer to the preset storage system performance requirement than the actual performance index of the lock-free data channel when the dynamic adjustment is not performed.
When the storage system is applied, the preset storage system performance requirement is corrected in real time through dynamic performance requirement feedback of an user and actual performance of the storage system, so that the lock-free data channel of the storage system is dynamically adjusted based on the data acquired in real time, the actual performance index of the adjusted lock-free data channel is closer to the preset storage system performance requirement, occupation of the storage system on system resources is further optimized, and resource utilization efficiency of the storage system is improved with less system resource occupation.
Further, the dynamic adjustment includes reducing an airless data channel adjustment and increasing an airless data channel adjustment, the reducing an airless data channel adjustment including: determining a data channel needing to be reduced, not carrying out new task allocation on the data channel needing to be reduced, and releasing the data channel needing to be reduced after waiting for the completion of the task carried out in the data channel needing to be reduced; the adding lock-free data channel adjustment includes: and determining the number of the data channels needing to be increased, increasing the deployment of the corresponding number of the data channels without lock, and binding corresponding system resources for the increased data channels without lock.
Further, by calculating the difference between the current system performance requirement and the actual performance of the system, the number of the lockless data channels which need to be increased or decreased on each node of the system is calculated, if the calculated result is the number of the lockless data channels, the lockless data channels are increased, and if the calculated result is the number of the lockless data channels, the lockless data channels are decreased.
Further, in step 1), the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information is obtained by testing the hardware configured by the storage system.
In order to solve the technical problems, the invention also provides a system for improving the utilization efficiency of storage system resources, which comprises a storage management unit and a management Agent unit, wherein the storage management unit comprises a system controller, and the system controller is used for calculating the deployment information of the lock-free data channel required to be deployed when meeting the preset storage system performance requirement according to the preset storage system performance requirement and the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information, and sending deployment instruction information comprising the deployment information to the management Agent unit; the deployment information includes specification and quantity;
the management Agent unit is used for deploying the specification and the number of the lockless data channels according to the deployment information of the lockless data channels to be deployed when receiving the deployment instruction information, and binding corresponding system resources for each lockless data channel.
The beneficial effects are as follows: because each lock-free data channel binds the appointed system resource, the invention has no problems of resource contention, resource rotation waiting, context switching, mutual influence and the like in the lock-free data channels, can accurately estimate how much system resource is needed to occupy in order to reach the appointed performance index (IOPS, time delay, bandwidth and the like), deploys the lock-free data channels based on how much system resource is needed to occupy, can more accurately and intelligently optimize the occupation of the system resource by a storage system, and can provide data storage service with better performance and larger capacity with less occupation of the system resource (refer to the consumption of the storage system to CPU, memory, network bandwidth, disk medium and other key system resources).
Further, the system controller is further configured to correct the preset performance requirement of the storage system in real time according to the dynamic performance requirement feedback of the corresponding storage system user and the actual performance of the storage system when the storage system corresponding to the lock-free data channel binding the corresponding system resource is applied, and dynamically adjust the lock-free data channel binding the corresponding system resource according to the corrected performance requirement of the storage system, so that the actual performance index of the dynamically adjusted lock-free data channel is closer to the preset performance requirement of the storage system than the actual performance index of the lock-free data channel when the dynamic adjustment is not performed.
When the storage system is applied, the preset storage system performance requirement is corrected in real time through dynamic performance requirement feedback of an user and actual performance of the storage system, so that the lock-free data channel of the storage system is dynamically adjusted based on the data acquired in real time, the actual performance index of the adjusted lock-free data channel is closer to the preset storage system performance requirement, occupation of the storage system on system resources is further optimized, and resource utilization efficiency of the storage system is improved with less system resource occupation.
Further, the dynamic adjustment includes reducing an airless data channel adjustment and increasing an airless data channel adjustment, the reducing an airless data channel adjustment including: determining a data channel needing to be reduced, not carrying out new task allocation on the data channel needing to be reduced, and releasing the data channel needing to be reduced after waiting for the completion of the task carried out in the data channel needing to be reduced; the adding lock-free data channel adjustment includes: and determining the number of the data channels needing to be increased, increasing the deployment of the corresponding number of the data channels without lock, and binding corresponding system resources for the increased data channels without lock.
Further, by calculating the difference between the current system performance requirement and the actual performance of the system, the number of the lockless data channels which need to be increased or decreased on each node of the system is calculated, if the calculated result is the number of the lockless data channels, the lockless data channels are increased, and if the calculated result is the number of the lockless data channels, the lockless data channels are decreased.
Further, the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information is obtained through testing the hardware configured by the storage system.
Drawings
FIG. 1 is a system block diagram of the present invention for improving the efficiency of utilization of storage system resources;
FIG. 2 is a flow chart of an initial installation deployment method in the method for improving the utilization efficiency of the resources of the storage system;
FIG. 3 is a flow chart of a method for performance continuous feedback and triggering dynamic control adjustments in a method for improving the utilization efficiency of storage system resources according to the present invention;
FIG. 4 is a flow chart of a method for dynamically controlling and adjusting system resource consumption in a method for improving the utilization efficiency of storage system resources according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent.
System embodiments to improve storage system resource utilization efficiency:
in order to achieve more accurate and intelligent control of occupation of the storage system to system resources, the embodiment is used for controlling occupation of the system resources by the storage system. Specifically, the system resource consumption of the storage system software can be continuously and dynamically optimized according to the external rated performance index of the storage system or the dynamic performance requirement of a service system accessing the storage system, such as CPU, memory, network bandwidth and other system resources which are required to be consumed by the storage system software, and the feedback system based on the performance requirement of the storage system is also provided, so that the data storage service with better performance and more stability is provided for the outside with less and more optimized system resource occupation (the consumption of the storage system on the CPU, memory, network bandwidth, disk media and other key system resources).
Fig. 1 is a schematic diagram of a system structure of the present embodiment, where the system structure includes a storage management unit (i.e. the storage management platform in fig. 1) and a management Agent unit, and the storage management unit includes a system controller, a system performance monitoring unit, a lock-free data channel performance baseline testing unit, and other management operation and maintenance components, and specifically functions of each component are as follows: the storage management platform provides a unified storage management function for the whole storage system or the storage cluster; the management agent runs on each storage node, responds to the instruction of the storage management platform, and executes corresponding operation on the storage node; executing a new storage IO model based on the lock-free data channel by the lock-free data channel, and realizing end-to-end processing for the storage IO after segmentation; the lock-free data channel performance baseline test is used for testing the performance index (defined by RTC_1 (P1, P2, …, pn)) which can be achieved by a certain configuration of lock-free data channels (defined by RTC_1 (N1, N2, …, N); system performance monitoring, namely continuously monitoring performance indexes (defined by systems (P1, P2, …, pn)) of a System on the current actual storage System environment; the system controller is a core control component of the system and comprises the following core functions:
1. an administrator of the storage System may define System performance index requirements (defined in systems (P1, P2, …, pn)) on the System controller.
2. And driving a performance baseline test of the lock-free data channel to acquire the performance which can be provided by the lock-free data channels with different specifications.
3. Providing an API interface allows the system performance monitoring component to continuously report the performance that the system is currently actually providing.
4. Providing an API interface allows the external system to continuously feedback defining its currently required performance.
5. And (5) completing the initialized installation and deployment of the lock-free data channels on each node.
6. Based on the performance actually provided by the current system and the performance required by the external system, the number of IO fragments entering each lock-free data channel is dynamically controlled, and the number and the specification of the lock-free data channels on each node are dynamically adjusted.
A plurality of lock-free data channels are established in the system as required, the storage IO is segmented into different lock-free data channels according to the ID, and end-to-end data processing is realized in the data channels until the data is persisted on a storage medium and information is confirmed for a client. In a lock-free data channel, a polling mode is adopted to respond and enter the IO (input/output) slice of the channel for processing, a single-thread model is used in the whole processing process, no process/thread context switching, no interrupt processing, no lock waiting and no CPU (Central processing Unit) time slice rotation are adopted, and therefore extremely high IO processing speed can be achieved.
Each lock-free data channel is bound with a plurality of memories of 1 or more CPU cores and is responsible for processing the read-write times per second (IOPS) and the bandwidth of a specified number of disk media or is responsible for processing a network of the specified bandwidth, wherein the network can span multiple hardware machine networks, including but not limited to a TCP/IP network or an RDMA network or an InfiniBand network, and can also be an interconnection network for interconnecting a plurality of nodes in the same machine frame, including but not limited to various internal interconnection buses, SAS buses and the like. Because each lock-free data channel binds the appointed system resource, the lock-free data channels have no problems of resource contention, resource rotation waiting, context switching, mutual influence and the like, and can accurately estimate how much system resource is needed to occupy in order to reach the appointed performance index (IOPS, time delay, bandwidth and the like).
Each lock-free data channel can reach (P1, P2, …, pn) performance indexes (e.g., p1=iops, p2=single-pass delay, p3=delay to reach P1-specified IOPS, …) after occupying (N1, N2, …, nn) system resources (e.g., n1=cpu core number, n2=cpu frequency and level, n3=memory size, …), and such a correspondence is called lock-free data channel performance baseline, and a set of lock-free data channel performance baselines can be obtained through laboratory testing based on some common general hardware combinations, as a benchmark when formally deploying and using storage system software.
The specific method for improving the utilization efficiency of the storage system resources based on the system of the embodiment comprises an initial installation and deployment method flow, a performance continuous feedback method flow, a dynamic control adjustment triggering method flow and a dynamic control adjustment system resource consumption method flow.
As shown in fig. 2, the flow of the initial installation and deployment method in this embodiment is as follows:
1) Defining storage system performance requirements in a system controller: system (P1, P2, …, pn);
2) Judging whether the hardware of the current system configuration has laboratory baseline data or not, namely judging whether the hardware of the current system configuration has a performance baseline of the lock-free data channel or not, if not, executing the performance baseline test of the lock-free data channel on the current hardware, recording the performance of the lock-free data channel executed on the current hardware, and taking the performance baseline data as the laboratory baseline data of the next installation and deployment, wherein the performance baselines of the lock-free data channels of different configurations are obtained at the moment (n in the performance baselines of the following formula is the same as n in Pn, n in RTC_n is not necessarily the same as n in Pn): rtc_1 (N1, N2, …, nn) =rtc_1 (P1, P2, …, pn), rtc_2 (N1, N2, …, nn) =rtc_2 (P1, P2, …, pn), …, rtc_n (N1, N2, …, nn) =rtc_n (P1, P2, …, pn); if the hardware of the current system configuration has the performance base line of the lock-free data channel, the performance base line of the lock-free data channel of different configurations is directly obtained;
3) The System controller calculates the specification and the number of the lockless data channels to be deployed on different nodes of the System according to the performance requirement System (P1, P2, …, pn) of the System and performance baselines rtc_1 (N1, N2, …, nn) =rtc_1 (P1, P2, …, pn), rtc_2 (N1, N2, …, nn) =rtc_2 (P1, P2, …, pn), …, rtc_n (N1, N2, …, nn) =rtc_n (P1, P2, …, pn) of the lockless data channels of different specifications;
4) The system controller instructs the management agent on each node to execute the actual installation and deployment of the lock-free data channel through the management agent, and binds the corresponding system resources (in this embodiment, by using means of binding cores, resource isolation, and the like) to the lock-free data channel.
As shown in fig. 3, the method flow of the performance persistent feedback and triggering the dynamic control adjustment in this embodiment is as follows:
1) The storage system completes initial installation and deployment and starts to operate;
2) The service management system (user of the storage system) feeds back the dynamic performance requirements in real time through the API provided by the system controller: system_dynamic (P1, P2, …, pn);
3) The system controller monitors the actual performance pressure of the system in real time by managing the Agent, so as to correct the system performance requirement in real time according to the dynamic performance requirement feedback of the service system obtained in the step 2) and the actual performance pressure monitored by the system: system (P1, P2, …, pn);
4) Judging whether the difference between the current system performance requirement and the reference performance which can be provided by the system is greater than a certain threshold (the certain threshold is a set value) or not according to the reference performance which can be provided by the lock-free data channel which is currently deployed by the system and the real-time correction system performance requirement obtained in the step 3);
5) If the actual resource consumption of the system is greater than a certain threshold value, the system controller dynamically controls and adjusts the actual resource consumption of the system; if not, returning to the step 3).
As shown in fig. 4, the method flow of dynamically controlling and adjusting the system resource consumption in this embodiment is as follows:
1) Acquiring the difference d between the current system performance requirement and the reference performance which can be provided by the system; i.e. difference d = current system performance requirement-reference performance that the system can provide;
2) The system controller calculates the specification or the number of the lock-free data channels which need to be increased or decreased on each node of the system based on the difference d obtained in the step 1);
in this embodiment, whether the lock-free data channel needs to be increased or decreased is determined according to the positive and negative of the difference d, and then a specific specification or number of the lock-free data channel is determined to be adjusted based on a specific value of the difference d (i.e. a specific specification or number of the increase or decrease is determined). Specifically, if the difference d is a positive number, the number or specification of the lock-free data channels needs to be increased; if the difference d is negative, it indicates that the number or specification of lock-free data channels needs to be reduced. And according to the difference d, the number u of the data channels needing to be increased or decreased without lock can be calculated, and the calculation formula is as follows: number u = round up (absolute value (d)/reference performance of the lock-free data channel). When the value of the difference d is smaller, the lowest threshold (i.e. when the absolute value of the difference d is smaller than the lowest threshold) which needs to be adjusted by increasing or decreasing the number of the lockless data channels is not reached, the lockless data channels with different specifications can be replaced for adjustment, and the calculation formula is as follows: absolute value (difference d) < absolute value (reference performance of new lock-free data channel-reference performance of old lock-free data channel).
3) If the calculated result is that the lock-free data channel needs to be reduced, adjusting a slicing (slicing) rule of the stored IO, and not bearing a new IO on the lock-free data channel to be released, waiting for the completion of the existing IO processing on the lock-free data channel to be released, and sending an instruction to a management Agent on each node by the system controller to complete the release of the lock-free data channel so as to release system resources;
if the calculated result is that the lock-free data channel needs to be added, the system controller sends an instruction to the management Agent on each node, adds and deploys a new lock-free data channel, binds corresponding system resources to the lock-free data channel, adjusts the slicing (slicing) rule of the storage IO, enables the storage IO to be distributed and flows into the newly added lock-free data channel, and achieves improvement of system performance.
The invention has the following effects on optimizing the resource consumption of the storage system: while optimizing resource consumption, providing a smooth performance experience; the system resource consumption saved by the optimization can be better supported for super fusion deployment; on the premise of meeting the service requirement, the hardware specification is reduced, and the hardware investment is reduced; under the software definition, the software and hardware decoupling deployment mode can better adapt and support different types of hardware with different specifications; the cloud primary environment with dynamic elasticity is better adapted; more calculation work can be done on the storage system, various calculation force unloading and operator sinking capabilities are realized.
The embodiment of the method for improving the utilization efficiency of the storage system resources comprises the following steps:
according to the method, according to the preset storage system performance requirement and the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information, the deployment information of the lock-free data channel required to be deployed when the preset storage system performance requirement is met is calculated; and deploying the specifications and the number of the lock-free data channels according to the deployment information of the lock-free data channels to be deployed, and binding corresponding system resources for each lock-free data channel. The specific method steps for improving the utilization efficiency of the storage system resources are described in detail in the system embodiment for improving the utilization efficiency of the storage system resources, which are not described herein.
The above description is only a preferred embodiment of the present invention, and the patent protection scope of the present invention is defined by the claims, and all equivalent structural changes made by the specification and the drawings of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for improving the utilization efficiency of a storage system resource, comprising the steps of:
1) According to the preset storage system performance requirement and the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information, computing the deployment information of the lock-free data channel required to be deployed when the preset storage system performance requirement is met; the deployment information includes specification and quantity;
2) And deploying the specifications and the number of the lock-free data channels according to the deployment information of the lock-free data channels to be deployed, and binding corresponding system resources for each lock-free data channel.
2. The method for improving the utilization efficiency of storage system resources according to claim 1, wherein in step 2), when the storage system corresponding to the lockless data channel binding the corresponding system resources is applied, according to the dynamic performance requirement feedback of the corresponding storage system user and the actual performance of the storage system, the preset storage system performance requirement is corrected in real time, and according to the corrected storage system performance requirement, the lockless data channel binding the corresponding system resources of each lockless data channel is dynamically adjusted, so that the actual performance index of the lockless data channel after the dynamic adjustment is closer to the preset storage system performance requirement than the actual performance index of the lockless data channel when the lockless data channel is not dynamically adjusted.
3. The method of claim 2, wherein the dynamic adjustment includes reducing an airless data channel adjustment and increasing an airless data channel adjustment, the reducing an airless data channel adjustment including: determining a data channel needing to be reduced, not carrying out new task allocation on the data channel needing to be reduced, and releasing the data channel needing to be reduced after waiting for the completion of the task carried out in the data channel needing to be reduced; the adding lock-free data channel adjustment includes: and determining the number of the data channels needing to be increased, increasing the deployment of the corresponding number of the data channels without lock, and binding corresponding system resources for the increased data channels without lock.
4. A method for improving the utilization efficiency of a storage system resource according to claim 3, wherein the number of the lockless data channels to be added or subtracted on each node of the system is calculated by calculating the difference between the current system performance requirement and the actual performance of the system, if the calculation result is the number of the lockless data channels to be added, the adding of the lockless data channels is performed, and if the calculation result is the number of the lockless data channels to be subtracted, the subtracting of the lockless data channels is performed.
5. The method for improving the utilization efficiency of storage system resources according to claim 1, wherein in step 1), the correspondence between the deployment information of the lock-free data channel and the performance index reached by the deployment information is obtained by testing hardware configured by the storage system.
6. The system for improving the utilization efficiency of the storage system resources is characterized by comprising a storage management unit and a management Agent unit, wherein the storage management unit comprises a system controller, the system controller is used for calculating the deployment information of the lock-free data channel required to be deployed when meeting the preset storage system performance requirement according to the preset storage system performance requirement and the corresponding relation between the deployment information of the lock-free data channel and the performance index reached by the deployment information, and sending deployment instruction information comprising the deployment information to the management Agent unit; the deployment information includes specification and quantity;
the management Agent unit is used for deploying the specification and the number of the lockless data channels according to the deployment information of the lockless data channels to be deployed when receiving the deployment instruction information, and binding corresponding system resources for each lockless data channel.
7. The system according to claim 6, wherein the system controller is further configured to correct the preset storage system performance requirement in real time according to the dynamic performance requirement feedback of the corresponding storage system user and the actual performance of the storage system when the storage system corresponding to the lock-free data channel binding the corresponding system resource is applied, and dynamically adjust the lock-free data channel binding the corresponding system resource according to the corrected storage system performance requirement, so that the actual performance index of the dynamically adjusted lock-free data channel is closer to the preset storage system performance requirement than the actual performance index of the lock-free data channel when the lock-free data channel is not dynamically adjusted.
8. The system for improving utilization of storage system resources of claim 7, wherein the dynamic adjustment comprises a decrease in lock-free data channel adjustment and an increase in lock-free data channel adjustment, the decrease in lock-free data channel adjustment comprising: determining a data channel needing to be reduced, not carrying out new task allocation on the data channel needing to be reduced, and releasing the data channel needing to be reduced after waiting for the completion of the task carried out in the data channel needing to be reduced; the adding lock-free data channel adjustment includes: and determining the number of the data channels needing to be increased, increasing the deployment of the corresponding number of the data channels without lock, and binding corresponding system resources for the increased data channels without lock.
9. The system for improving the utilization efficiency of a storage system resource according to claim 8, wherein the number of the lockless data channels to be added or subtracted on each node of the system is calculated by calculating the difference between the current system performance requirement and the actual performance of the system, if the calculation result is the number of the lockless data channels to be added, the adding of the lockless data channels is performed, and if the calculation result is the number of the lockless data channels to be subtracted, the subtracting of the lockless data channels is performed.
10. The system for improving the utilization efficiency of storage system resources according to claim 6, wherein the correspondence between the deployment information of the lock-free data channel and the performance index reached by the deployment information is obtained by testing hardware configured by the storage system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310840847.XA CN116880752A (en) | 2023-07-07 | 2023-07-07 | Method and system for improving utilization efficiency of storage system resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310840847.XA CN116880752A (en) | 2023-07-07 | 2023-07-07 | Method and system for improving utilization efficiency of storage system resources |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116880752A true CN116880752A (en) | 2023-10-13 |
Family
ID=88267362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310840847.XA Pending CN116880752A (en) | 2023-07-07 | 2023-07-07 | Method and system for improving utilization efficiency of storage system resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116880752A (en) |
-
2023
- 2023-07-07 CN CN202310840847.XA patent/CN116880752A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11016956B2 (en) | Database management system with database hibernation and bursting | |
| CN111522636B (en) | Application container adjusting method, application container adjusting system, computer readable medium and terminal device | |
| US7139846B1 (en) | Computer system and method for performing low impact backup operations | |
| CN106133693B (en) | Moving method, device and the equipment of virtual machine | |
| US8341439B2 (en) | Power management apparatus and method thereof and power control system | |
| WO2012066604A1 (en) | Server system and method for managing the same | |
| US20150312167A1 (en) | Maximizing server utilization within a datacenter | |
| CN110196767B (en) | Service resource control method, device, equipment and storage medium | |
| US20050154576A1 (en) | Policy simulator for analyzing autonomic system management policy of a computer system | |
| US20120102503A1 (en) | Green computing via event stream management | |
| US8898674B2 (en) | Memory databus utilization management system and computer program product | |
| US20080271030A1 (en) | Kernel-Based Workload Management | |
| CN102111337A (en) | Method and system for task scheduling | |
| US10521258B2 (en) | Managing test services in a distributed production service environment | |
| CN109960591B (en) | Cloud application resource dynamic scheduling method for tenant resource encroachment | |
| CN110018781B (en) | Disk flow control method and device and electronic equipment | |
| WO2019011262A1 (en) | Method and apparatus for resource allocation | |
| CN102339233A (en) | Cloud computing centralized management platform | |
| CN110543355A (en) | method for automatically balancing cloud platform resources | |
| US9195514B2 (en) | System and method for managing P-states and C-states of a system | |
| US20130239114A1 (en) | Fine Grained Adaptive Throttling of Background Processes | |
| CN107203256B (en) | Energy-saving distribution method and device under network function virtualization scene | |
| US10033620B1 (en) | Partitioned performance adaptive policies and leases | |
| CN116880752A (en) | Method and system for improving utilization efficiency of storage system resources | |
| US11182189B2 (en) | Resource optimization for virtualization environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |