CN113741807A - Method, system, equipment and storage medium for improving system storage performance - Google Patents
Method, system, equipment and storage medium for improving system storage performance Download PDFInfo
- Publication number
- CN113741807A CN113741807A CN202110867262.8A CN202110867262A CN113741807A CN 113741807 A CN113741807 A CN 113741807A CN 202110867262 A CN202110867262 A CN 202110867262A CN 113741807 A CN113741807 A CN 113741807A
- Authority
- CN
- China
- Prior art keywords
- performance
- model
- layering
- disk
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000001514 detection method Methods 0.000 claims abstract description 13
- 230000015654 memory Effects 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 12
- 230000001133 acceleration Effects 0.000 abstract description 7
- 230000000737 periodic effect Effects 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 7
- 230000001360 synchronised effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000010438 heat treatment Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000007774 longterm Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007334 memory performance Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000013517 stratification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a method, a system, equipment and a storage medium for improving the storage performance of a system, wherein the method comprises the steps of presetting and layering a disk module according to the disk medium, and periodically detecting the performance of an I/O (input/output) model of the disk module after the presetting and layering to obtain the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block; judging whether the actual performance of the I/O model is the same as the preset layering performance, if the actual performance of the I/O model is different from the preset layering performance in a preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; based on the method, the method comprehensively confirms the rationality of the data hierarchy of the system with the preset disk medium according to the periodic disk performance detection and calibrates the data hierarchy according to the size data block, can always provide a high-efficiency hierarchical acceleration function for the system, and further improves the performance of the whole system.
Description
Technical Field
The invention belongs to the technical field of distributed file system storage, and particularly relates to a method, a system, equipment and a storage medium for improving the storage performance of a system.
Background
The memory array is composed of a large number of memory cells, and each memory cell can store 1-bit binary data (0, 1). Memory cells are typically arranged in a matrix of N rows by M columns. The method is characterized in that a plurality of disks form an array and are used as a single disk, data are stored in different disks in a segmenting (striping) mode, and when the data are accessed, related disks in the array act together, so that the data access time is greatly reduced, and meanwhile, the space utilization rate is better. Current unified array storage typically employs a layered acceleration function for the disk media to improve system performance. Different layering is carried out according to different types of magnetic disks such as FLASH, SAS, NL SAS and the like, hot data runs on an SLC FLASH layer, warm data runs on a TLC FLASH layer, cold data runs on a low-rotation-speed HDD layer and the like. Wherein, the SLC Single-Level Cell is a 1bit per Cell, 1 memory Cell can store 1bit data, and only two charging values of 0 and 1 exist. A Triple-Level Cell, i.e., a 3-bit per Cell, can store 3-bit data in 1 memory Cell.
However, the method of layering according to media is simple and rough, layering finally depends on different performances of the magnetic disks, according to different technical maturity or different process gaps, the SLC FLASH performance of some manufacturers is lower than that of MLC FLASH of other manufacturers, and if SLC of a low-performance manufacturer is set with run-hot data and the MLC of a relatively high-performance manufacturer runs-hot data, the hot data layer can become a bottleneck in IO stream, and the performance of the whole system is further influenced.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method, a system, equipment and a storage medium for improving the storage performance of the system, the rationality of the data hierarchy of the system where the system is located is comprehensively confirmed with a preset disk medium according to the periodic disk performance detection, and the data hierarchy is respectively calibrated according to the large data block and the small data block, so that the high-efficiency hierarchy acceleration function can be always provided for the system, and the performance of the whole system is further improved.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method of improving system memory performance, comprising the steps of:
periodically detecting the performance of an I/O model of the disk module after preset layering to obtain the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block;
judging whether the actual performance of the I/O model is the same as the preset layering performance, if the actual performance of the I/O model is different from the preset layering performance in a preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; the performance difference ratio is a ratio of predicted layering performance to actual performance.
Further, before the periodically detecting the I/O model performance of the disk module after the layering, the method further includes: and carrying out preset layering on the disk module according to the disk medium, and sending the preset layering information to an operating system.
Further, the pre-layering the disk module according to the disk medium includes:
storing the hot data to FLASH;
storing the temperature data to the SAS;
cold data is stored to the NL SAS.
Further, the I/O model specifically includes a big data block sequential storage model, a big data block random storage model, a small data block sequential storage model, and a small data block random storage model;
the range of the large data block is [256KB,1M ]; the small data block has a range of (0KB,16KB ].
Further, the method for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model includes:
and periodically polling the I/O models to obtain the performance value fed back by each I/O model.
Further, the adjusting the layering of the disk modules according to the ratio of the performance difference values includes:
judging whether the current system is fully loaded, if so, setting the first proportion of the performance difference ratio to be 10-15%, and adjusting the layering of the disk modules according to the first proportion; and if the disk module is not fully loaded, setting a second proportion of the performance difference ratio to be 3-5%, and adjusting the layering of the disk module according to the second proportion.
Further, the method further comprises the step of not adjusting if the actual performance of the I/O model is the same as the performance of the preset layer.
The invention also provides a system for improving the storage performance of the system, which comprises a detection module and an adjustment module;
the detection module is used for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block;
the adjusting module is used for judging whether the actual performance of the I/O model is the same as the preset layering performance or not, and if the actual performance of the I/O model is different from the preset layering performance in the preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; the performance difference ratio is a ratio of predicted layering performance to actual performance.
The invention also proposes a device comprising:
a memory for storing a computer program;
a processor for implementing the method steps when executing the computer program.
The invention also proposes a readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the method steps.
The effect provided in the summary of the invention is only the effect of the embodiment, not all the effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
the invention has proposed a method for improving the system storage performance, system, apparatus and storage medium, said method comprises presetting and layering the disk module according to the disk medium, and will, preset and delaminate the information to send to the operating system, detect I/O model performance of the disk module after presetting and layering periodically and get the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block; judging whether the actual performance of the I/O model is the same as the preset layering performance, if the actual performance of the I/O model is different from the preset layering performance in a preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; the performance difference ratio is the ratio of predicted stratification performance to actual performance. The method comprehensively confirms the rationality of the data hierarchy of the system with the preset disk medium according to the periodic disk performance detection, respectively calibrates the data hierarchy according to the size data block, ensures that a high-efficiency hierarchy acceleration function can be provided for the system all the time, and further improves the performance of the whole system.
According to the invention, the layering of the disk modules is adjusted according to the proportion of the performance difference ratio, if the performance difference of the same media disk of the same manufacturer is caused by the use time length and the process difference, the disk modules can be judged to be different data layers, and the dynamic adjustment of the invention ensures that a high-efficiency layering acceleration function can be provided for the system all the time.
Based on a method for improving the system storage performance, the invention also provides a system, equipment and a storage medium for improving the system storage performance, the process of realizing the method is realized in a modularized way, and the method also has the function of the method, and is not described herein again.
Drawings
Fig. 1 is a flowchart of a method for improving system storage performance according to embodiment 1 of the present invention;
fig. 2 is a schematic topology diagram of a module implementation of a method for improving system storage performance according to embodiment 1 of the present invention;
fig. 3 is a schematic diagram of a system for improving system storage performance according to embodiment 2 of the present invention.
Detailed Description
In order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
Example 1
The embodiment 1 of the invention provides a method for improving the storage performance of a system, which solves the technical problem that the system performance is influenced by layering, simple response and rough answer according to different media in the prior art.
Fig. 1 is a flowchart of a method for improving system storage performance according to embodiment 1 of the present invention.
In step S101, the system initializes and divides the storage module into a plurality of data layers according to a preset storage medium. FLASH was Tier0, SAS was Tier1 and NL SAS was Tier 2. Storing the instant heating data to FLASH; storing the temperature data to the SAS; cold data is stored to the NL SAS.
In step S102, periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model;
wherein the I/O model is used to characterize the storage mode of the data block. The I/O model in the invention includes but is not limited to the following four modes, a big data block sequence storage model, a big data block random storage model, a small data sequence storage model and a small data block random storage model; wherein the large data block has a range of [256KB,1M ]; the range of small data blocks is (0KB,16KB ].
The method for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model comprises the following steps:
and periodically polling the I/O models to obtain the performance value fed back by each I/O model. Namely, the four I/O models are simulated to issue data, and the performance value fed back by each I/O model is obtained. Different I/O models are then put into the performance optimized data layer.
In the present invention, the time period differs depending on the medium. The time period is set to be relatively short, generally 24 hours to 72 hours, if the system is initialized, the time period is set to be 15 days if the system is stable, and the time period may be set to be 30 days if the system is operated for a longer time and the system is more stable. The time period may also be set shorter, say less than 24 hours, if the I/O model detected by the system is changing all the time. If the I/O model detected by the system is unchanged for a long time, such as sequential reading of a long-term large data block, the time can also be set to be more than 30 days.
In step S103, it is determined whether the actual performance of the I/O model is the same as the preset layering performance, and if they are different within the preset period, the layering of the disk module is adjusted according to the ratio of the performance difference; the performance difference ratio is a ratio of predicted layering performance to actual performance.
And if the actual performance of the I/O model is the same as the performance of the preset layer, no adjustment is made.
The preset period in the present invention may be three periods. The protection scope of the present invention is not limited to the number of cycles listed in example 1, and can be set by those skilled in the art according to practical situations.
Adjusting the layering of the disk modules according to the ratio of the performance difference values comprises:
judging whether the current system is fully loaded, if so, setting the first proportion of the performance difference ratio to be 10-15%, and adjusting the layering of the disk modules according to the first proportion; and if the disk module is not fully loaded, setting the second proportion of the performance difference ratio to be 3-5%, and adjusting the layering of the disk module according to the second proportion.
According to this principle, the same media disk of the same manufacturer may be determined as different data layers if the performance gap is caused by the use time and process variation. In addition, the real-time disk performance is also layered according to different IO models.
Fig. 2 is a schematic topology diagram of a module implementation of a method for improving system storage performance according to embodiment 1 of the present invention; the method for improving the system storage performance disclosed in embodiment 1 of the present invention is implemented by using a dynamic hierarchical management module, where the dynamic hierarchical management module is located on a board card and is mainly applied to programmable logic devices such as an ARM. In addition, the topological graph also comprises an OS module, a hard disk module, an indication module, a wireless module and a serial port module
An OS module: the module carries out IO issuing according to a data layering mode provided by the dynamic layering management module.
A disk module: and the dynamic hierarchical management module directly manages and sends various disk management information to the dynamic hierarchical management module.
An indication module: the module is positioned on the board card and is directly controlled by the serial port module to externally indicate the real-time state of the current dynamic hierarchical management module.
A wireless module: the serial port module signal can be converted into wireless signals such as WIFI, and information interaction can be carried out between the outside and the dynamic hierarchical management module without an entity serial port line.
A serial port module: and information interaction and related parameter setting between the outside and the dynamic hierarchical management module can be performed through the serial port module.
According to the method for improving the storage performance of the system, the rationality of the data hierarchy of the system is comprehensively confirmed with the preset disk medium according to the periodic disk performance detection, and the data hierarchy is respectively calibrated according to the large data block and the small data block, so that a high-efficiency hierarchy acceleration function can be always provided for the system, and the performance of the whole system is improved.
Example 2
Based on the method for improving the system storage performance provided by the embodiment 1 of the present invention, the embodiment 2 of the present invention also provides a system for improving the system storage performance. Fig. 3 is a schematic diagram of a system for improving system storage performance according to embodiment 2 of the present invention. The system comprises a detection module and an adjustment module;
the detection module is used for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block;
the adjusting module is used for judging whether the actual performance of the I/O model is the same as the preset layering performance or not, and if the actual performance of the I/O model is different from the preset layering performance in the preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; the performance difference ratio is a ratio of predicted layering performance to actual performance.
The system also includes a conversion module; the dividing module is used for initializing the system and dividing the storage module into a plurality of data layers according to a preset storage medium.
FLASH was Tier0, SAS was Tier1 and NL SAS was Tier 2. Storing the instant heating data to FLASH; storing the temperature data to the SAS; cold data is stored to the NL SAS.
Wherein, the detection module realizes in-process:
wherein the I/O model is used to characterize the storage mode of the data block. The I/O model in the invention includes but is not limited to the following four modes, a big data block sequence storage model, a big data block random storage model, a small data sequence storage model and a small data block random storage model; wherein the large data block has a range of [256KB,1M ]; the range of small data blocks is (0KB,16KB ].
The method for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model comprises the following steps:
and periodically polling the I/O models to obtain the performance value fed back by each I/O model. Namely, the four I/O models are simulated to issue data, and the performance value fed back by each I/O model is obtained. Different I/O models are then put into the performance optimized data layer.
In the present invention, the time period differs depending on the medium. The time period is set to be relatively short, generally 24 hours to 72 hours, if the system is initialized, the time period is set to be 15 days if the system is stable, and the time period may be set to be 30 days if the system is operated for a longer time and the system is more stable. The time period may also be set shorter, say less than 24 hours, if the I/O model detected by the system is changing all the time. If the I/O model detected by the system is unchanged for a long time, such as sequential reading of a long-term large data block, the time can also be set to be more than 30 days.
The detection module is realized in the process that: and if the actual performance of the I/O model is the same as the performance of the preset layer, no adjustment is made.
The preset period in the present invention may be three periods. The protection scope of the present invention is not limited to the number of cycles listed in example 1, and can be set by those skilled in the art according to practical situations.
Adjusting the layering of the disk modules according to the ratio of the performance difference values comprises:
judging whether the current system is fully loaded, if so, setting the first proportion of the performance difference ratio to be 10-15%, and adjusting the layering of the disk modules according to the first proportion; and if the disk module is not fully loaded, setting the second proportion of the performance difference ratio to be 3-5%, and adjusting the layering of the disk module according to the second proportion.
According to this principle, the same media disk of the same manufacturer may be determined as different data layers if the performance gap is caused by the use time and process variation. In addition, the real-time disk performance is also layered according to different IO models.
The embodiment 2 of the invention provides a system for improving the storage performance of the system, which comprehensively confirms the rationality of the data hierarchy of the system with a preset magnetic disk medium according to the periodic magnetic disk performance detection, and respectively calibrates the data hierarchy according to the size data block, thereby ensuring that a high-efficiency hierarchical acceleration function can be always provided for the system, and further improving the performance of the whole system.
Example 3
The invention also proposes a device comprising:
a memory for storing a computer program;
a processor for implementing the method steps when executing the computer program as follows:
in step S101, the system initializes and divides the storage module into a plurality of data layers according to a preset storage medium. FLASH was Tier0, SAS was Tier1 and NL SAS was Tier 2. Storing the instant heating data to FLASH; storing the temperature data to the SAS; cold data is stored to the NL SAS.
In step S102, periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model;
wherein the I/O model is used to characterize the storage mode of the data block. The I/O model in the invention includes but is not limited to the following four modes, a big data block sequence storage model, a big data block random storage model, a small data sequence storage model and a small data block random storage model; wherein the large data block has a range of [256KB,1M ]; the range of small data blocks is (0KB,16KB ].
The method for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model comprises the following steps:
and periodically polling the I/O models to obtain the performance value fed back by each I/O model. Namely, the four I/O models are simulated to issue data, and the performance value fed back by each I/O model is obtained. Different I/O models are then put into the performance optimized data layer.
In the present invention, the time period differs depending on the medium. The time period is set to be relatively short, generally 24 hours to 72 hours, if the system is initialized, the time period is set to be 15 days if the system is stable, and the time period may be set to be 30 days if the system is operated for a longer time and the system is more stable. The time period may also be set shorter, say less than 24 hours, if the I/O model detected by the system is changing all the time. If the I/O model detected by the system is unchanged for a long time, such as sequential reading of a long-term large data block, the time can also be set to be more than 30 days.
In step S103, it is determined whether the actual performance of the I/O model is the same as the preset layering performance, and if they are different within the preset period, the layering of the disk module is adjusted according to the ratio of the performance difference; the performance difference ratio is a ratio of predicted layering performance to actual performance.
And if the actual performance of the I/O model is the same as the performance of the preset layer, no adjustment is made.
The preset period in the present invention may be three periods. The protection scope of the present invention is not limited to the number of cycles listed in example 1, and can be set by those skilled in the art according to practical situations.
Adjusting the layering of the disk modules according to the ratio of the performance difference values comprises:
judging whether the current system is fully loaded, if so, setting the first proportion of the performance difference ratio to be 10-15%, and adjusting the layering of the disk modules according to the first proportion; and if the disk module is not fully loaded, setting the second proportion of the performance difference ratio to be 3-5%, and adjusting the layering of the disk module according to the second proportion.
According to this principle, the same media disk of the same manufacturer may be determined as different data layers if the performance gap is caused by the use time and process variation. In addition, the real-time disk performance is also layered according to different IO models.
Need to explain: the technical solution of the present invention also provides an electronic device, including: the communication interface can carry out information interaction with other equipment such as network equipment and the like; the processor is connected with the communication interface to realize information interaction with other equipment, and is used for executing the method for improving the storage performance of the system provided by one or more technical schemes when running the computer program, and the computer program is stored on the memory. Of course, in practice, the various components in an electronic device are coupled together by a bus system. It will be appreciated that a bus system is used to enable communications among the components. The bus system includes a power bus, a control bus, and a status signal bus in addition to a data bus. The memory in the embodiments of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program for operating on an electronic device. It will be appreciated that the memory can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memories described in the embodiments of the present application are intended to comprise, without being limited to, these and any other suitable types of memory. The method disclosed in the embodiments of the present application may be applied to a processor, or may be implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The processor may be a general purpose processor, a DSP (Digital Signal Processing, i.e., a chip capable of implementing Digital Signal Processing technology), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The processor may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in a memory where a processor reads the programs in the memory and in combination with its hardware performs the steps of the method as previously described. When the processor executes the program, corresponding processes in the methods of the embodiments of the present application are implemented, and for brevity, are not described herein again.
Example 4
The invention also proposes a readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method steps of:
in step S101, the system initializes and divides the storage module into a plurality of data layers according to a preset storage medium. FLASH was Tier0, SAS was Tier1 and NL SAS was Tier 2. Storing the instant heating data to FLASH; storing the temperature data to the SAS; cold data is stored to the NL SAS.
In step S102, periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model;
wherein the I/O model is used to characterize the storage mode of the data block. The I/O model in the invention includes but is not limited to the following four modes, a big data block sequence storage model, a big data block random storage model, a small data sequence storage model and a small data block random storage model; wherein the large data block has a range of [256KB,1M ]; the range of small data blocks is (0KB,16KB ].
The method for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model comprises the following steps:
and periodically polling the I/O models to obtain the performance value fed back by each I/O model. Namely, the four I/O models are simulated to issue data, and the performance value fed back by each I/O model is obtained. Different I/O models are then put into the performance optimized data layer.
In the present invention, the time period differs depending on the medium. The time period is set to be relatively short, generally 24 hours to 72 hours, if the system is initialized, the time period is set to be 15 days if the system is stable, and the time period may be set to be 30 days if the system is operated for a longer time and the system is more stable. The time period may also be set shorter, say less than 24 hours, if the I/O model detected by the system is changing all the time. If the I/O model detected by the system is unchanged for a long time, such as sequential reading of a long-term large data block, the time can also be set to be more than 30 days.
In step S103, it is determined whether the actual performance of the I/O model is the same as the preset layering performance, and if they are different within the preset period, the layering of the disk module is adjusted according to the ratio of the performance difference; the performance difference ratio is a ratio of predicted layering performance to actual performance.
And if the actual performance of the I/O model is the same as the performance of the preset layer, no adjustment is made.
The preset period in the present invention may be three periods. The protection scope of the present invention is not limited to the number of cycles listed in example 1, and can be set by those skilled in the art according to practical situations.
Adjusting the layering of the disk modules according to the ratio of the performance difference values comprises:
judging whether the current system is fully loaded, if so, setting the first proportion of the performance difference ratio to be 10-15%, and adjusting the layering of the disk modules according to the first proportion; and if the disk module is not fully loaded, setting the second proportion of the performance difference ratio to be 3-5%, and adjusting the layering of the disk module according to the second proportion.
According to this principle, the same media disk of the same manufacturer may be determined as different data layers if the performance gap is caused by the use time and process variation. In addition, the real-time disk performance is also layered according to different IO models.
For a description of a relevant part in the device and the storage medium for improving the system storage performance provided in the embodiment of the present application, reference may be made to a detailed description of a corresponding part in the method for improving the system storage performance provided in embodiment 1 of the present application, and details are not described here again.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include elements inherent in the list. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, the scope of the present invention is not limited thereto. Various modifications and alterations will occur to those skilled in the art based on the foregoing description. And are neither required nor exhaustive of all embodiments. On the basis of the technical scheme of the invention, various modifications or changes which can be made by a person skilled in the art without creative efforts are still within the protection scope of the invention.
Claims (10)
1. A method for improving system storage performance, comprising the steps of:
periodically detecting the performance of an I/O model of the disk module after preset layering to obtain the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block;
judging whether the actual performance of the I/O model is the same as the preset layering performance, if the actual performance of the I/O model is different from the preset layering performance in a preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; the performance difference ratio is a ratio of predicted layering performance to actual performance.
2. The method of claim 1, wherein before the periodically checking the I/O model performance of the disk module after the layering, the method further comprises: and carrying out preset layering on the disk module according to the disk medium, and sending the preset layering information to an operating system.
3. The method of claim 2, wherein the pre-layering disk modules according to disk media comprises:
storing the hot data to FLASH;
storing the temperature data to the SAS;
cold data is stored to the NL SAS.
4. The method for improving the storage performance of the system according to claim 1, wherein the I/O model specifically comprises a big data block sequential storage model, a big data block random storage model, a small data block sequential storage model, and a small data block random storage model;
the range of the large data block is [256KB,1M ]; the small data block has a range of (0KB,16KB ].
5. The method according to claim 4, wherein the method for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model comprises:
and periodically polling the I/O models to obtain the performance value fed back by each I/O model.
6. The method of claim 1, wherein the adjusting the hierarchy of the disk modules according to the ratio of the performance difference comprises:
judging whether the current system is fully loaded, if so, setting the first proportion of the performance difference ratio to be 10-15%, and adjusting the layering of the disk modules according to the first proportion; and if the disk module is not fully loaded, setting a second proportion of the performance difference ratio to be 3-5%, and adjusting the layering of the disk module according to the second proportion.
7. The method of claim 1, further comprising not adjusting if the actual performance of the I/O model is the same as the predetermined tier performance.
8. A system for improving the storage performance of the system is characterized by comprising a detection module and an adjustment module;
the detection module is used for periodically detecting the performance of the I/O model of the disk module after the preset layering to obtain the actual performance of the I/O model; the I/O model is used for representing the storage mode of the data block;
the adjusting module is used for judging whether the actual performance of the I/O model is the same as the preset layering performance or not, and if the actual performance of the I/O model is different from the preset layering performance in the preset period, adjusting the layering of the disk module according to the ratio of the performance difference values; the performance difference ratio is a ratio of predicted layering performance to actual performance.
9. An apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110867262.8A CN113741807B (en) | 2021-07-29 | 2021-07-29 | Method, system, equipment and storage medium for improving system storage performance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110867262.8A CN113741807B (en) | 2021-07-29 | 2021-07-29 | Method, system, equipment and storage medium for improving system storage performance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113741807A true CN113741807A (en) | 2021-12-03 |
| CN113741807B CN113741807B (en) | 2023-08-11 |
Family
ID=78729487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110867262.8A Active CN113741807B (en) | 2021-07-29 | 2021-07-29 | Method, system, equipment and storage medium for improving system storage performance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113741807B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109240611A (en) * | 2018-08-28 | 2019-01-18 | 郑州云海信息技术有限公司 | The cold and hot data hierarchy method of small documents, small documents data access method and its device |
| CN109496292A (en) * | 2018-10-16 | 2019-03-19 | 深圳市锐明技术股份有限公司 | A kind of disk management method, disk management device and electronic equipment |
| CN111263931A (en) * | 2017-11-17 | 2020-06-09 | 国际商业机器公司 | Policy-based optimization of cloud resources on tiered storage operations |
| US20200257450A1 (en) * | 2017-10-30 | 2020-08-13 | Alibaba Group Holding Limited | Data hierarchical storage and hierarchical query method and apparatus |
-
2021
- 2021-07-29 CN CN202110867262.8A patent/CN113741807B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200257450A1 (en) * | 2017-10-30 | 2020-08-13 | Alibaba Group Holding Limited | Data hierarchical storage and hierarchical query method and apparatus |
| CN111263931A (en) * | 2017-11-17 | 2020-06-09 | 国际商业机器公司 | Policy-based optimization of cloud resources on tiered storage operations |
| CN109240611A (en) * | 2018-08-28 | 2019-01-18 | 郑州云海信息技术有限公司 | The cold and hot data hierarchy method of small documents, small documents data access method and its device |
| CN109496292A (en) * | 2018-10-16 | 2019-03-19 | 深圳市锐明技术股份有限公司 | A kind of disk management method, disk management device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113741807B (en) | 2023-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102370760B1 (en) | Zone formation for zoned namespaces | |
| KR20210079394A (en) | ZNSs in Solid State Drives | |
| US10185609B2 (en) | Virtual timer for data retention | |
| US10235069B2 (en) | Load balancing by dynamically transferring memory range assignments | |
| US8099543B2 (en) | Methods of operarting memory devices within a communication protocol standard timeout requirement | |
| CN105183379A (en) | Mixed memory data backup system and method | |
| CN114730604A (en) | Dynamic ZNS open zone activity restriction | |
| US10163502B2 (en) | Selective performance level modes of operation in a non-volatile memory | |
| CN112148525A (en) | Enabling faster and adjusted device initialization time | |
| US12013762B2 (en) | Meta data protection against unexpected power loss in a memory system | |
| CN112596681A (en) | Re-reading command processing method, flash memory controller and solid state disk | |
| KR20130068915A (en) | Memory system, and method for controlling operation thereof | |
| CN110515549B (en) | Data migration method and device | |
| US11041763B2 (en) | Adaptive throttling | |
| JP2022172443A (en) | Method and device relating to memory chip row hammer threat back pressure signal and host-side response | |
| CN104575598A (en) | Resistive memory device, operating method thereof, and system having the same | |
| KR102589609B1 (en) | Snapshot management in partitioned storage | |
| CN113741807B (en) | Method, system, equipment and storage medium for improving system storage performance | |
| US11748013B2 (en) | Grouping blocks based on power cycle and power on time | |
| KR20240073166A (en) | Controlled system management based on storage device thermal load | |
| US11704047B2 (en) | Temperature readings for memory devices to reduce temperature compensation errors | |
| US20230251792A1 (en) | Memory Device Based Accelerated Deep-Learning System | |
| US11507508B2 (en) | Prediction-based selective flushing of data to memory | |
| CN114924696A (en) | Method, apparatus, medium, and program product for storage management | |
| US20230251935A1 (en) | Storage device for storing model checkpoints of recommendation deep-learning models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |