CN112783446B - Data writing method and system of storage system - Google Patents
Data writing method and system of storage system Download PDFInfo
- Publication number
- CN112783446B CN112783446B CN202110089916.9A CN202110089916A CN112783446B CN 112783446 B CN112783446 B CN 112783446B CN 202110089916 A CN202110089916 A CN 202110089916A CN 112783446 B CN112783446 B CN 112783446B
- Authority
- CN
- China
- Prior art keywords
- maximum performance
- request response
- per
- data write
- performance per
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000004044 response Effects 0.000 claims abstract description 145
- 238000012545 processing Methods 0.000 claims abstract description 99
- 238000013507 mapping Methods 0.000 claims abstract description 46
- 230000008569 process Effects 0.000 claims abstract description 23
- 238000012360 testing method Methods 0.000 claims description 19
- 230000003750 conditioning effect Effects 0.000 claims description 6
- 230000001105 regulatory effect Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
- G06F3/0641—De-duplication techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data writing method and a data writing system of a storage system, wherein the method comprises the following steps: continuously receiving data writing requests, and determining whether to execute a deduplication process on the data writing requests based on predetermined performance impact strengths or dynamically adjusted performance differences; responding to the execution flow, searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request, and establishing and saving mapping information of the physical block address mapping from the logical block address to the physical block address of the data writing request in response to the hit; in response to not performing the flow or missing an existing physical block address mapping, a data write request is performed to the storage system to write the data in the data write request. The invention can reduce the processing resources occupied by the deduplication feature to alleviate or eliminate the performance impact on the storage system.
Description
Technical Field
The present invention relates to the field of journaling, and more particularly, to a data writing method and system for a storage system.
Background
The characteristic of data deduplication (deduplication for short) is a characteristic of improving the utilization efficiency of a storage space in a Storage Area Network (SAN) storage system, and by calculating and storing Fingerprints (FP) and mapping information from the FP to a Physical Block Address (PBA) for a written data block, a processing flow is to calculate the fingerprints FP of the data block first when the data block is written, and query the existing mapping information from the FP to the PBA according to the FP, if the fingerprints FP are found, read out data content corresponding to the PBA for weak hashing (a hash algorithm and a hash collision rate of a length are higher), compare the data content with the data block content to be written one by one to see whether the fingerprints are the same, if the fingerprints are the same, determine that the data block is a duplicate data block, and if the strong hashing does not need to read the data content corresponding to the PBA, and determine that the data block is the duplicate data as long as the FPs are the same. If the repeated data block is found, the data space PBA does not need to be allocated, and the LBA (logical block address) - > PBA mapping information is directly stored, wherein the PBA comes from the FP- > PBA mapping information obtained by query, otherwise, the data space PBA is allocated, the content of the data block is written into the PBA, and the FP- > PBA and LBA- > PBA mapping information is newly established.
Due to the fact that deduplication characteristics need to calculate data block fingerprints, store and query FP- > PBA mapping information, and even need to read data of PBA for comparison, the influence of deduplication starting characteristics on the performance of a storage system is obvious, the performance of the storage system has higher priority in some application scenarios, deduplication rate priority is lower, and conventional implementation cannot meet the requirement of guaranteeing lower performance influence of the storage system under the condition of deduplication starting.
Aiming at the problem that the performance is affected by the fact that a deduplication characteristic is used to occupy processing resources of a storage system in the prior art, no effective solution is available at present.
Disclosure of Invention
In view of this, an embodiment of the present invention is to provide a data writing method and system for a storage system, which can reduce processing resources occupied by a deduplication feature to alleviate or eliminate performance impact on the storage system.
In view of the above object, a first aspect of the embodiments of the present invention provides a data writing method for a storage system, including the following steps:
continuously receiving data writing requests, and determining whether to execute a deduplication process on the data writing requests based on predetermined performance impact strengths or dynamically adjusted performance differences;
searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request in response to the execution flow, and establishing and saving mapping information of the logical block address to the physical block address mapping of the physical block address of the data writing request in response to the hit;
in response to not performing the flow or missing an existing physical block address mapping, a data write request is performed to the storage system to write the data in the data write request.
In some embodiments, determining whether to perform a deduplication process for a data write request based on a predetermined performance impact strength comprises:
checking the maximum performance per second, the conditioning time period, and the maximum performance impact strength of the storage system;
determining a deduplication processing cycle according to a maximum performance per second and an adjustment time period;
determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle;
specifying a number of data write requests corresponding to the number of processing skips among the received data write requests per deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not specified to execute the flow.
In some embodiments, determining a deduplication processing cycle based on a maximum performance per second and a conditioning time period comprises: determining a deduplication processing cycle according to a product of a maximum performance per second and an adjustment time period; determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle comprises: the number of processing skips is determined based on a product of the performance impact strength divided by the maximum performance impact strength and the deduplication processing cycle.
In some embodiments, determining whether to perform a deduplication flow for a data write request based on the dynamically adjusted performance difference comprises:
testing the maximum performance per second and the request response delay of the storage system under different dynamic adjustment proportions to iteratively adjust the dynamic adjustment proportions until the maximum performance per second and the request response delay meet the maximum performance expectation per second and the request response delay expectation per second;
determining the number of processing skips based on the dynamic adjustment proportion and the repeated data deletion processing cycle after the iterative adjustment;
specifying a number of data write requests corresponding to the number of processing skips among the received data write requests per deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not specified to execute the flow.
In some embodiments, testing the maximum performance per second and the request response latency of the storage system at different dynamic scaling ratios to iteratively adjust the dynamic scaling until the maximum performance per second and the request response latency satisfy the maximum performance per second and the request response latency expectations comprises:
setting the dynamic adjustment proportion to be 0 to test the storage system, and obtaining a first maximum performance per second and a first request response time delay;
setting the dynamic adjustment ratio to be 1 to test the storage system, and obtaining a second maximum performance per second and a second request response time delay;
in response to the second maximum performance per second being greater than the maximum performance expectation per second or the second request response delay being less than the request response delay expectation, directly setting the dynamically adjusted scaling after the iterative adjustment to 1;
in response to the first maximum performance per second being less than the maximum performance expectation per second or the first request response delay being greater than the request response delay expectation, directly setting the dynamically adjusted scaling after iterative adjustment to 0;
in response to the iteratively adjusted dynamic scaling not being set directly, dynamically updating the dynamic scaling to a ratio of a difference between the first maximum performance per second and the maximum performance per second expectation to a difference between the first maximum performance per second and the second maximum performance per second expectation;
retesting the storage system based on the updated dynamic adjustment ratio to obtain a third maximum performance per second and a third request response delay;
and iteratively updating the dynamic adjustment proportion by a determined step amplitude according to the third maximum performance per second and the third request response delay and the size relation between the maximum performance expectation per second and the request response delay expectation until the third maximum performance per second and the third request response delay meet the maximum performance expectation per second and the request response delay expectation.
In some embodiments, iteratively updating the dynamic adjustment scale at the determined step size based on the third maximum performance per second and the third request response latency, in relation to the magnitude of the maximum performance per second expectation and the request response latency expectation, until the third maximum performance per second and the third request response latency satisfy the maximum performance per second expectation and the request response latency expectation comprises:
in response to the third maximum performance per second being greater than the maximum performance expectation per second and the third request response latency being less than the request response latency expectation, superimposing a step magnitude for the dynamic adjustment scale to update the dynamic adjustment scale, and iteratively testing the memory system using the updated dynamic adjustment scale to update the third maximum performance per second and the third request response latency until the third maximum performance per second is less than the maximum performance expectation per second or the third request response latency is greater than the request response latency expectation to set the current dynamic adjustment scale minus the step magnitude to the iteratively adjusted dynamic adjustment scale;
in response to the third maximum performance per second being less than the maximum performance expectation per second or the third request response latency being greater than the request response latency expectation, the step magnitude is subtracted from the dynamic adjustment scale to update the dynamic adjustment scale, and the memory system is iteratively tested using the updated dynamic adjustment scale to update the third maximum performance per second and the third request response latency until the third maximum performance per second is greater than the maximum performance expectation per second and the third request response latency is less than the request response latency expectation to set the current dynamic adjustment scale to the iteratively adjusted dynamic adjustment scale.
In some embodiments, determining the number of processing skips based on the iteratively adjusted dynamic adjustment ratio and the deduplication processing cycle comprises: and determining the processing skipping number based on the product of the dynamic adjustment ratio after the iterative adjustment and the repeated data deleting processing circulation.
A second aspect of an embodiment of the present invention provides a data writing system for a storage system, including:
a processor; and
a memory storing program code executable by the processor, the program code when executed performing the steps of:
continuously receiving data writing requests, and determining whether to execute a deduplication process on the data writing requests based on predetermined performance impact strengths or dynamically adjusted performance differences;
searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request in response to the execution flow, and establishing and saving mapping information of the logical block address to the physical block address mapping of the physical block address of the data writing request in response to the hit;
in response to not performing the flow or missing an existing physical block address mapping in the previous step, a data write request is performed to the storage system to write the data in the data write request.
In some embodiments, determining whether to perform a deduplication process for a data write request based on a predetermined performance impact strength comprises:
checking the maximum performance per second, the conditioning time period, and the maximum performance impact strength of the storage system;
determining a deduplication processing cycle according to the maximum performance per second and the adjustment time period;
determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle;
specifying a number of data write requests corresponding to the number of processing skips among the received data write requests per deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not specified to execute the flow.
In some embodiments, determining whether to perform a deduplication flow for a data write request based on the dynamically adjusted performance difference comprises:
testing the maximum performance per second and the request response delay of the storage system under different dynamic adjustment proportions to iteratively adjust the dynamic adjustment proportions until the maximum performance per second and the request response delay meet the maximum performance expectation per second and the request response delay expectation per second;
determining the number of processing skips based on the dynamic adjustment proportion and the repeated data deleting processing cycle after the iterative adjustment;
specifying, among the received data write requests for each deduplication processing cycle, a number of data write requests corresponding to the number of processing skips so as not to execute the flow, and causing unspecified data write requests to execute the flow.
The invention has the following beneficial technical effects: according to the data writing method and the data writing system of the storage system, provided by the embodiment of the invention, whether a repeated data deleting process is executed on the data writing request is determined by continuously receiving the data writing request and based on the predetermined performance influence strength or the dynamically adjusted performance difference; searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request in response to the execution flow, and establishing and saving mapping information of the logical block address to the physical block address mapping of the physical block address of the data writing request in response to the hit; in response to the technical scheme of not executing the flow or missing the existing physical block address mapping and executing the data writing request to the storage system to write the data in the data writing request, the processing resource occupied by the deduplication feature can be reduced to alleviate or eliminate the performance impact on the storage system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a data writing method of a storage system according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the foregoing, a first aspect of the embodiments of the present invention provides an embodiment of a data writing method for a storage system, which reduces processing resources occupied by a deduplication feature to mitigate or eliminate an impact on performance of the storage system. Fig. 1 is a schematic flow chart illustrating a data writing method of a storage system according to the present invention.
The data writing method of the storage system, as shown in fig. 1, includes the following steps:
step S101, continuously receiving a data writing request, and determining whether to execute a data de-duplication process on the data writing request based on predetermined performance impact strength or dynamically adjusted performance difference;
step S103, responding to the execution flow, searching whether the existing physical block address mapping is hit in the storage system based on the metadata of the data writing request, and establishing and saving mapping information of the logical block address of the data writing request to the physical block address mapping of the physical block address in response to the hit;
step S105, in response to not performing the flow or missing an existing physical block address mapping, performs a data write request to the storage system to write the data in the data write request.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), or the like. Embodiments of the computer program may achieve the same or similar effects as any of the preceding method embodiments to which it corresponds.
In some embodiments, determining whether to perform a deduplication process for a data write request based on a predetermined performance impact strength comprises:
checking the maximum performance per second, the conditioning time period, and the maximum performance impact strength of the storage system;
determining a deduplication processing cycle according to the maximum performance per second and the adjustment time period;
determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle;
specifying a number of data write requests corresponding to the number of processing skips among the received data write requests per deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not specified to execute the flow.
In some embodiments, determining a deduplication processing cycle based on the maximum performance per second and the adjustment time period comprises: determining a deduplication processing cycle based on a product of a maximum performance per second and an adjustment time period; determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle comprises: the number of processing skips is determined based on a product of the quotient of the performance impact strength and the maximum performance impact strength and the deduplication processing cycle.
In some embodiments, determining whether to perform a deduplication flow for a data write request based on the dynamically adjusted performance difference comprises:
testing the maximum performance per second and the request response delay of the storage system under different dynamic adjustment proportions to iteratively adjust the dynamic adjustment proportions until the maximum performance per second and the request response delay meet the maximum performance expectation per second and the request response delay expectation per second;
determining the number of processing skips based on the dynamic adjustment proportion and the repeated data deleting processing cycle after the iterative adjustment;
specifying a number of data write requests corresponding to the number of processing skips among the received data write requests per deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not specified to execute the flow.
In some embodiments, testing the maximum performance per second and the request response latency of the storage system at different dynamic scaling ratios to iteratively adjust the dynamic scaling until the maximum performance per second and the request response latency satisfy the maximum performance per second and the request response latency expectations comprises:
setting the dynamic adjustment proportion to be 0 to test the storage system, and obtaining a first maximum performance per second and a first request response time delay;
setting the dynamic adjustment ratio to be 1 to test the storage system, and obtaining a second maximum performance per second and a second request response time delay;
in response to the second maximum performance per second being greater than the maximum performance expectation per second or the second request response delay being less than the request response delay expectation, directly setting the dynamically adjusted scaling after the iterative adjustment to 1;
in response to the first maximum performance per second being less than the maximum performance expectation per second or the first request response delay being greater than the request response delay expectation, directly setting the dynamically adjusted scaling after iterative adjustment to 0;
in response to the iteratively adjusted dynamic scaling not being set directly, dynamically updating the dynamic scaling to a ratio of a difference between the first maximum performance per second and the maximum performance per second expectation to a difference between the first maximum performance per second and the second maximum performance per second expectation;
retesting the storage system based on the updated dynamic adjustment ratio to obtain a third maximum performance per second and a third request response delay;
and iteratively updating the dynamic adjustment proportion by a determined step amplitude according to the third maximum performance per second and the third request response time delay and the size relation between the maximum performance per second and the request response time delay expectation until the third maximum performance per second and the third request response time delay meet the maximum performance expectation per second and the request response time delay expectation.
In some embodiments, iteratively updating the dynamic adjustment scale at the determined step size based on the third maximum performance per second and the third request response latency, in relation to the magnitude of the maximum performance per second expectation and the request response latency expectation, until the third maximum performance per second and the third request response latency satisfy the maximum performance per second expectation and the request response latency expectation comprises:
in response to the third maximum performance per second being greater than the maximum performance expectation per second and the third request response latency being less than the request response latency expectation, superimposing a step magnitude for the dynamic adjustment scale to update the dynamic adjustment scale, and iteratively testing the memory system using the updated dynamic adjustment scale to update the third maximum performance per second and the third request response latency until the third maximum performance per second is less than the maximum performance expectation per second or the third request response latency is greater than the request response latency expectation to set the current dynamic adjustment scale minus the step magnitude to the iteratively adjusted dynamic adjustment scale;
in response to the third maximum performance per second being less than the maximum performance expectation per second or the third request response latency being greater than the request response latency expectation, the step magnitude is subtracted from the dynamic adjustment scale to update the dynamic adjustment scale, and the memory system is iteratively tested using the updated dynamic adjustment scale to update the third maximum performance per second and the third request response latency until the third maximum performance per second is greater than the maximum performance expectation per second and the third request response latency is less than the request response latency expectation to set the current dynamic adjustment scale to the iteratively adjusted dynamic adjustment scale.
In some embodiments, determining the number of processing skips based on the iteratively adjusted dynamic adjustment ratio and the deduplication processing cycle comprises: and determining the processing skipping number based on the product of the dynamic adjustment ratio after the iterative adjustment and the repeated data deleting processing circulation.
The following further illustrates embodiments of the invention in terms of specific examples.
The embodiment of the invention adjusts the ratio of the data writing request to execute the deduplication characteristic processing flow by increasing the data writing request judgment, thereby adjusting the influence of the deduplication characteristic on the system performance.
The regulation mode supports two kinds: the first is to adjust by "influence strength" parameters, and the second is to specify per second system performance (iops) or latency (latency) as an adjustment target.
The method for adjusting the influence strength of the first adjustment mode is to set an influence strength parameter AffectStrength with an effective value of 0 to MaxAffectStrength, and adjust the influence of the deduplication on the system performance according to the proportion of the AffectStrength converted into the adjustment write request for performing the deduplication processing. The specific implementation method comprises the following steps:
before the deduplication process is started, the write request is split according to the deduplication granularity (gransize), and the split write request (denoted as granwriterequest) includes a logical volume to be written (logical volume ID), an address in the logical volume to be written (start LBA, abbreviated LBA, and gransize alignment), and data. The write request GrantWriteRequest is counted as WriteReqCount, and each time a GrantWriteRequest is created, the WriteReqCount is incremented by 1. Skipping the deduplication process for the SkipDedupCount requests every ReqRegulaceCycle continuous requests, wherein:
(1) SkipDedupCount = reqregureclucatecycle, affectStrength/MaxAffectStrength, and rounding;
(2) Wherein ReqRegulatoreCycle is a period for counting and regulating write request deduplication processing, and the method for selecting and calculating ReqRegulatoreCycle is related to regulation period and system maximum performance.
For example, assuming MaxAffectStrength is 100, reqRegulateCycle can choose to adjust at a 0.1 second period, and system maximum iops is noted as MaxIops
ReqRegulateCycle＝0.1*MaxIops
SkipDedupCount＝[ReqRegulateCycle*AffectStrength/MaxAffectStrength]
Let MaxIops =200000, then
ReqRegulateCycle＝0.1*MaxIops＝20000
SkipDedupCount＝[20000*AffectStrength/100]＝[200x AffectStrength]
If AffectStrength =30 is set, the skipduductcount =6000, that is, the deduplication processing flow is skipped for 6000 write requests every 20000 write requests, specifically, 6000 requests may be skipped continuously, or 600 requests may be skipped continuously every 2000 requests.
And the dynamic regulation method of the second regulation mode is to specify the expected per-second system performance (marked as targetIops) or the time delay (marked as TargetDelay), and is realized by dynamically regulating the proportion of the write requests for executing the deduplication processing and detecting the difference between the actual per-second system performance and the targetIops or the difference between the actual time delay and the TargetDelay. The specific implementation method comprises the following steps:
(1) Setting the regulation ratio RegulatorRatio to be 0, namely skipping the deduplication processing flow by all the write requests, skipping the deduplication processing flow according to the regulation ratio, then measuring Iops and time Delay of the system, and recording Iops1 and Delay1.
(2) And setting the regulation ratio Regulatorratio to be 1, namely executing a deduplication processing flow for all the write requests, skipping the deduplication processing flow according to the regulation ratio, measuring Iops and time Delay of the system, and recording Iops2 and Delay2.
(3) If the Iops2 is larger than the targetIops or the Delay2 is smaller than the TargetDelay, the use adjustment proportion is 1;
(4) If the Iops1 is smaller than the targetIops or the Delay1 is larger than the TargetDelay, the use adjustment proportion is 0;
(5) And if the steps 3 and 4 are not true, setting the regulation ratio RegulatorRatio to be equal to (Iops 1-targetIops)/(Iops 1-Iops 2), skipping the re-puncturing processing flow according to the regulation ratio, and measuring the Iops and the time delay of the system.
(6) If iops is greater than or equal to TargetIops and delay is less than or equal to TargetDelay in step 5, decreasing the RegulateRatio by a step amplitude regulatestypte, and measuring iops and delay until iops is less than TargetIops or delay is greater than TargetDelay, adopting the previous regulateration as an adjustment ratio, wherein the step amplitude regulatestypte is a value between 0 and 1, and can be calculated according to the maximum adjustment times. For example, if the maximum adjustment number is set to 10, the regulated sequence is equal to (1-regulated ratio)/10.
(7) If iops is less than TargetIops or delay is greater than TargetDelay in step 5, the RegulateRatio is increased by the step size regulatestemp, and then iops and delay are measured until iops is greater than or equal to TargetIops and delay is less than or equal to TargetDelay, and then the current RegulateRatio is used as the adjusting proportion.
As can be seen from the foregoing embodiments, in the data writing method of the storage system according to the embodiments of the present invention, a data writing request is continuously received, and whether to perform a deduplication process on the data writing request is determined based on a predetermined performance impact strength or a dynamically adjusted performance difference; searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request in response to the execution flow, and establishing and saving mapping information of the logical block address to the physical block address mapping of the physical block address of the data writing request in response to the hit; the technical scheme of executing the data writing request to the storage system to write the data in the data writing request in response to not executing the flow or missing the existing physical block address mapping can reduce the processing resources occupied by the deduplication feature to alleviate or eliminate the performance impact on the storage system.
It should be particularly noted that, the steps in the embodiments of the data writing method of the storage system described above can be mutually intersected, replaced, added, and deleted, so that the data writing method of the storage system implemented by these reasonable permutation and combination transformations shall also belong to the scope of the present invention, and shall not limit the scope of the present invention to the described embodiments.
In view of the above, a second aspect of the embodiments of the present invention provides an embodiment of a data writing system of a storage system, which reduces processing resources occupied by a deduplication feature to mitigate or eliminate performance impact on the storage system. The system comprises:
a processor; and
a memory storing program code executable by the processor, the program code when executed performing the steps of:
continuously receiving data writing requests, and determining whether to execute a deduplication process on the data writing requests based on predetermined performance impact strengths or dynamically adjusted performance differences;
searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request in response to the execution flow, and establishing and saving mapping information of the logical block address to the physical block address mapping of the physical block address of the data writing request in response to the hit;
in response to not performing the flow or missing an existing physical block address mapping, a data write request is performed to the storage system to write the data in the data write request.
In some embodiments, determining whether to perform a deduplication process for a data write request based on a predetermined performance impact strength comprises:
checking the maximum performance per second, the conditioning time period, and the maximum performance impact strength of the storage system;
determining a deduplication processing cycle according to the maximum performance per second and the adjustment time period;
determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle;
specifying, among the received data write requests for each deduplication processing cycle, a number of data write requests corresponding to the number of processing skips so as not to execute the flow, and causing unspecified data write requests to execute the flow.
In some embodiments, determining whether to perform a deduplication flow for a data write request based on the dynamically adjusted performance difference comprises:
testing the maximum performance per second and the request response time delay of the storage system under different dynamic adjustment proportions to iteratively adjust the dynamic adjustment proportions until the maximum performance per second and the request response time delay meet the maximum performance expectation per second and the request response time delay expectation per second;
determining the number of processing skips based on the dynamic adjustment proportion and the repeated data deleting processing cycle after the iterative adjustment;
specifying a number of data write requests corresponding to the number of processing skips among the received data write requests per deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not specified to execute the flow.
As can be seen from the foregoing embodiments, the system provided in the embodiments of the present invention determines whether to perform a deduplication process on a data write request by continuously receiving the data write request and based on a predetermined performance impact strength or a dynamically adjusted performance difference; searching whether the existing physical block address mapping is hit or not in the storage system based on the metadata of the data writing request in response to the execution flow, and establishing and saving mapping information of the logical block address to the physical block address mapping of the physical block address of the data writing request in response to the hit; the technical scheme of executing the data writing request to the storage system to write the data in the data writing request in response to not executing the flow or missing the existing physical block address mapping can reduce the processing resources occupied by the deduplication feature to alleviate or eliminate the performance impact on the storage system.
It should be particularly noted that the above-mentioned embodiment of the system adopts an embodiment of the data writing method of the storage system to specifically describe the working process of each module, and those skilled in the art can easily think that these modules are applied to other embodiments of the data writing method of the storage system. Of course, since the steps in the data writing method embodiment of the storage system can be mutually intersected, replaced, added, and deleted, these reasonable permutation and combination transformations should also belong to the scope of the present invention, and should not limit the scope of the present invention to the embodiment.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of an embodiment of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (6)
1. A data writing method of a storage system, comprising performing the steps of:
continuously receiving data write requests and determining whether to perform a deduplication process for the data write requests based on a predetermined performance impact strength or a dynamically adjusted performance difference;
in response to executing the flow, searching whether an existing physical block address mapping is hit in the storage system based on metadata of the data write request, and in response to the hit, establishing and saving mapping information of a logical block address of the data write request to the physical block address mapping of the physical block address;
in response to not performing the flow or missing an existing physical block address mapping, performing the data write request to a storage system to write data in the data write request;
wherein determining whether to perform a deduplication process for the data write request based on a predetermined performance impact strength comprises: checking the maximum performance per second, the conditioning time period, and the maximum performance impact strength of the storage system; determining a deduplication processing cycle according to the maximum performance per second and the adjustment time period; determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle; designating a number of the data write requests corresponding to the processing-skipped number among the received data write requests per the deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not designated to execute the flow;
determining whether to perform a deduplication process for the data write request based on the dynamically adjusted performance difference comprises: testing the maximum performance per second and the request response delay of the storage system under different dynamic adjustment proportions to iteratively adjust the dynamic adjustment proportions until the maximum performance per second and the request response delay meet the maximum performance per second expectation and the request response delay expectation; determining the number of processing skips based on the dynamically adjusted proportion and the repeated data deleting processing cycle after the iterative adjustment; designating a number of the data write requests corresponding to the processing-skipped number among the received data write requests per the deduplication processing cycle so as not to execute the flow, and causing the data write requests not designated to execute the flow.
2. The method of claim 1, wherein determining a deduplication processing cycle based on the maximum performance per second and the adjustment time period comprises: determining the deduplication processing cycle according to the product of the maximum performance per second and the adjustment time period;
determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle comprises: determining the number of processing skips based on a product of the deduplication processing cycle and a quotient of the performance impact strength and the maximum performance impact strength.
3. The method of claim 1, wherein testing the maximum performance per second and request response latencies of the storage system at different dynamic scaling rates to iteratively adjust the dynamic scaling rates until the maximum performance per second and request response latencies meet a maximum performance per second and request response latency expectations comprises:
setting the dynamic adjustment proportion to be 0 to test the storage system, and obtaining a first maximum performance per second and a first request response time delay;
setting the dynamic adjustment proportion to be 1 to test the storage system, and obtaining a second maximum performance per second and a second request response time delay;
directly setting the iteratively adjusted dynamic adjustment ratio to 1 in response to the second maximum performance per second being greater than the maximum performance per second expectation or the second request response latency being less than the request response latency expectation;
in response to the first maximum performance per second being less than the maximum performance per second expectation or the first request response latency being greater than the request response latency expectation, directly setting the iteratively adjusted dynamic adjustment ratio to 0;
dynamically updating the dynamically adjusted scale to a ratio of a difference between the first maximum performance per second and the maximum performance expectation per second and a difference between the first maximum performance per second and the second maximum performance in response to the dynamically adjusted scale after the iterative adjustment not being directly set;
retesting the storage system based on the updated dynamic adjustment ratio to obtain a third maximum performance per second and a third request response delay;
iteratively updating the dynamic adjustment scale by a determined step size according to the third maximum performance per second and the third request response delay, and the magnitude relationship between the maximum performance per second expectation and the request response delay expectation until the third maximum performance per second and the third request response delay satisfy the maximum performance per second and the request response delay expectation.
4. The method of claim 3, wherein iteratively updating the dynamically adjusted scale at a determined step size in accordance with the third maximum performance per second and the third request response latency, in relation to the magnitude of the maximum performance per second expectation and the request response latency expectation, until the third maximum performance per second and the third request response latency satisfy the maximum performance per second and request response latency expectations comprises:
in response to the third maximum performance per second being greater than the maximum performance per second expectation and the third request response latency being less than the request response latency expectation, superimposing the step magnitude for the dynamic adjustment scale to update the dynamic adjustment scale and iteratively testing the memory system using the updated dynamic adjustment scale to update the third maximum performance per second and third request response latency until the third maximum performance per second is less than the maximum performance per second expectation or the third request response latency is greater than the request response latency expectation to set the current dynamic adjustment scale minus the step magnitude to the iteratively adjusted dynamic adjustment scale;
in response to the third maximum performance per second being less than the maximum performance per second expectation or the third request response latency being greater than the request response latency expectation, subtracting the step magnitude from the dynamic scaling factor to update the dynamic scaling factor, and iteratively testing the memory system using the updated dynamic scaling factor to update the third maximum performance per second and the third request response latency until the third maximum performance per second is greater than the maximum performance per second expectation and the third request response latency is less than the request response latency expectation to set the current dynamic scaling factor to the iteratively adjusted dynamic scaling factor.
5. The method of claim 1, wherein determining a number of processing skips based on the iteratively adjusted dynamic scaling and deduplication processing cycles comprises: determining the number of processing skips based on a product of the iteratively adjusted dynamic adjustment ratio multiplied by the deduplication processing cycle.
6. A data writing system of a storage system, comprising:
a processor; and
a memory storing program code executable by the processor, the program code when executed performing the steps of:
continuously receiving data write requests and determining whether to perform a deduplication process for the data write requests based on a predetermined performance impact strength or a dynamically adjusted performance difference;
in response to executing the flow, searching whether an existing physical block address mapping is hit in the storage system based on metadata of the data write request, and in response to the hit, establishing and saving mapping information of a logical block address of the data write request to the physical block address mapping of the physical block address;
in response to not performing the flow or missing an existing physical block address mapping, performing the data write request to a storage system to write data in the data write request;
wherein determining whether to perform a deduplication process for the data write request based on a predetermined performance impact strength comprises: checking the maximum performance per second, the adjustment time period and the maximum performance impact strength of the storage system; determining a deduplication processing cycle according to the maximum performance per second and the adjustment time period; determining a number of processing skips based on the performance impact strength, the maximum performance impact strength, and the deduplication processing cycle; designating a number of the data write requests corresponding to the processing-skipped number among the received data write requests per the deduplication processing cycle so as not to execute the flow, and causing the data write requests that are not designated to execute the flow;
determining whether to perform a deduplication process for the data write request based on the dynamically adjusted performance difference comprises: testing the maximum performance per second and the request response delay of the storage system under different dynamic adjustment proportions to iteratively adjust the dynamic adjustment proportions until the maximum performance per second and the request response delay meet the maximum performance per second expectation and the request response delay expectation; determining the number of processing skips based on the dynamically adjusted proportion and the repeated data deleting processing cycle after the iterative adjustment; designating a number of the data write requests corresponding to the processing-skipped number among the received data write requests per the deduplication processing cycle so as not to execute the flow, and causing the data write requests not designated to execute the flow.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110089916.9A CN112783446B (en) | 2021-01-22 | 2021-01-22 | Data writing method and system of storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110089916.9A CN112783446B (en) | 2021-01-22 | 2021-01-22 | Data writing method and system of storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112783446A CN112783446A (en) | 2021-05-11 |
| CN112783446B true CN112783446B (en) | 2023-01-10 |
Family
ID=75758638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110089916.9A Active CN112783446B (en) | 2021-01-22 | 2021-01-22 | Data writing method and system of storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783446B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982122A (en) * | 2012-11-13 | 2013-03-20 | 浪潮电子信息产业股份有限公司 | Repeating data deleting method suitable for mass storage system |
| CN108459826A (en) * | 2018-02-01 | 2018-08-28 | 杭州宏杉科技股份有限公司 | A kind of method and device of processing I/O Request |
-
2021
- 2021-01-22 CN CN202110089916.9A patent/CN112783446B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982122A (en) * | 2012-11-13 | 2013-03-20 | 浪潮电子信息产业股份有限公司 | Repeating data deleting method suitable for mass storage system |
| CN108459826A (en) * | 2018-02-01 | 2018-08-28 | 杭州宏杉科技股份有限公司 | A kind of method and device of processing I/O Request |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112783446A (en) | 2021-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102042859B1 (en) | Semiconductor device and operating method thereof | |
| CN107430551B (en) | Data caching method, storage control device and storage equipment | |
| CN113377291B (en) | Data processing method, device, equipment and medium of cache equipment | |
| CN112100293A (en) | Data processing method, data access method, data processing device, data access device and computer equipment | |
| US11762827B2 (en) | B-plus tree access method and apparatus, and computer-readable storage medium | |
| US11907164B2 (en) | File loading method and apparatus, electronic device, and storage medium | |
| CN109240607B (en) | File reading method and device | |
| CN110297743B (en) | Load testing method and device and storage medium | |
| CN111625515A (en) | Method and device for processing operation request of aggregated small files | |
| US11461239B2 (en) | Method and apparatus for buffering data blocks, computer device, and computer-readable storage medium | |
| US11327939B2 (en) | Method and device for indexing dirty data in storage system page | |
| CN111930307B (en) | Data reading method, device and equipment and computer readable storage medium | |
| CN109582649A (en) | A kind of metadata storing method, device, equipment and readable storage medium storing program for executing | |
| CN111752482A (en) | Method and system for writing data in full-flash distributed storage system | |
| CN107423425A (en) | A kind of data quick storage and querying method to K/V forms | |
| CN112783446B (en) | Data writing method and system of storage system | |
| CN116340198B (en) | Data writing method and device of solid state disk and solid state disk | |
| CN112084121B (en) | Hard disk pre-reading method and device, computer readable storage medium and electronic equipment | |
| CN110286858B (en) | Data processing method and related equipment | |
| CN111124943B (en) | Data processing method, device, equipment and storage medium | |
| CN102662866B (en) | File cache management method and memory management system | |
| CN113297106A (en) | Data replacement method based on hybrid storage, related method, device and system | |
| CN113568573B (en) | Data storage method, data storage device, storage medium and product | |
| CN117539407B (en) | Data storage method, electronic device, and computer-readable storage medium | |
| US11442658B1 (en) | System and method for selecting a write unit size for a block storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |