CN102096556B - Method for copying data as well as method, device and system for reading data - Google Patents
Method for copying data as well as method, device and system for reading data Download PDFInfo
- Publication number
- CN102096556B CN102096556B CN201010590713XA CN201010590713A CN102096556B CN 102096556 B CN102096556 B CN 102096556B CN 201010590713X A CN201010590713X A CN 201010590713XA CN 201010590713 A CN201010590713 A CN 201010590713A CN 102096556 B CN102096556 B CN 102096556B
- Authority
- CN
- China
- Prior art keywords
- buffer area
- rate
- data
- access
- disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0888—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches using selective caching, e.g. bypass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The embodiment of the invention provides a method for copying data as well as a method, device and system for reading the data. The method for copying the data comprises the steps: confirming a data copying speed of copying data to a cache region according to the access frequency of the cache region, the load of a magnetic disc and the load of the cache region when the access frequency of the cache region is not more than a high frequency threshold value; and copying the data from the magnetic disc to the cache region according to the copying speed. The method for reading the data comprises the steps: judging whether read request access data are memorized in the cache region after receiving the read request which is used for accessing the magnetic disc; and confirming to read the read request access data from the cache region or the magnetic disc according to the load factor of the magnetic disc and the access balance rate of the cache region when the read request access data are memorized in the cache region. The embodiment of the invention is good for improving the overall performance of a memory system.
Description
Technical field
The embodiment of the present invention relates to memory technology, relates in particular to a kind of method of copies data, method and apparatus and the system of reading out data.
Background technology
Solid state hard disc (Solid State Disk or Solid State Drive), also claim electronic hard disc or solid-state electronic dish, the hard disk that is formed by control module and solid-state storage unit, wherein the solid-state storage unit can be the storage chips such as dynamic RAM (Dynamic Random Access Memory, DRAM) or flash memory (FLASH).Solid state hard disc and common hard disc relatively have following advantage: start soon, do not have motor to accelerate the process of rotation; Without magnetic head, read at random fast, read latency is minimum; Relatively-stationary reading the time.Because addressing time and the data storage location of solid state hard disc are irrelevant, so disk fragments can not affect the time of reading.
In the storage system of SSD as buffer memory (cache), all access region are divided into several data blocks according to fixed size, each data block is provided with the count value of its cold and hot degree of sign, from disk to the SSD copies data time, according to cold and hot degree, all data blocks of disk are sorted, identify all hot spot data pieces, copy in SSD.When having read request to hit in SSD data, reading out data from SSD.Because the read latency of SSD is far smaller than traditional magnetic disk, thereby can improve system performance.Wherein, cold and hot degree is the parameter of the frequency that represents that data block is accessed, and more frequent accessed data are hotter, otherwise colder.
Yet, in existing SSD cache storage system, during data that access is stored in SSD, only read from SSD, therefore the performance of the whole storage system reading out data reading performance of SSD dish that places one's entire reliance upon; In addition, during to SSD copy hot spot data, do not consider opportunity and the copying speed of copies data.Therefore, existing storage system overall performance is lower.
Summary of the invention
The embodiment of the present invention provides a kind of method of copies data, method and apparatus and the system of reading out data, in order to improve the overall performance of storage system.
The embodiment of the present invention provides a kind of method of copies data, comprising:
When the access frequency of buffer area is no more than the high-frequency threshold value, according to the rate of load condensate of the access frequency of described buffer area, described disk and the rate of load condensate of described buffer area, determine the copying speed to described buffer area copies data;
According to described copying speed, from described disk to described buffer area copies data.
The embodiment of the present invention provides a kind of device of copies data, comprising: the frequency judge module is used for judging whether the access frequency of buffer area surpasses the high-frequency threshold value;
The First Speed determination module is used for when the access frequency of buffer area is no more than the high-frequency threshold value, according to the rate of load condensate of the access frequency of described buffer area, described disk and the rate of load condensate of described buffer area, determines the copying speed to described buffer area copies data;
The copy module is used for according to described copying speed, from described disk to described buffer area copies data.
The embodiment of the present invention provides a kind of method of reading out data, comprising:
After receiving the read request of access disk, judge the data that whether store described read request access in buffer area;
When storing the data of read request access in described buffer area, according to the rate of load condensate of described disk and the balanced rate of access of described buffer area, determine to read the data of described read request access from buffer area or disk.
The embodiment of the present invention provides a kind of device of reading out data, comprising: judge module and read module;
Described judge module after being used for receiving the read request of access disk, judges the data that whether store described read request access in buffer area;
Described read module is used for when described buffer area stores the data of read request access, according to the rate of load condensate of described disk and the balanced rate of access of described buffer area, determines to read the data of described read request access from buffer area or disk.
The embodiment of the present invention also provides a kind of storage system, comprising: disk and solid state hard disc, also comprise the device of above-mentioned copies data, and/or, the device of reading out data.
The method and apparatus of embodiment of the present invention copies data and system during from disk to the buffer area copies data, determine copy opportunity according to the access frequency in current cache district.And according to the access determination of power load copying speed of access load and the buffer area of disk.Thereby do not affect regular traffic when making to the buffer area copies data, be conducive to improve the overall performance of storage system.
The method and apparatus of embodiment of the present invention reading out data and system, when buffer area stores the data of read request access, according to the rate of load condensate of described disk and the balanced rate of access of described buffer area, determine to read the data of described read request access from disk or buffer area, make the access of system not exclusively depend on the performance of buffer area, reach the purpose of buffer area and disk access equilibrium, improved the overall performance of storage system.
Description of drawings
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, the below will do to introduce simply to the accompanying drawing of required use in embodiment or description of the Prior Art, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
Fig. 1 is embodiment of the method one process flow diagram of copies data provided by the invention;
Fig. 2 is embodiment of the method two process flow diagrams of copies data provided by the invention;
Fig. 3 is embodiment of the method three process flow diagrams of copies data provided by the invention;
Fig. 4 is embodiment of the method one process flow diagram of reading out data provided by the invention;
Fig. 5 is embodiment of the method two process flow diagrams of reading out data provided by the invention;
Fig. 6 is the structural representation of the device embodiment one of copies data provided by the invention;
Fig. 7 is the structural representation of the device embodiment two of copies data provided by the invention;
Fig. 8 is the structural representation of the device embodiment three of copies data provided by the invention;
Fig. 9 is the structural representation of the device embodiment one of reading out data provided by the invention;
Figure 10 is the structural representation of the device embodiment two of reading out data provided by the invention;
Figure 11 is a kind of storage system example structure schematic diagram provided by the invention;
Figure 12 is another kind of storage system example structure schematic diagram provided by the invention.
Embodiment
Further illustrate the technical scheme of the embodiment of the present invention below in conjunction with the drawings and specific embodiments, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that obtains under the creative work prerequisite.
Fig. 1 is embodiment of the method one process flow diagram of copies data provided by the invention, and the method for embodiment of the present invention copies data refers to from disk to the buffer area copy data.That is to say, after copies data, disk and buffer area are all preserved the data that are copied.As shown in Figure 1, the present embodiment comprises:
Step 11: when the access frequency of buffer area is no more than the high-frequency threshold value, according to the access frequency of buffer area, the rate of load condensate of disk and the rate of load condensate of buffer area, determine the copying speed to the buffer area copies data.
The access frequency of buffer area, IO (input and output) number in expression access cache district accounts for the ratio of total IO number, also referred to as the hit rate of buffer area.Total IO number is the summation of reading the IO number of buffer area and reading the IO number of disk.Before from disk (for example traditional mechanical disk) to the buffer area copies data, judge that whether the access frequency of buffer area is lower than the high-frequency threshold value.If, according to the access determination of power load copying speed of access load and the buffer area of disk, with according to this copying speed to the buffer area copies data.The access frequency of buffer area is no more than the high-frequency threshold value, and the hot spot data of expression buffer area is few, and utilization factor is not high, for the utilization factor that further improves buffer area can be from disk to buffer area copies data.
Step 12: according to copying speed, from disk to the buffer area copies data.
Further, as shown in Figure 2, when the access frequency of buffer area surpassed the high-frequency threshold value, the method for copies data was as follows:
Step 21: when the access frequency of buffer area surpasses the high-frequency threshold value, determine whether the rate of load condensate of disk and the rate of load condensate of buffer area all do not surpass the first rate of load condensate threshold values.
The access frequency of buffer area surpasses the high-frequency threshold value, and the expression buffer area has been stored more hot spot data, and the utilization factor of buffer area is higher, follow-up need to whether the decision from disk to the buffer area copies data according to the rate of load condensate of disk and buffer area.
Step 22: when the rate of load condensate of the rate of load condensate of disk and buffer area all surpasses the first rate of load condensate threshold values, determine copying speed to described buffer area copies data according to system's maximum bandwidth.
If the rate of load condensate of the rate of load condensate of disk and buffer area wherein any one higher than the first rate of load condensate threshold values, shows that buffer area and disk all are in busy state, do not carry out the operation of copies data for not affecting system performance.Rate of load condensate represents that namely the IO of disk or buffer area processes busy extent, and rate of load condensate is higher, and the processing of expression disc is busier.
The method of the present embodiment copies data during from disk to the buffer area copies data, is determined copy opportunity according to the access frequency in current cache district.And determine copying speed according to the access rate of load condensate of disk and the access rate of load condensate of buffer area.Thereby do not affect regular traffic when making to the buffer area copies data, be conducive to improve the overall performance of storage system.
Fig. 3 is embodiment of the method three process flow diagrams of copies data provided by the invention, and in the present embodiment, the storage medium of buffer area can be SSD, and disk can be the conventional disc drivers such as HDD (Hard Disk Drive).As shown in Figure 3, the present embodiment comprises:
Step 31: user's scene is set so that different maximum copying speed M, the maximum bandwidth N of system to be set.
Step 32: whether the access frequency that judges SSD surpasses the high-frequency threshold value.If do not surpass execution in step 33, if surpass execution in step 36.
Step 33; Whether the access frequency that judges SSD surpasses the low frequency threshold value.If surpass execution in step 34, if do not surpass execution in step 35.
The access frequency of SSD does not surpass the high-frequency threshold value, and the hot spot data of expression buffer area is few, can be from HDD to the SDD copies data.But when the access frequency of SSD surpassed the low frequency threshold value, the hot spot data of storage that shows SDD was very few and close to idle condition, copied hot spot data for the effect of performance SDD needing in time from disk to buffer area.When the access frequency of SSD surpasses the low frequency threshold value but surpasses the high-frequency threshold values, the processing that shows SDD has been in certain busy condition, speed from from HDD to the SDD copies data need be considered the load of SDD and the load of HDD, thereby avoids the impact on the entire system performance.
Step 34; When the access frequency of SSD surpassed the low frequency threshold value, the copying speed when determining from HDD to the SSD copies data was: M * (1-β), and copies data.
Wherein, β is the mxm. in the maximum load rate of the maximum load rate of HDD and SDD, the maximum copying speed of M for allowing.During copies data, service feature being had certain impact, with less than the speed copies data of M the time, is acceptable on the impact of service feature.
Step 35: when the access frequency of SSD did not surpass the low frequency threshold value, the copying speed when determining from HDD to the SSD copies data was: 10% * N, and copies data.
Wherein, N is system's maximum bandwidth, and little when machine-readable, N is bandwidth corresponding to maximum IOPS; Bulk is when machine-readable, and N is maximum bandwidth.IOPS (I/Os per second): i.e. per second input and output number of times.The IO frequency of the system that refers to treatable maximum within the unit interval.
Step 36: when the access frequency of SSD surpasses the high-frequency threshold value, judge whether the rate of load condensate of HDD and the rate of load condensate of SDD all do not surpass the first rate of load condensate threshold values.If execution in step 37, otherwise execution in step 38.
The first rate of load condensate threshold values can be 95%.When the access frequency of SDD surpasses the high-frequency threshold values, if wherein any one more than or equal to 95%, is considered as system busy, not copies data to the first rate of load condensate of the rate of load condensate of HDD and SDD.
Step 37: when the rate of load condensate of the rate of load condensate of HDD and SDD all surpassed the first rate of load condensate threshold values, the copying speed when determining from HDD to the SSD copies data was: 10% * N, and copies data.N value herein is identical with N value in step 35.
When the access frequency of SDD surpasses the high-frequency threshold values, if the rate of load condensate of the rate of load condensate of HDD and SDD does not all surpass the first rate of load condensate threshold values, but show the copy of HDD and SDD deal with data, determine that copying speed is 10% * N.
Step 38; When the rate of load condensate that surpasses the first rate of load condensate threshold values or SDD in the rate of load condensate of HDD overloads the rate threshold values, copies data not.
The method of the present embodiment copies data, according to the different user scene setting different maximum copying speed M and the maximum bandwidth N of system.When the access frequency of SDD surpasses the high-frequency threshold values, and the rate of load condensate of the rate of load condensate of HDD and SDD is when all surpassing the first rate of load condensate threshold values, to the buffer area copies data, otherwise copies data not.In addition, when the access frequency of SDD does not surpass the high-frequency threshold values and surpasses the low frequency threshold values, the speed when determining to the SDD copies data according to system's maximum bandwidth.When the access frequency of SDD does not surpass the low frequency threshold values, the speed when determining to the SDD copies data according to the maximum copying speed that allows.Access speed when dynamically adjusting copies data due to the present invention according to the access frequency of SSD does not affect other business of system, and has improved the utilization factor of SSD Cache.
Fig. 4 is embodiment of the method one process flow diagram of reading out data provided by the invention, and as shown in Figure 4, the present embodiment comprises:
Step 41: after receiving read request, judge the data that whether store described read request access in buffer area.
Step 42: when storing the data of read request access in buffer area, according to the rate of load condensate of disk and the balanced rate of access of buffer area, determine to read the data of described read request access from buffer area or disk.
Can not only store hot spot data in SSD, when copies data in SDD, need to suitably one piece of data not copied in SDD according to actual read number according to business characteristic, system like this both can read from SSD during reading out data, can read from HDD again.
The method of the present embodiment reading out data, when buffer area stores the data of read request access, according to the rate of load condensate of described mechanical hard disk and the balanced rate of access of described buffer area, determine to read the data of described read request access from disk or buffer area, make the access of system not exclusively depend on the performance of buffer area, reach the purpose of buffer area and disk access equilibrium, improved the overall performance of storage system.
Fig. 5 is embodiment of the method two process flow diagrams of reading out data provided by the invention, and in the present embodiment, the storage medium of buffer area is SSD, and disk is HDD.As shown in Figure 5, the present embodiment comprises:
Step 51: user's scene is set so that the balanced rate of different access, the second rate of load condensate threshold value and balanced rate threshold value to be set.
Step 52: whether the data of read request access are in SSD.In the time of not in SSD, execution in step 53, otherwise execution in step 54.
Step 53: reading out data from HDD.
The data of read request access not in SSD, reading out data from HDD.
Step 54: the data of read request access judge whether the HDD rate of load condensate reaches the second rate of load condensate threshold value in SSD.If reach execution in step 55, otherwise execution in step 56.
Step 55: the data that read the read request access from SSD.
The HDD rate of load condensate reaches the second rate of load condensate threshold value, and the visit capacity of expression HDD is larger, has reached the processing limit of HDD.When the HDD rate of load condensate reaches the second rate of load condensate threshold value, read the data of read request access from SSD.Can be by the busy extent of HDD, for example the IO of HDD postpones situation, determines the second rate of load condensate threshold values.
When step 56:HDD rate of load condensate does not reach the second rate of load condensate threshold value, judge whether the balanced rate of access of SSD reaches balanced rate threshold value.Reach balanced rate threshold value, execution in step 53; Otherwise execution in step 55.
Whether the balanced rate of the access ratio of the data volume that reads from HDD (data volume that reads from SSD with) that judges SSD reaches balanced rate threshold value.Quantity, the performance of each SSD and the performance setting of HDD of the SSD that balanced rate threshold value comprises according to SSD Cache.For example, there are m SSD and n HDD in system, and the performance of each SSD is P, and the performance of each HDD is Q.The SSD group total load that can support is m * P so, and it is n * Q that HDD organizes the total load that can support.Balanced rate threshold value is exactly the ratio of m * P and n * Q.
When the balanced rate of the access of SSD reached balanced rate threshold value, the visit capacity of expression SDD was larger, had reached the processing limit of HDD, read the data of read request access from HDD.When the balanced rate of the access of SSD does not reach balanced rate threshold value, read the data of read request access from SSD.
Different business scenarios needs different access balance policies.For SSD, owing to having short access delay characteristic, make it much higher with machine-readable Performance Ratio traditional magnetic disk, but its bandwidth and traditional magnetic disk to compare advantage little.So take with machine-readable during as main business, as internet, applications, SSD access large percentage, and when bulk was read to be the business of leading, as the IPTV business, HDD accessed large percentage.
The method of the present embodiment reading out data when storing the data of read request access in SDD, is HDD when having reached processing limit if the rate of load condensate of HDD has reached capacity, and direct reading out data from SDD is to alleviate the access load of HDD; If the rate of load condensate of HDD does not reach capacity, and the balanced rate of the access of the SDD visit capacity that to surpass balanced rate threshold values be SDD is when too much, and reading out data from HDD is to alleviate the access load of SDD; If the rate of load condensate of HDD does not reach capacity, and the balanced rate of the access of the SDD visit capacity that also to surpass balanced rate threshold values be SDD is when less, and reading out data from SDD is to improve access speed.Therefore, make SDD and HDD to be utilized simultaneously, reached the balanced purpose of access, maximize the overall performance of having brought into play storage system.
Fig. 6 is the structural representation of the device embodiment one of copies data provided by the invention.As shown in Figure 6, the present embodiment comprises: First Speed determination module 61 and copy module 62 and frequency judge module 63.
First Speed determination module 61, be used for when frequency judge module 63 determines that the access frequency of buffer area is no more than the high-frequency threshold value, according to the access frequency of buffer area, the rate of load condensate of disk and the rate of load condensate of buffer area, determine the copying speed to the buffer area copies data;
Further, as shown in Figure 7, the device of copies data also comprises: rate of load condensate judge module 64 and second speed determination module 65.
Rate of load condensate judge module 64 is used for when frequency judge module 63 determines that the access frequency of buffer area surpasses the high-frequency threshold value, judges whether the rate of load condensate of disk and the rate of load condensate of buffer area all do not surpass the first rate of load condensate threshold values;
Second speed determination module 65 is used for when rate of load condensate judge module 64 determines that the rate of load condensate of the rate of load condensate of disks and buffer area all surpasses the first rate of load condensate threshold values, determines copying speed to the buffer area copies data according to system's maximum bandwidth.
The working mechanism of above-mentioned each module does not repeat them here referring to describing in Fig. 1 and the corresponding embodiment of Fig. 2.
The device of the present embodiment copies data during from disk to the buffer area copies data, is determined copy opportunity according to the access frequency in current cache district.And according to the access determination of power load copying speed of access load and the buffer area of disk.Thereby do not affect regular traffic when making to the buffer area copies data, be conducive to improve the overall performance of storage system.
Fig. 8 is the structural representation of the device embodiment three of copies data provided by the invention.As shown in Figure 8, First Speed determination module 61 comprises: First Speed determining unit 611 and second speed determining unit 612.
First Speed determining unit 611, be used for when frequency judge module 63 determines that the access frequency of buffer area surpasses the low frequency threshold value, copying speed maximum according to maximal value and system in the maximum load rate of the maximum load rate of disk and buffer area determined the copying speed to the buffer area copies data.
Second speed determining unit 612 is used for when frequency judge module 63 determines that the access frequency of buffer area is no more than copying speed from the low frequency threshold value to buffer area copies data definite according to system's maximum bandwidth.
The working mechanism of above-mentioned each module does not repeat them here referring to describing in the corresponding embodiment of Fig. 3.
The device of the present embodiment copies data, when the access frequency of buffer area surpasses the high-frequency threshold values, and the rate of load condensate of the rate of load condensate of disk and buffer area is when all surpassing the first rate of load condensate threshold values, to the buffer area copying speed, otherwise copies data not.In addition, when the access frequency of buffer area does not surpass the high-frequency threshold values and surpasses the low frequency threshold values, the speed when determining to the buffer area copies data according to system's maximum bandwidth.When the access frequency of buffer area does not surpass the low frequency threshold values, the speed when determining to the buffer area copies data according to the maximum copying speed that allows.Access speed when dynamically adjusting copies data due to the embodiment of the present invention according to the access frequency of SSD does not affect other business of system, and has improved the utilization factor of SSD Cache.
Fig. 9 is the structural representation of the device embodiment one of reading out data provided by the invention.As shown in Figure 9, the present embodiment comprises: judge module 91 and read module 92.
Read module 92 is used for when judge module 91 determines that buffer area stores the data of read request access, according to the rate of load condensate of disk and the balanced rate of access of buffer area, determines to read the data of read request access from buffer area or disk.
The working mechanism of above-mentioned each module does not repeat them here referring to describing in the corresponding embodiment of Fig. 4.
The device of the present embodiment reading out data, when buffer area stores the data of read request access, according to the rate of load condensate of mechanical hard disk and the balanced rate of access of buffer area, determine to read the data of read request access from disk or buffer area, make the access of system not exclusively depend on the performance of buffer area, reach the purpose of buffer area and disk access equilibrium, improved the overall performance of storage system.
Figure 10 is the structural representation of the device embodiment two of reading out data provided by the invention.As shown in figure 10, read module 92 comprises: the first judging unit 921, reading unit 922 and the second judging unit 923.
The first judging unit 921 is used for judging whether the rate of load condensate of disk reaches the second rate of load condensate threshold value.
Reading unit 922 is used for reading the data of read request access from buffer area when the first judging unit 921 determines that the rate of load condensate of disk reaches the second rate of load condensate threshold value.
The second judging unit 923 is used for when the first judging unit 921 determines that the rate of load condensate of disk does not reach the second rate of load condensate threshold value, judges whether the balanced rate of access of buffer area reaches balanced rate threshold value; The balanced rate of the access of buffer area is the number percent of the data volume that reads and the data volume that reads from disk from buffer area; Quantity, the performance of each solid state hard disc and the performance setting of disk of the solid state hard disc that balanced rate threshold value comprises according to buffer area;
Reading unit 922 when also determining that for the second judging unit 923 the balanced rate of access of buffer areas does not reach balanced rate threshold value, reads the data of read request access from buffer area;
Reading unit 922 also is used for the second judging unit 923 and determines that the balanced rate of access of buffer area reaches balanced rate threshold value, reads the data of read request access from disk.
The working mechanism of above-mentioned each module does not repeat them here referring to describing in the corresponding embodiment of Fig. 5.
The device of the present embodiment reading out data when storing the data of read request access in buffer area, is disk when having reached processing limit if the rate of load condensate of disk has reached capacity, directly reading out data from buffer area; If the rate of load condensate of disk does not reach capacity, and the balanced rate of the access of the buffer area visit capacity that to surpass balanced rate threshold values be buffer area is when too much, reading out data from disk; If the rate of load condensate of disk does not reach capacity, and the balanced rate of the access of the buffer area visit capacity that also to surpass balanced rate threshold values be buffer area is when less, reading out data from buffer area.Therefore, make buffer area and disk to be utilized simultaneously, reached the balanced purpose of access, maximize the overall performance of having brought into play storage system.
Figure 11 is a kind of storage system example structure schematic diagram provided by the invention.As shown in figure 11.The present embodiment storage system comprises: disk 111 and solid state hard disc 112 also comprise the device 113 of copies data.Wherein disk 111 can be traditional mechanical hard disk, and the device 113 of copies data can be the device of the copies data of describing in the corresponding embodiment of Fig. 6 to Fig. 8, and solid state hard disc 112 is a kind of medium of the buffer area described in the corresponding embodiment of Fig. 6 to Fig. 8.In addition, the present embodiment storage system also can comprise the device of the reading out data of describing in the corresponding embodiment of Fig. 9 to Figure 10.
Figure 12 is another kind of storage system example structure schematic diagram provided by the invention.As shown in figure 12.The present embodiment storage system comprises: disk 121 and solid state hard disc 122 also comprise the device 123 of reading out data.Wherein disk 121 can be traditional mechanical hard disk, and the device 123 of reading out data can be the device of the reading out data of describing in the corresponding embodiment of Fig. 9 to Figure 10, and solid state hard disc 122 is a kind of medium of the buffer area described in the corresponding embodiment of Fig. 6 to Fig. 8.In addition, the present embodiment storage system also can comprise the device of the copies data of describing in the corresponding embodiment of Fig. 6 to Fig. 8.
One of ordinary skill in the art will appreciate that: all or part of step that realizes said method embodiment can be completed by the hardware that programmed instruction is correlated with, aforesaid program can be stored in a computer read/write memory medium, this program is carried out the step that comprises said method embodiment when carrying out; And aforesaid storage medium comprises: the various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
It should be noted that at last: above embodiment only in order to technical scheme of the present invention to be described, is not intended to limit; Although with reference to previous embodiment, the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: it still can be modified to the technical scheme that aforementioned each embodiment puts down in writing, and perhaps part technical characterictic wherein is equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution break away from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (4)
1. the method for a copies data, is characterized in that, comprising:
When the access frequency of buffer area is no more than the high-frequency threshold value but surpasses the low frequency threshold value, copying speed maximum according to maximal value and system in the maximum load rate of the maximum load rate of disk and described buffer area determined the copying speed to described buffer area copies data;
When the access frequency of described buffer area is no more than described low frequency threshold value, copying speed to described buffer area copies data definite according to system's maximum bandwidth;
When the access frequency of buffer area surpasses described high-frequency threshold value, judge whether the rate of load condensate of described disk and the rate of load condensate of described buffer area all do not surpass the first rate of load condensate threshold values;
When the rate of load condensate of the rate of load condensate of disk and buffer area all surpasses the first rate of load condensate threshold values, determine copying speed to described buffer area copies data according to system's maximum bandwidth;
According to described copying speed, from described disk to described buffer area copies data.
2. the device of a copies data, is characterized in that, comprising:
The frequency judge module is used for judging whether the access frequency of buffer area surpasses the high-frequency threshold value;
The First Speed determination module,
Described First Speed determination module comprises:
The First Speed determining unit, be used for that access frequency at described buffer area is no more than described high-frequency threshold value but when surpassing the low frequency threshold value, copying speed maximum according to maximal value and system in the maximum load rate of the maximum load rate of disk and described buffer area determined the copying speed to described buffer area copies data;
The second speed determining unit, concrete being used for when the access frequency of described buffer area is no more than described low frequency threshold value, copying speed to described buffer area copies data definite according to system's maximum bandwidth;
The rate of load condensate judge module is used for when the access frequency of buffer area surpasses the high-frequency threshold value, determines whether the rate of load condensate of disk and the rate of load condensate of buffer area all do not surpass the first rate of load condensate threshold values;
The second speed determination module is used for when the rate of load condensate of the rate of load condensate of disk and buffer area all surpasses the first rate of load condensate threshold values, determines copying speed to described buffer area copies data according to system's maximum bandwidth;
The copy module is used for according to described copying speed, from described disk to described buffer area copies data.
3. the method for a reading out data, is characterized in that, comprising:
After receiving read request, judge the data that whether store described read request access in buffer area;
When storing the data of read request access in described buffer area, according to the rate of load condensate of disk and the balanced rate of access of described buffer area, determine to read the data of described read request access from buffer area or disk; When the rate of load condensate of described disk reaches the second rate of load condensate threshold value, read the data of described read request access from described buffer area;
When the rate of load condensate of described disk does not reach described the second rate of load condensate threshold value, judge whether the balanced rate of access of described buffer area reaches balanced rate threshold value; The balanced rate of the access of described buffer area is the number percent of the data volume that reads and the data volume that reads from described disk from buffer area; Quantity, the performance of each described solid state hard disc and the performance setting of described disk of the solid state hard disc that described balanced rate threshold value comprises according to described buffer area;
If the balanced rate of the access of described buffer area does not reach balanced rate threshold value, read the data of described read request access from described buffer area;
If the balanced rate of the access of described buffer area reaches balanced rate threshold value, read the data of described read request access from described disk.
4. the device of a reading out data, is characterized in that, comprising:
Judge module after being used for receiving read request, judges the data that whether store described read request access in buffer area;
Read module is used for when described buffer area stores the data of read request access, according to the rate of load condensate of disk and the balanced rate of access of described buffer area, determines to read the data of described read request access from described buffer area or described disk; The first judging unit is used for judging whether the rate of load condensate of described disk reaches the second rate of load condensate threshold value;
Reading unit is used for when the rate of load condensate of described disk reaches described the second rate of load condensate threshold value, reads the data of described read request access from described buffer area;
The second judging unit is used for when the rate of load condensate of described disk does not reach described the second rate of load condensate threshold value, judges whether the balanced rate of access of described buffer area reaches balanced rate threshold value; The balanced rate of the access of described buffer area is the number percent of the data volume that reads and the data volume that reads from described disk from buffer area; Quantity, the performance of each described solid state hard disc and the performance setting of described disk of the solid state hard disc that described balanced rate threshold value comprises according to described buffer area;
Described reading unit when also not reaching balanced rate threshold value for the balanced rate of the access of described buffer area, reads the data of described read request access from described buffer area;
Described reading unit, the balanced rate of access that also is used for described buffer area reaches described balanced rate threshold value, reads the data of described read request access from described disk.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010590713XA CN102096556B (en) | 2010-12-03 | 2010-12-03 | Method for copying data as well as method, device and system for reading data |
| PCT/CN2011/080788 WO2012071950A1 (en) | 2010-12-03 | 2011-10-14 | Method for copying data, and method, device and system for reading data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010590713XA CN102096556B (en) | 2010-12-03 | 2010-12-03 | Method for copying data as well as method, device and system for reading data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102096556A CN102096556A (en) | 2011-06-15 |
| CN102096556B true CN102096556B (en) | 2013-11-06 |
Family
ID=44129666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010590713XA Active CN102096556B (en) | 2010-12-03 | 2010-12-03 | Method for copying data as well as method, device and system for reading data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102096556B (en) |
| WO (1) | WO2012071950A1 (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096556B (en) * | 2010-12-03 | 2013-11-06 | 华为数字技术（成都）有限公司 | Method for copying data as well as method, device and system for reading data |
| CN103294407B (en) * | 2012-03-05 | 2017-07-25 | 北京联想核芯科技有限公司 | Storage device and data read-write method |
| CN105653524B (en) * | 2014-11-10 | 2021-07-30 | 创新先进技术有限公司 | Data storage method, device and system |
| CN105487811A (en) * | 2015-02-09 | 2016-04-13 | 哈尔滨安天科技股份有限公司 | File reading and writing method and system based on buffering alignment |
| CN105068768B (en) * | 2015-08-26 | 2020-06-26 | 华为技术有限公司 | Write strategy control method and device |
| CN106775436B (en) * | 2015-11-24 | 2019-10-25 | 群联电子股份有限公司 | Data access method, memorizer control circuit unit and memory |
| CN105893118B (en) * | 2016-03-30 | 2019-11-26 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN106155931B (en) * | 2016-06-30 | 2020-07-07 | 百度在线网络技术（北京）有限公司 | Method and device for limiting speed of disk reading |
| CN108874324B (en) * | 2018-07-17 | 2021-08-31 | 郑州云海信息技术有限公司 | Access request processing method, device, equipment and readable storage medium |
| CN110737393B (en) * | 2018-07-20 | 2023-08-11 | 伊姆西Ip控股有限责任公司 | Data reading method, apparatus and computer program product |
| CN110119251B (en) * | 2019-05-14 | 2023-01-03 | 北京杰思安全科技有限公司 | Buffer control method and system based on disk large flow |
| CN112540933A (en) * | 2020-11-26 | 2021-03-23 | 华云数据控股集团有限公司 | Cache reading and writing method and device and electronic equipment |
| CN112597104B (en) * | 2021-01-11 | 2023-07-04 | 武汉飞骥永泰科技有限公司 | Small file performance optimization method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1924834A (en) * | 2006-09-08 | 2007-03-07 | 华为技术有限公司 | High-speed memory pre-read method and device |
| CN101135994A (en) * | 2007-09-07 | 2008-03-05 | 杭州华三通信技术有限公司 | Method and apparatus for dividing cache space and cache controller thereof |
| CN101493795A (en) * | 2008-01-24 | 2009-07-29 | 杭州华三通信技术有限公司 | Storage system, storage controller, and cache implementing method in the storage system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7120759B2 (en) * | 2003-08-29 | 2006-10-10 | International Business Machines Corporation | Storage system and method for prestaging data in a cache for improved performance |
| CN101645928B (en) * | 2009-08-26 | 2012-07-25 | 成都市华为赛门铁克科技有限公司 | Content resource caching method, device and system |
| CN101668046B (en) * | 2009-10-13 | 2012-12-19 | 成都市华为赛门铁克科技有限公司 | Resource caching method, device and system thereof |
| CN101819509A (en) * | 2010-04-19 | 2010-09-01 | 清华大学深圳研究生院 | Solid state disk read-write method |
| CN102096556B (en) * | 2010-12-03 | 2013-11-06 | 华为数字技术（成都）有限公司 | Method for copying data as well as method, device and system for reading data |
-
2010
- 2010-12-03 CN CN201010590713XA patent/CN102096556B/en active Active
-
2011
- 2011-10-14 WO PCT/CN2011/080788 patent/WO2012071950A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1924834A (en) * | 2006-09-08 | 2007-03-07 | 华为技术有限公司 | High-speed memory pre-read method and device |
| CN101135994A (en) * | 2007-09-07 | 2008-03-05 | 杭州华三通信技术有限公司 | Method and apparatus for dividing cache space and cache controller thereof |
| CN101493795A (en) * | 2008-01-24 | 2009-07-29 | 杭州华三通信技术有限公司 | Storage system, storage controller, and cache implementing method in the storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2012071950A1 (en) | 2012-06-07 |
| CN102096556A (en) | 2011-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102096556B (en) | Method for copying data as well as method, device and system for reading data | |
| CN102609360B (en) | Data processing method, data processing device and data processing system | |
| US9229653B2 (en) | Write spike performance enhancement in hybrid storage systems | |
| TWI416323B (en) | Method,system and semiconductor device for management workload | |
| US8595451B2 (en) | Managing a storage cache utilizing externally assigned cache priority tags | |
| US9053019B2 (en) | Non-volatile memory device, a data processing device using the same, and a swapping method used by the data processing and non-volatile memory devices | |
| CN106547476B (en) | Method and apparatus for data storage system | |
| KR101663066B1 (en) | Solid state memory command queue in hybrid device | |
| CN110032333B (en) | Memory system and method of operating the same | |
| EP2397945A1 (en) | Programming method and device for a buffer cache in a solid-state disk system | |
| US20090172264A1 (en) | System and method of integrating data accessing commands | |
| KR20130112210A (en) | Page replace method and memory system using the same | |
| CN102981783A (en) | Cache accelerating method based on Nand Flash | |
| US8856439B2 (en) | Method and device for utilizing application-level prior knowledge for selectively storing data in higher performance media | |
| CN102203749A (en) | Writing method and device of solid state driver under multi-level cache | |
| US20220391089A1 (en) | Dissimilar Write Prioritization in ZNS Devices | |
| KR20170002866A (en) | Adaptive Cache Management Method according to the Access Chracteristics of the User Application in a Distributed Environment | |
| CN104252422A (en) | Memory access method and memory controller | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| CN103714010A (en) | Storage device write-in method and storage device | |
| US11061598B2 (en) | Optimized handling of multiple copies in storage management | |
| CN103530236A (en) | Method and device for obtaining hybrid hard disk | |
| CN117472285A (en) | Intelligent operation acceleration method for solid state disk, computer equipment and storage medium | |
| CN112347044A (en) | Object storage optimization method based on SPDK | |
| US20170269834A1 (en) | Data storage device and data management method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Applicant after: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. Address before: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Applicant before: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES Co.,Ltd. |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES CO., LTD. TO: HUAWEI DIGITAL TECHNOLOGY (CHENGDU) CO., LTD. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220829 Address after: No. 1899 Xiyuan Avenue, high tech Zone (West District), Chengdu, Sichuan 610041 Patentee after: Chengdu Huawei Technologies Co.,Ltd. Address before: 611731 Qingshui River District, Chengdu hi tech Zone, Sichuan, China Patentee before: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. |