CN102096556A - Method for copying data as well as method, device and system for reading data - Google Patents
Method for copying data as well as method, device and system for reading data Download PDFInfo
- Publication number
- CN102096556A CN102096556A CN201010590713XA CN201010590713A CN102096556A CN 102096556 A CN102096556 A CN 102096556A CN 201010590713X A CN201010590713X A CN 201010590713XA CN 201010590713 A CN201010590713 A CN 201010590713A CN 102096556 A CN102096556 A CN 102096556A
- Authority
- CN
- China
- Prior art keywords
- buffer area
- rate
- data
- disk
- load condensate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0888—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches using selective caching, e.g. bypass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The embodiment of the invention provides a method for copying data as well as a method, device and system for reading the data. The method for copying the data comprises the steps: confirming a data copying speed of copying data to a cache region according to the access frequency of the cache region, the load of a magnetic disc and the load of the cache region when the access frequency of the cache region is not more than a high frequency threshold value; and copying the data from the magnetic disc to the cache region according to the copying speed. The method for reading the data comprises the steps: judging whether read request access data are memorized in the cache region after receiving the read request which is used for accessing the magnetic disc; and confirming to read the read request access data from the cache region or the magnetic disc according to the load factor of the magnetic disc and the access balance rate of the cache region when the read request access data are memorized in the cache region. The embodiment of the invention is good for improving the overall performance of a memory system.
Description
Technical field
The embodiment of the invention relates to memory technology, relates in particular to a kind of method of copies data, the method and apparatus and the system of reading of data.
Background technology
Solid state hard disc (Solid State Disk or Solid State Drive), also claim electronic hard disc or solid-state electronic dish, the hard disk of forming by control module and solid-state storage unit, wherein the solid-state storage unit can be dynamic RAM (Dynamic Random Access Memory, DRAM) or flash memory storage chips such as (FLASH).Solid state hard disc and common hard disk relatively have following advantage: start soon, do not have motor to quicken the process of rotation; Without magnetic head, read at random fast, read latency is minimum; Relatively-stationary time for reading.Because the addressing time of solid state hard disc and data storage location are irrelevant, so disk fragments can not influence time for reading.
In the storage system of SSD as buffer memory (cache), the all-access zone is divided into several data blocks according to fixed size, each data block is provided with the count value of its cold and hot degree of sign, from disk during to the SSD copies data, according to cold and hot degree all data blocks of disk are sorted, identify all hot spot data pieces, copy among the SSD.When read request is hit among the SSD data, reading of data from SSD.Because the read latency of SSD is far smaller than traditional magnetic disk, thereby can improve system performance.Wherein, cold and hot degree is the parameter of the frequency that is used for representing that data block is accessed, and frequent more accessed data are hot more, otherwise cold more.
Yet, in the existing SSD cache storage system, during data that visit is stored among the SSD, only from SSD, read, therefore the performance of the whole storage system reading of data reading performance of SSD dish that places one's entire reliance upon; In addition, when SSD copy hot spot data, do not consider the opportunity and the copying speed of copies data.Therefore, existing storage system overall performance is lower.
Summary of the invention
The embodiment of the invention provides a kind of method of copies data, the method and apparatus and the system of reading of data, in order to improve the overall performance of storage system.
The embodiment of the invention provides a kind of method of copies data, comprising:
When the access frequency of buffer area is no more than the high-frequency threshold value,, determine copying speed to described buffer area copies data according to the rate of load condensate of the access frequency of described buffer area, described disk and the rate of load condensate of described buffer area;
According to described copying speed, from described disk to described buffer area copies data.
The embodiment of the invention provides a kind of device of copies data, comprising: the frequency judge module is used to judge whether the access frequency of buffer area surpasses the high-frequency threshold value;
The first speed determination module is used for when the access frequency of buffer area is no more than the high-frequency threshold value, according to the rate of load condensate of the access frequency of described buffer area, described disk and the rate of load condensate of described buffer area, determines the copying speed to described buffer area copies data;
The copy module, be used for according to described copying speed, from described disk to described buffer area copies data.
The embodiment of the invention provides a kind of method of reading of data, comprising:
After receiving the read request of visit disk, judge the data that whether store described read request visit in the buffer area;
When in described buffer area, storing the data of read request visit,, determine from buffer area or disk, to read the data of described read request visit according to the rate of load condensate of described disk and the balanced rate of visit of described buffer area.
The embodiment of the invention provides a kind of device of reading of data, comprising: judge module and read module;
Described judge module after being used to receive the read request of visit disk, is judged the data that whether store described read request visit in the buffer area;
Described read module is used for when described buffer area stores the data of read request visit, according to the rate of load condensate of described disk and the balanced rate of visit of described buffer area, determines to read the data of described read request visit from buffer area or disk.
The embodiment of the invention also provides a kind of storage system, comprising: disk and solid state hard disc, also comprise the device of above-mentioned copies data, and/or, the device of reading of data.
The method and apparatus of embodiment of the invention copies data and system when the buffer area copies data, determine copy opportunity according to the access frequency in current cache district from disk.And determine copying speed according to the visit load of the visit load of disk and buffer area.Thereby making does not influence regular traffic when the buffer area copies data, help improving the overall performance of storage system.
The method and apparatus of embodiment of the invention reading of data and system, when buffer area stores the data of read request visit, according to the rate of load condensate of described disk and the balanced rate of visit of described buffer area, determine from disk still is buffer area, to read the data of described read request visit, make the visit of system not exclusively depend on the performance of buffer area, reach the purpose of buffer area and disk access equilibrium, improved the overall performance of storage system.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is method embodiment one process flow diagram of copies data provided by the invention;
Fig. 2 is method embodiment two process flow diagrams of copies data provided by the invention;
Fig. 3 is method embodiment three process flow diagrams of copies data provided by the invention;
Fig. 4 is method embodiment one process flow diagram of reading of data provided by the invention;
Fig. 5 is method embodiment two process flow diagrams of reading of data provided by the invention;
Fig. 6 is the structural representation of the device embodiment one of copies data provided by the invention;
Fig. 7 is the structural representation of the device embodiment two of copies data provided by the invention;
Fig. 8 is the structural representation of the device embodiment three of copies data provided by the invention;
Fig. 9 is the structural representation of the device embodiment one of reading of data provided by the invention;
Figure 10 is the structural representation of the device embodiment two of reading of data provided by the invention;
Figure 11 is a kind of storage system example structure synoptic diagram provided by the invention;
Figure 12 is an another kind of storage system example structure synoptic diagram provided by the invention.
Embodiment
Further specify the technical scheme of the embodiment of the invention below in conjunction with the drawings and specific embodiments, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that is obtained under the creative work prerequisite.
Fig. 1 is method embodiment one process flow diagram of copies data provided by the invention, and the method for embodiment of the invention copies data is meant from disk to the buffer area copy data.That is to say that after copies data, disk and buffer area are all preserved the data that are copied.As shown in Figure 1, present embodiment comprises:
Step 11: when the access frequency of buffer area is no more than the high-frequency threshold value,, determine copying speed to the buffer area copies data according to the access frequency of buffer area, the rate of load condensate of disk and the rate of load condensate of buffer area.
The access frequency of buffer area, IO (input and output) number in expression access cache district accounts for the ratio of total IO number, is also referred to as the hit rate of buffer area.Total IO number is the summation of reading the IO number of buffer area and reading the IO number of disk.Before the buffer area copies data, judge whether the access frequency of buffer area is lower than the high-frequency threshold value from disk (for example traditional mechanical disk).If, then determine copying speed according to the visit load of the visit load of disk and buffer area, with according to this copying speed to the buffer area copies data.The access frequency of buffer area is no more than the high-frequency threshold value, and the hot spot data of expression buffer area is few, and utilization factor is not high, for the utilization factor that further improves buffer area can be from disk copies data to buffer area.
Step 12: according to copying speed, from disk to the buffer area copies data.
Further, as shown in Figure 2, when the access frequency of buffer area surpassed the high-frequency threshold value, the method for copies data was as follows:
Step 21: when the access frequency of buffer area surpasses the high-frequency threshold value, determine whether the rate of load condensate of disk and the rate of load condensate of buffer area all do not surpass the first rate of load condensate threshold values.
The access frequency of buffer area surpasses the high-frequency threshold value, and the expression buffer area has been stored more hot spot data, and the utilization factor of buffer area is higher, follow-up need be according to the rate of load condensate decision of disk and buffer area whether from disk to the buffer area copies data.
Step 22: when the rate of load condensate of the rate of load condensate of disk and buffer area all surpasses the first rate of load condensate threshold values, determine copying speed to described buffer area copies data according to system's maximum bandwidth.
If the rate of load condensate of the rate of load condensate of disk and buffer area wherein any one is higher than the first rate of load condensate threshold values, show that buffer area and disk all are in busy state, do not carry out the operation of copies data for not influencing system performance.Rate of load condensate represents that promptly the IO of disk or buffer area handles busy extent, and rate of load condensate is high more, and the processing of expression disc is busy more.
The method of present embodiment copies data when the buffer area copies data, is determined copy opportunity according to the access frequency in current cache district from disk.And determine copying speed according to the visit rate of load condensate of disk and the visit rate of load condensate of buffer area.Thereby making does not influence regular traffic when the buffer area copies data, help improving the overall performance of storage system.
Fig. 3 is method embodiment three process flow diagrams of copies data provided by the invention, and the storage medium of buffer area can be SSD in the present embodiment, and disk can be HDD conventional disc drivers such as (Hard Disk Drive).As shown in Figure 3, present embodiment comprises:
Step 31: user's scene is set so that different maximum copying speed M, the maximum bandwidth N of system to be set.
Step 32: whether the access frequency of judging SSD surpasses the high-frequency threshold value.If do not surpass execution in step 33, if surpass then execution in step 36.
The access frequency of SSD does not surpass the high-frequency threshold value, and the hot spot data of expression buffer area is few, can be from HDD to the SDD copies data.But when the access frequency of SSD surpassed the low frequency threshold value, the hot spot data of storage that shows SDD was very few and approach idle condition, for the effect of performance SDD needs in time copy hot spot data from disk to buffer area.When the access frequency of SSD surpasses the low frequency threshold value but surpasses the high-frequency threshold values, the processing that shows SDD has been in certain busy condition, need consider the load of SDD and the load of HDD from HDD to the speed of SDD copies data, thereby avoid influence the entire system performance.
Wherein, β is the mxm. in the maximum load rate of the maximum load rate of HDD and SDD, the maximum copying speed of M for allowing.During copies data service feature being had certain influence, with less than the speed copies data of M the time, is acceptable to the influence of service feature.
Step 35: when the access frequency of SSD surpasses the low frequency threshold value, determine to be: 10% * N, and copies data from the copying speed of HDD when the SSD copies data.
Wherein, N is system's maximum bandwidth, and little when machine-readable, N is the bandwidth of maximum IOPS correspondence; Bulk is when machine-readable, and N is a maximum bandwidth.IOPS (I/Os per second): i.e. per second input and output number of times.The IO frequency of the system that refers to treatable maximum in the unit interval.
Step 36: when the access frequency of SSD surpasses the high-frequency threshold value, judge whether the rate of load condensate of HDD and the rate of load condensate of SDD all do not surpass the first rate of load condensate threshold values.If execution in step 37, otherwise execution in step 38.
The first rate of load condensate threshold values can be 95%.When the access frequency of SDD surpassed the high-frequency threshold values, if wherein any one is more than or equal to 95% for first rate of load condensate of the rate of load condensate of HDD and SDD, it was busy then to be considered as system, not copies data.
Step 37: when the rate of load condensate of the rate of load condensate of HDD and SDD all surpasses the first rate of load condensate threshold values, determine to be: 10% * N, and copies data from the copying speed of HDD when the SSD copies data.N value herein is identical with N value in the step 35.
When the access frequency of SDD surpasses the high-frequency threshold values, if the rate of load condensate of the rate of load condensate of HDD and SDD does not all surpass the first rate of load condensate threshold values, but show the copy of HDD and SDD deal with data, determine that copying speed is 10% * N.
The method of present embodiment copies data, according to the different user scene setting different maximum copying speed M and the maximum bandwidth N of system.When the access frequency of SDD surpasses the high-frequency threshold values, and the rate of load condensate of the rate of load condensate of HDD and SDD is when all surpassing the first rate of load condensate threshold values, to the buffer area copies data, otherwise copies data not.In addition, when the access frequency of SDD surpasses the high-frequency threshold values and surpasses the low frequency threshold values, determine speed when the SDD copies data according to system's maximum bandwidth.When the access frequency of SDD surpasses the low frequency threshold values, determine speed when the SDD copies data according to the maximum copying speed that allows.Because the access speed when the present invention dynamically adjusts copies data according to the access frequency of SSD does not influence other business of system, and has improved the utilization factor of SSD Cache.
Fig. 4 is method embodiment one process flow diagram of reading of data provided by the invention, and as shown in Figure 4, present embodiment comprises:
Step 41: after receiving read request, judge the data that whether store described read request visit in the buffer area.
Step 42: when in buffer area, storing the data of read request visit,, determine from buffer area or disk, to read the data of described read request visit according to the rate of load condensate of disk and the balanced rate of visit of buffer area.
Can not only store hot spot data among the SSD, when copies data in SDD, need suitably one piece of data not copied in SDD according to actual read number according to business characteristic, system like this both can read from SSD during reading of data, can read from HDD again.
The method of present embodiment reading of data, when buffer area stores the data of read request visit, according to the rate of load condensate of described mechanical hard disk and the balanced rate of visit of described buffer area, determine from disk still is buffer area, to read the data of described read request visit, make the visit of system not exclusively depend on the performance of buffer area, reach the purpose of buffer area and disk access equilibrium, improved the overall performance of storage system.
Fig. 5 is method embodiment two process flow diagrams of reading of data provided by the invention, and the storage medium of buffer area is SSD in the present embodiment, and disk is HDD.As shown in Figure 5, present embodiment comprises:
Step 51: user's scene is set so that the balanced rate of different visits, the second rate of load condensate threshold value and balanced rate threshold value to be set.
Step 52: whether the data of read request visit are in SSD.In the time of not in SSD, execution in step 53, otherwise execution in step 54.
Step 53: reading of data from HDD.
The data of read request visit not in SSD, reading of data from HDD.
Step 54: the data of read request visit judge whether the HDD rate of load condensate reaches the second rate of load condensate threshold value in SSD.If reach execution in step 55, otherwise execution in step 56.
Step 55: the data that from SSD, read the read request visit.
The HDD rate of load condensate reaches the second rate of load condensate threshold value, and the visit capacity of expression HDD is bigger, has reached the processing limit of HDD.When the HDD rate of load condensate reaches the second rate of load condensate threshold value, from SSD, read the data of read request visit.Can be by the busy extent of HDD, for example the IO of HDD postpones situation, determines the second rate of load condensate threshold values.
When step 56:HDD rate of load condensate does not reach the second rate of load condensate threshold value, judge whether the balanced rate of visit of SSD reaches balanced rate threshold value.Reach balanced rate threshold value, then execution in step 53; Otherwise execution in step 55.
Whether the balanced rate of visit (ratio of data volume that reads from SSD and the data volume that reads from HDD) of judging SSD reaches balanced rate threshold value.Quantity, the performance of each SSD and the performance setting of HDD of the SSD that balanced rate threshold value comprises according to SSD Cache.For example, there are m SSD and n HDD in system, and the performance of each SSD is P, and the performance of each HDD is Q.The SSD group total load that can support is m * P so, and it is n * Q that HDD organizes the total load that can support.Balanced rate threshold value is exactly the ratio of m * P and n * Q.
When the balanced rate of the visit of SSD reached balanced rate threshold value, the visit capacity of expression SDD was bigger, has reached the processing limit of HDD, reads the data of read request visit from HDD.When the balanced rate of the visit of SSD does not reach balanced rate threshold value, from SSD, read the data of read request visit.
Different business scenarios needs different visit balance policies.For SSD owing to have short access delay characteristic, make its with machine-readable performance than high many of traditional magnetic disk, but its bandwidth and traditional magnetic disk to compare advantage little.So based on machine-readable when professional, as internet, applications, SSD visits large percentage, and to read in bulk be main when professional, and as the IPTV business, HDD visits large percentage.
The method of present embodiment reading of data when storing the data of read request visit among the SDD, is HDD when having reached processing limit if the rate of load condensate of HDD has reached capacity, and direct reading of data from SDD is to alleviate the visit load of HDD; If the rate of load condensate of HDD does not reach capacity, and the balanced rate of the visit of the SDD visit capacity that to surpass balanced rate threshold values be SDD is when too much, and reading of data from HDD is to alleviate the visit load of SDD; If the rate of load condensate of HDD does not reach capacity, and the balanced rate of the visit of the SDD visit capacity that also to surpass balanced rate threshold values be SDD more after a little while, and reading of data from SDD is to improve access speed.Therefore, make that SDD and HDD can obtain utilizing simultaneously, reached the balanced purpose of visit, the overall performance of storage system has been brought into play in maximization.
Fig. 6 is the structural representation of the device embodiment one of copies data provided by the invention.As shown in Figure 6, present embodiment comprises: the first speed determination module 61 and copy module 62 and frequency judge module 63.
The first speed determination module 61, be used for when frequency judge module 63 determines that the access frequency of buffer area is no more than the high-frequency threshold value, according to the access frequency of buffer area, the rate of load condensate of disk and the rate of load condensate of buffer area, determine copying speed to the buffer area copies data;
Further, as shown in Figure 7, the device of copies data also comprises: rate of load condensate judge module 64 and second speed determination module 65.
Rate of load condensate judge module 64 is used for when frequency judge module 63 determines that the access frequency of buffer area surpasses the high-frequency threshold value, judges whether the rate of load condensate of disk and the rate of load condensate of buffer area all do not surpass the first rate of load condensate threshold values;
Second speed determination module 65 is used for when rate of load condensate judge module 64 determines that the rate of load condensate of the rate of load condensate of disks and buffer area all surpasses the first rate of load condensate threshold values, determines copying speed to the buffer area copies data according to system's maximum bandwidth.
The working mechanism of above-mentioned each module does not repeat them here referring to describing among Fig. 1 and the corresponding embodiment of Fig. 2.
The device of present embodiment copies data when the buffer area copies data, is determined copy opportunity according to the access frequency in current cache district from disk.And determine copying speed according to the visit load of the visit load of disk and buffer area.Thereby making does not influence regular traffic when the buffer area copies data, help improving the overall performance of storage system.
Fig. 8 is the structural representation of the device embodiment three of copies data provided by the invention.As shown in Figure 8, the first speed determination module 61 comprises: the first speed determining unit 611 and second speed determining unit 612.
The first speed determining unit 611, be used for when frequency judge module 63 determines that the access frequency of buffer area surpasses the low frequency threshold value, according to the maximum copying speed of maximal value and system in the maximum load rate of the maximum load rate of disk and buffer area, determine copying speed to the buffer area copies data.
Second speed determining unit 612 is used for when frequency judge module 63 determines that the access frequency of buffer area is no more than the low frequency threshold value, according to the definite copying speed to the buffer area copies data of system's maximum bandwidth.
The working mechanism of above-mentioned each module does not repeat them here referring to describing among the corresponding embodiment of Fig. 3.
The device of present embodiment copies data, when the access frequency of buffer area surpasses the high-frequency threshold values, and the rate of load condensate of the rate of load condensate of disk and buffer area is when all surpassing the first rate of load condensate threshold values, to the buffer area copying speed, otherwise copies data not.In addition, when the access frequency of buffer area surpasses the high-frequency threshold values and surpasses the low frequency threshold values, determine speed when the buffer area copies data according to system's maximum bandwidth.When the access frequency of buffer area surpasses the low frequency threshold values, determine speed when the buffer area copies data according to the maximum copying speed that allows.Because the access speed the when embodiment of the invention is dynamically adjusted copies data according to the access frequency of SSD does not influence other business of system, and has improved the utilization factor of SSD Cache.
Fig. 9 is the structural representation of the device embodiment one of reading of data provided by the invention.As shown in Figure 9, present embodiment comprises: judge module 91 and read module 92.
Read module 92 is used for when judge module 91 determines that buffer area stores the data of read request visit, according to the rate of load condensate of disk and the balanced rate of visit of buffer area, determines to read the data of read request visit from buffer area or disk.
The working mechanism of above-mentioned each module does not repeat them here referring to describing among the corresponding embodiment of Fig. 4.
The device of present embodiment reading of data, when buffer area stores the data of read request visit, according to the rate of load condensate of mechanical hard disk and the balanced rate of visit of buffer area, determine from disk still is buffer area, to read the data of read request visit, make the visit of system not exclusively depend on the performance of buffer area, reach the purpose of buffer area and disk access equilibrium, improved the overall performance of storage system.
Figure 10 is the structural representation of the device embodiment two of reading of data provided by the invention.As shown in figure 10, read module 92 comprises: first judging unit 921, reading unit 922 and second judging unit 923.
First judging unit 921 is used to judge whether the rate of load condensate of disk reaches the second rate of load condensate threshold value.
Reading unit 922 is used for reading the data of read request visit from buffer area when first judging unit 921 determines that the rate of load condensate of disk reaches the second rate of load condensate threshold value.
Reading unit 922 when also being used for second judging unit 923 and determining that the balanced rate of visit of buffer areas does not reach balanced rate threshold value, reads the data of read request visit from buffer area;
Reading unit 922 also is used for second judging unit 923 and determines that the balanced rate of visit of buffer area reaches balanced rate threshold value, reads the data of read request visit from disk.
The working mechanism of above-mentioned each module does not repeat them here referring to describing among the corresponding embodiment of Fig. 5.
The device of present embodiment reading of data when storing the data of read request visit in the buffer area, is a disk when having reached processing limit if the rate of load condensate of disk has reached capacity, directly reading of data from buffer area; If the rate of load condensate of disk does not reach capacity, and the balanced rate of the visit of the buffer area visit capacity that to surpass balanced rate threshold values be buffer area is when too much, reading of data from disk; If the rate of load condensate of disk does not reach capacity, and the balanced rate of the visit of the buffer area visit capacity that also to surpass balanced rate threshold values be buffer area more after a little while, reading of data from buffer area.Therefore, make that buffer area and disk can obtain utilizing simultaneously, reached the balanced purpose of visit, the overall performance of storage system has been brought into play in maximization.
Figure 11 is a kind of storage system example structure synoptic diagram provided by the invention.As shown in figure 11.The present embodiment storage system comprises: disk 111 and solid state hard disc 112 also comprise the device 113 of copies data.Wherein disk 111 can be traditional mechanical hard disk, and the device 113 of copies data can be the device of the copies data of describing among the corresponding embodiment of Fig. 6 to Fig. 8, and solid state hard disc 112 is a kind of medium of the buffer area described among the corresponding embodiment of Fig. 6 to Fig. 8.In addition, the present embodiment storage system also can comprise the device of the reading of data of describing among the corresponding embodiment of Fig. 9 to Figure 10.
Figure 12 is an another kind of storage system example structure synoptic diagram provided by the invention.As shown in figure 12.The present embodiment storage system comprises: disk 121 and solid state hard disc 122 also comprise the device 123 of reading of data.Wherein disk 121 can be traditional mechanical hard disk, and the device 123 of reading of data can be the device of the reading of data of describing among the corresponding embodiment of Fig. 9 to Figure 10, and solid state hard disc 122 is a kind of medium of the buffer area described among the corresponding embodiment of Fig. 6 to Fig. 8.In addition, the present embodiment storage system also can comprise the device of the copies data of describing among the corresponding embodiment of Fig. 6 to Fig. 8.
One of ordinary skill in the art will appreciate that: all or part of step that realizes said method embodiment can be finished by the relevant hardware of programmed instruction, aforesaid program can be stored in the computer read/write memory medium, this program is carried out the step that comprises said method embodiment when carrying out; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
It should be noted that at last: above embodiment only in order to technical scheme of the present invention to be described, is not intended to limit; Although with reference to previous embodiment the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: it still can be made amendment to the technical scheme that aforementioned each embodiment put down in writing, and perhaps part technical characterictic wherein is equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution break away from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (11)
1. the method for a copies data is characterized in that, comprising:
When the access frequency of buffer area is no more than the high-frequency threshold value,, determine copying speed to described buffer area copies data according to the access frequency of described buffer area, the rate of load condensate of disk and the rate of load condensate of described buffer area;
According to described copying speed, from described disk to described buffer area copies data.
2. the method for copies data according to claim 1 is characterized in that, also comprises:
When the access frequency of buffer area surpasses described high-frequency threshold value, judge whether the rate of load condensate of described disk and the rate of load condensate of described buffer area all surpass the first rate of load condensate threshold values;
When the rate of load condensate of the rate of load condensate of disk and buffer area all surpasses the first rate of load condensate threshold values, determine copying speed to described buffer area copies data according to system's maximum bandwidth.
3. the method for copies data according to claim 1 and 2 is characterized in that, according to the access frequency of described buffer area, the rate of load condensate of disk and the rate of load condensate of described buffer area, determines the copying speed to described buffer area copies data, comprising:
When the access frequency of described buffer area is no more than described high-frequency threshold value but surpasses the low frequency threshold value, according to the maximum copying speed of maximal value and system in the maximum load rate of the maximum load rate of described disk and described buffer area, determine copying speed to described buffer area copies data;
When the access frequency of described buffer area is no more than described low frequency threshold value, determine copying speed to described buffer area copies data according to system's maximum bandwidth.
4. the device of a copies data is characterized in that, comprising:
The frequency judge module is used to judge whether the access frequency of buffer area surpasses the high-frequency threshold value;
The first speed determination module is used for when the access frequency of buffer area is no more than the high-frequency threshold value, according to the access frequency of described buffer area, the rate of load condensate of disk and the rate of load condensate of described buffer area, determines the copying speed to described buffer area copies data;
The copy module, be used for according to described copying speed, from described disk to described buffer area copies data.
5. the device of copies data according to claim 4 is characterized in that, also comprises: rate of load condensate judge module and second speed determination module;
Described rate of load condensate judge module is used for when the access frequency of buffer area surpasses the high-frequency threshold value, determines whether the rate of load condensate of disk and the rate of load condensate of buffer area all do not surpass the first rate of load condensate threshold values;
Described second speed determination module is used for when the rate of load condensate of the rate of load condensate of disk and buffer area all surpasses the first rate of load condensate threshold values, determines copying speed to described buffer area copies data according to system's maximum bandwidth.
6. according to the device of claim 4 or 5 described copies data, it is characterized in that the described first speed determination module comprises:
The first speed determining unit, be used for that access frequency at described buffer area is no more than described high-frequency threshold value but when surpassing the low frequency threshold value, according to the maximum copying speed of maximal value and system in the maximum load rate of the maximum load rate of described disk and described buffer area, determine copying speed to described buffer area copies data;
The second speed determining unit when being used for access frequency at described buffer area and being no more than described low frequency threshold value, is determined copying speed to described buffer area copies data according to system's maximum bandwidth.
7. the method for a reading of data is characterized in that, comprising:
After receiving read request, judge the data that whether store described read request visit in the buffer area;
When in described buffer area, storing the data of read request visit,, determine from buffer area or disk, to read the data of described read request visit according to the rate of load condensate of disk and the balanced rate of visit of described buffer area.
8. the method for reading of data according to claim 7 is characterized in that, according to the rate of load condensate of described disk and the balanced rate of visit of described buffer area, determines to read the data of described read request visit from buffer area or disk, comprising:
When the rate of load condensate of described disk reaches the second rate of load condensate threshold value, read the data of described read request visit from described buffer area;
When the rate of load condensate of described disk does not reach the described second rate of load condensate threshold value, judge whether the balanced rate of visit of described buffer area reaches balanced rate threshold value; The balanced rate of the visit of described buffer area is the number percent of data volume that reads and the data volume that reads from described disk from buffer area; Quantity, the performance of each described solid state hard disc and the performance setting of described disk of the solid state hard disc that described balanced rate threshold value comprises according to described buffer area.
If the balanced rate of the visit of described buffer area does not reach balanced rate threshold value, read the data of described read request visit from described buffer area;
If the balanced rate of the visit of described buffer area reaches balanced rate threshold value, read the data of described read request visit from described disk.
9. the device of a reading of data is characterized in that, comprising:
Judge module after being used to receive read request, is judged the data that whether store described read request visit in the buffer area;
Read module is used for when described buffer area stores the data of read request visit, according to the rate of load condensate of disk and the balanced rate of visit of described buffer area, determines to read the data of described read request visit from described buffer area or described disk.
10. the device of reading of data according to claim 9 is characterized in that, described read module comprises:
First judging unit is used to judge whether the rate of load condensate of described disk reaches the second rate of load condensate threshold value;
First reading unit when being used for rate of load condensate at described disk and reaching the described second rate of load condensate threshold value, reads the data of described read request visit from described buffer area;
Described second judging unit when being used for rate of load condensate at described disk and not reaching the described second rate of load condensate threshold value, judges whether the balanced rate of visit of described buffer area reaches balanced rate threshold value; The balanced rate of the visit of described buffer area is the number percent of data volume that reads and the data volume that reads from described disk from buffer area; Quantity, the performance of each described solid state hard disc and the performance setting of described disk of the solid state hard disc that described balanced rate threshold value comprises according to described buffer area;
When described reading unit, the balanced rate of visit that also is used for described buffer area do not reach balanced rate threshold value, read the data of described read request visit from described buffer area;
Described reading unit, the balanced rate of visit that also is used for described buffer area reaches described balanced rate threshold value, reads the data of described read request visit from described disk.
11. a storage system comprises: disk and solid state hard disc, it is characterized in that, also comprise device as each described copies data of claim 4 to 6, and/or, the device of claim 9 or 10 described reading of data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010590713XA CN102096556B (en) | 2010-12-03 | 2010-12-03 | Method for copying data as well as method, device and system for reading data |
| PCT/CN2011/080788 WO2012071950A1 (en) | 2010-12-03 | 2011-10-14 | Method for copying data, and method, device and system for reading data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010590713XA CN102096556B (en) | 2010-12-03 | 2010-12-03 | Method for copying data as well as method, device and system for reading data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102096556A true CN102096556A (en) | 2011-06-15 |
| CN102096556B CN102096556B (en) | 2013-11-06 |
Family
ID=44129666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010590713XA Active CN102096556B (en) | 2010-12-03 | 2010-12-03 | Method for copying data as well as method, device and system for reading data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102096556B (en) |
| WO (1) | WO2012071950A1 (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012071950A1 (en) * | 2010-12-03 | 2012-06-07 | 成都市华为赛门铁克科技有限公司 | Method for copying data, and method, device and system for reading data |
| CN103294407A (en) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | Storage device and data read-write method |
| CN105487811A (en) * | 2015-02-09 | 2016-04-13 | 哈尔滨安天科技股份有限公司 | File reading and writing method and system based on buffering alignment |
| CN105893118A (en) * | 2016-03-30 | 2016-08-24 | 联想(北京)有限公司 | Information processing method and electronic device |
| CN106155931A (en) * | 2016-06-30 | 2016-11-23 | 百度在线网络技术（北京）有限公司 | The method and apparatus of speed limit read by a kind of disk |
| WO2017031958A1 (en) * | 2015-08-26 | 2017-03-02 | 华为技术有限公司 | Write strategy control method and device |
| CN106775436A (en) * | 2015-11-24 | 2017-05-31 | 群联电子股份有限公司 | Data access method, memorizer control circuit unit and memory |
| CN108874324A (en) * | 2018-07-17 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of access request processing method, device, equipment and readable storage medium storing program for executing |
| CN110119251A (en) * | 2019-05-14 | 2019-08-13 | 北京杰思安全科技有限公司 | A kind of buffer control method and system based on disk big flow |
| CN110737393A (en) * | 2018-07-20 | 2020-01-31 | 伊姆西Ip控股有限责任公司 | Data reading method, device and computer program product |
| CN112540933A (en) * | 2020-11-26 | 2021-03-23 | 华云数据控股集团有限公司 | Cache reading and writing method and device and electronic equipment |
| CN112597104A (en) * | 2021-01-11 | 2021-04-02 | 武汉飞骥永泰科技有限公司 | Small file performance optimization method and system |
| CN113836084A (en) * | 2014-11-10 | 2021-12-24 | 创新先进技术有限公司 | Data storage method, device and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050050279A1 (en) * | 2003-08-29 | 2005-03-03 | Chiu Lawrence Yium-Chee | Storage system and method for prestaging data in a cache for improved performance |
| CN1924834A (en) * | 2006-09-08 | 2007-03-07 | 华为技术有限公司 | High-speed memory pre-read method and device |
| CN101135994A (en) * | 2007-09-07 | 2008-03-05 | 杭州华三通信技术有限公司 | Method and apparatus for dividing cache space and cache controller thereof |
| CN101493795A (en) * | 2008-01-24 | 2009-07-29 | 杭州华三通信技术有限公司 | Storage system, storage controller, and cache implementing method in the storage system |
| CN101668046A (en) * | 2009-10-13 | 2010-03-10 | 成都市华为赛门铁克科技有限公司 | Resource caching method, resource obtaining method, device and system thereof |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645928B (en) * | 2009-08-26 | 2012-07-25 | 成都市华为赛门铁克科技有限公司 | Content resource caching method, device and system |
| CN101819509A (en) * | 2010-04-19 | 2010-09-01 | 清华大学深圳研究生院 | Solid state disk read-write method |
| CN102096556B (en) * | 2010-12-03 | 2013-11-06 | 华为数字技术（成都）有限公司 | Method for copying data as well as method, device and system for reading data |
-
2010
- 2010-12-03 CN CN201010590713XA patent/CN102096556B/en active Active
-
2011
- 2011-10-14 WO PCT/CN2011/080788 patent/WO2012071950A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050050279A1 (en) * | 2003-08-29 | 2005-03-03 | Chiu Lawrence Yium-Chee | Storage system and method for prestaging data in a cache for improved performance |
| CN1924834A (en) * | 2006-09-08 | 2007-03-07 | 华为技术有限公司 | High-speed memory pre-read method and device |
| CN101135994A (en) * | 2007-09-07 | 2008-03-05 | 杭州华三通信技术有限公司 | Method and apparatus for dividing cache space and cache controller thereof |
| CN101493795A (en) * | 2008-01-24 | 2009-07-29 | 杭州华三通信技术有限公司 | Storage system, storage controller, and cache implementing method in the storage system |
| CN101668046A (en) * | 2009-10-13 | 2010-03-10 | 成都市华为赛门铁克科技有限公司 | Resource caching method, resource obtaining method, device and system thereof |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012071950A1 (en) * | 2010-12-03 | 2012-06-07 | 成都市华为赛门铁克科技有限公司 | Method for copying data, and method, device and system for reading data |
| CN103294407A (en) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | Storage device and data read-write method |
| CN113836084A (en) * | 2014-11-10 | 2021-12-24 | 创新先进技术有限公司 | Data storage method, device and system |
| CN105487811A (en) * | 2015-02-09 | 2016-04-13 | 哈尔滨安天科技股份有限公司 | File reading and writing method and system based on buffering alignment |
| WO2017031958A1 (en) * | 2015-08-26 | 2017-03-02 | 华为技术有限公司 | Write strategy control method and device |
| CN106775436B (en) * | 2015-11-24 | 2019-10-25 | 群联电子股份有限公司 | Data access method, memorizer control circuit unit and memory |
| CN106775436A (en) * | 2015-11-24 | 2017-05-31 | 群联电子股份有限公司 | Data access method, memorizer control circuit unit and memory |
| CN105893118A (en) * | 2016-03-30 | 2016-08-24 | 联想(北京)有限公司 | Information processing method and electronic device |
| CN106155931A (en) * | 2016-06-30 | 2016-11-23 | 百度在线网络技术（北京）有限公司 | The method and apparatus of speed limit read by a kind of disk |
| CN106155931B (en) * | 2016-06-30 | 2020-07-07 | 百度在线网络技术（北京）有限公司 | Method and device for limiting speed of disk reading |
| CN108874324A (en) * | 2018-07-17 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of access request processing method, device, equipment and readable storage medium storing program for executing |
| CN108874324B (en) * | 2018-07-17 | 2021-08-31 | 郑州云海信息技术有限公司 | Access request processing method, device, equipment and readable storage medium |
| CN110737393A (en) * | 2018-07-20 | 2020-01-31 | 伊姆西Ip控股有限责任公司 | Data reading method, device and computer program product |
| CN110737393B (en) * | 2018-07-20 | 2023-08-11 | 伊姆西Ip控股有限责任公司 | Data reading method, apparatus and computer program product |
| CN110119251A (en) * | 2019-05-14 | 2019-08-13 | 北京杰思安全科技有限公司 | A kind of buffer control method and system based on disk big flow |
| CN110119251B (en) * | 2019-05-14 | 2023-01-03 | 北京杰思安全科技有限公司 | Buffer control method and system based on disk large flow |
| CN112540933A (en) * | 2020-11-26 | 2021-03-23 | 华云数据控股集团有限公司 | Cache reading and writing method and device and electronic equipment |
| CN112597104A (en) * | 2021-01-11 | 2021-04-02 | 武汉飞骥永泰科技有限公司 | Small file performance optimization method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102096556B (en) | 2013-11-06 |
| WO2012071950A1 (en) | 2012-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102096556B (en) | Method for copying data as well as method, device and system for reading data | |
| JP6106719B2 (en) | Data deduplication | |
| US9405485B2 (en) | Method and apparatus for writing data to a flash memory | |
| US10572379B2 (en) | Data accessing method and data accessing apparatus | |
| US11630766B2 (en) | Memory system and operating method thereof | |
| CN101398749B (en) | Static state attrition balancing method, device and system | |
| CN101788995A (en) | Hotspot data identification method and device | |
| WO2016048599A1 (en) | Caching technologies employing data compression | |
| CN105117180A (en) | Data storing method and device and solid state disc | |
| US11809723B2 (en) | Unbalanced plane management method, associated data storage device and controller thereof | |
| US11385831B2 (en) | Memory controller and storage device including the same | |
| CN108733324B (en) | Data reading and writing method, device and equipment of solid state disk and storage medium | |
| US20170004087A1 (en) | Adaptive cache management method according to access characteristics of user application in distributed environment | |
| CN103927129A (en) | Data storage method | |
| CN104252422A (en) | Memory access method and memory controller | |
| CN103714010A (en) | Storage device write-in method and storage device | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| CN103530236A (en) | Method and device for obtaining hybrid hard disk | |
| US20170269834A1 (en) | Data storage device and data management method thereof | |
| CN108334457B (en) | IO processing method and device | |
| CN113805812B (en) | Cache management method, device, equipment and storage medium | |
| CN113900591A (en) | Method and device for prolonging service life of memory, electronic equipment and storage medium | |
| TWI718635B (en) | Unbalanced plane management method, associated data storage device and controller thereof | |
| CN109992217B (en) | Service quality control method and device, electronic equipment and storage medium | |
| CN110119245B (en) | Method and system for operating NAND flash memory physical space to expand memory capacity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Applicant after: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. Address before: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Applicant before: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES Co.,Ltd. |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES CO., LTD. TO: HUAWEI DIGITAL TECHNOLOGY (CHENGDU) CO., LTD. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220829 Address after: No. 1899 Xiyuan Avenue, high tech Zone (West District), Chengdu, Sichuan 610041 Patentee after: Chengdu Huawei Technologies Co.,Ltd. Address before: 611731 Qingshui River District, Chengdu hi tech Zone, Sichuan, China Patentee before: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. |
|
| TR01 | Transfer of patent right |