CN102263818B - Method for storing and reading file data, and apparatus thereof - Google Patents
Method for storing and reading file data, and apparatus thereof Download PDFInfo
- Publication number
- CN102263818B CN102263818B CN 201110189501 CN201110189501A CN102263818B CN 102263818 B CN102263818 B CN 102263818B CN 201110189501 CN201110189501 CN 201110189501 CN 201110189501 A CN201110189501 A CN 201110189501A CN 102263818 B CN102263818 B CN 102263818B
- Authority
- CN
- China
- Prior art keywords
- data
- storage medium
- hard disk
- response time
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000003860 storage Methods 0.000 claims abstract description 200
- 230000004044 response Effects 0.000 claims abstract description 110
- 230000005540 biological transmission Effects 0.000 claims description 39
- 238000004364 calculation method Methods 0.000 claims description 11
- 239000012141 concentrate Substances 0.000 claims description 8
- 239000000203 mixture Substances 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 7
- 238000012546 transfer Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000000151 deposition Methods 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Images
Abstract
The invention discloses a method for storing and reading file data. The method comprises the following steps: arranging storage mediums according to response delay time, wherein an arrangement order is from short to long, and calculating differential time of the two adjacent storage mediums successively; calculating successively a data quantity transmitted under a full bandwidth in the differential time, setting a calculated result as an upper limit value of each write-in data quantity of the corresponding current storage medium and storing; successively storing data in a file to be stored to the storage medium according to the upper limit value of each write-in data quantity of each storage medium when a written data request appears; simultaneously sending or concentratedly sending a reading command to the storage medium which stores the required file data when a reading data request appears. The invention also discloses an apparatus of storing and reading the file data. The apparatus comprises a pre-set module, a write-in module and a reading module. By using the method and the apparatus of the invention, input and output performance of a storage system can be greatly raised without increasing costs.
Description
Technical field
The present invention relates to technical field of data storage, the method and apparatus that particularly relates to a kind of file data storing and read.
Background technology
Storage system refers in computer by various memory devices, control assembly and the management information of depositing program and data
The system that the equipment (hardware) of scheduling and algorithm (software) form.From the angle of using, a storage is to accomplish to deal with the fastest response speed as possible user's read-write requests.Therefore the read or write speed of data for weighing a performance of storage system, just seems very meaningful.And the non-volatile medium of depositing such as hard disk as a kind of memory device of extensive use, has considerable influence to this performance of the speed that reads and writes data of storage system.
At present, the non-volatile medium of depositing of data mainly contains SATA (Serial Advanced Technology Attachment, Serial Advanced Technology Attachment) hard disk, SAS (SerialAttached SCSI, Serial Attached SCSI (SAS)) hard disk and SSD (Solid State Disk, solid state hard disc) hard disk etc.The rotating speed of SATA hard disk is 7200 rev/mins at present, 8 to 12 milliseconds of the seek time average out to of magnetic head, internal data transfer speed average out to 110MB/ second.And the rotating speed of SAS hard disk is at 10000 rev/mins to 15000 rev/mins, 2 to 4 milliseconds of the seek time average out to of magnetic head, and internal transmission speed is suitable with the SATA hard disk.And there is not the seek time of magnetic head in the SSD hard disk owing to adopting semiconductor technology, thereby read or write speed can reach very high, and it is only tens of microseconds that read-write postpones.
All kinds of hard disks are due to the performance characteristics of self, and the response speed of the order that the user is sent is different, and the magnetic head seek time is shorter, internal data transfer is faster, more Gao Ze is shorter time of delay for rotating speed, responds sooner.The SSD hard disk is better than SAS hard disk and SATA hard disk on these performance index of read-write response, the SAS hard disk is better than the SATA hard disk.
Traditional take the SAN (StorageAreaNetwork of optical-fibre channel as the basis, storage area network) in storage system, the read-write of data be SAN Control Server by interior of equipment cabinet on behalf of carrying out, the multi-section application server is shared the readwrite bandwidth between SAN Control Server and hard disk array.Because application server is not directly come into contacts with hard disk array, therefore can't decide according to the bandwidth that application server is read and write data the layering deposit method of file.
For the SAN storage system of this quasi-tradition, data hierarchy memory technology that storage manufacturer adopts is based on " cold " and " heat " degree of file.The file of so-called " heat " is the higher file of access frequency, and the file of " cold " is the lower file of access frequency.In present storage system the way of main flow be with the file migration of " heat " to reading and writing faster on storage medium, as the SSD hard disk, the file of " cold " is migrated on the slower storage medium of read-write, as the SATA hard disk.By this lasting migration, make active file when being read, can obtain preferential response.Thereby the performance that improves the user is experienced.
In the middle of reality, kind of document is various, the access situation is numerous and complicated mixed and disorderly, be difficult to find comparatively reasonably access frequency " heat " and " cold " of dividing file, and, although this kind mode can obviously improve system to the response speed of the file of " heat ", for the file of " cold ", still there is long problem time of delay of response.And " cold " although the access frequency of file Single document is not high, and high and capacity is less due to short hard disk price comparison of response time, enterprise is due to cost consideration, and the number of files that is defined as " cold " file is a large amount of often.Therefore the number of files that adopts existing this storage mode, any time can obtain this speed of read-write requests faster is very limited.
Summary of the invention
The method and apparatus that the object of the present invention is to provide a kind of file data storing and read, solve data hierarchy method of the prior art and be merely able to promote the very problem of the read-write response speed of finite population file, make under the identical condition of storage medium configuration, there is more file to obtain the lifting of read-write response speed, thereby significantly promotes the whole input and output performance of storage system.
Be a kind of file data storing and the read method of realizing that the object of the invention provides, comprise step: A. arranges existing different types of storage medium in storage system from short to long according to response time, calculates successively the differential time of the response time of the response time of current kind storage medium and next class storage medium; B. calculate successively the data volume of the wide energy transmission of filled band in described differential time, result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves, come the storage medium of last position without each data writing amount higher limit; When C. write data requests occurring, the data in the file that will deposit according to each data writing amount higher limit of every kind of storage medium are deposited into storage medium successively; When D. read data request occurring, send simultaneously or concentrate the transmission reading order to the storage medium that has required file data.
The response time of storage medium is for receiving after read write command to the length of the time of delay of sending desired data from storage medium.
Wherein, described step B comprises: B1. sets application server towards the bandwidth value that reads and writes data of storage medium; B2. calculate in described differential time the data volume that can transmit during the wide transmission of data bandwidth filled band; B3. result of calculation be set as each data writing amount higher limit of corresponding current storage medium and preserve.
Wherein, described step C comprises: when write data requests appearred in C1., whether the data volume of the file that judgement will deposit in greater than the each data writing amount of the storage medium that ranks the first higher limit, was that execution in step C2, no, execution in step C3; C2. according to the every kind of each data writing amount of storage medium higher limit, the data of the file that will deposit in are deposited into two or more storage medium successively; C3. the judgement file that need to deposit in whether be read frequency higher than predeterminated frequency, be, this document is deposited into the storage medium that ranks the first, no, be deposited into other storage mediums.
Wherein, when described storage medium comprised SSD hard disk, SAS hard disk and SATA hard disk, described steps A comprised: A1. arranges three kinds of hard disks from short to long according to response time, and rank results is SSD hard disk, SAS hard disk, SATA hard disk; A2. calculate the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk; Calculate the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
Wherein, described step B comprises: B1. sets application server towards the bandwidth value that reads and writes data of the storage array of three kinds of hard disks compositions; B2. calculate the wide data volume that can transmission of in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk filled band; The wide data volume that can transmission of calculating filled band in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk; B3. will be in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk the wide data volume that can transmission of filled band be set as each data writing amount higher limit of SSD hard disk; Will be in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk the wide data volume that can transmission of filled band be set as each data writing amount higher limit of SAS hard disk, the SATA hard disk is without each data writing amount higher limit.
Wherein, described step C comprises: when write data requests appearred in C1., whether the data volume of the file that judgement will deposit in greater than each data writing amount higher limit of SSD hard disk, was that execution in step C2, no, execution in step C3; Data with each data writing amount higher limit equivalent of SSD hard disk in the file that C2. will deposit in are deposited into the SSD hard disk; Remaining data in file are deposited into the SAS hard disk; If file data also has residue, the remaining data in file is deposited into the SATA hard disk; C3. the judgement file that need to deposit in whether be read frequency higher than predeterminated frequency, be, this document is deposited into the SSD hard disk, no, be deposited into SAS hard disk or SATA hard disk.
The present invention also provides a kind of file data storing and reading device, comprise: set in advance module, be positioned on Control Server, be used for the existing different types of storage medium of storage system is arranged from short to long according to response time, calculate successively the differential time of the response time of the response time of current kind storage medium and next class storage medium; Calculate successively the wide data volume that can transmit of filled band in described differential time, result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves; Writing module is positioned on application server, and when being used for write data requests occurring, the data in the file that will deposit according to each data writing amount higher limit of every kind of storage medium are deposited into storage medium successively; Read module is positioned on application server, when being used for read data request occurring, sends simultaneously or concentrate the transmission reading order to the storage medium that has required file data.
The response time of storage medium is for receiving after read write command to the length of the time of delay of sending desired data from storage medium.
Wherein, the described module that sets in advance comprises: the differential time acquisition module, be used for the existing different types of storage medium of storage system is arranged from short to long according to response time, calculate successively the differential time of the response time of the response time of current kind storage medium and next class storage medium; Set the bandwidth module, be used for setting application server towards the bandwidth value that reads and writes data of storage medium; Computing module is used for calculating the wide data volume that can transmit of filled band in described differential time; Memory module is used for result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves.
the invention has the beneficial effects as follows: a kind of file data reading/writing method and device that the present invention describes, be stored in different storage mediums by the data sectional with a file, make application server after sending read-write requests simultaneously to several different storage mediums, carry out at other storage mediums in the time of the work such as tracking and internal data transfer, the fastest storage medium of response speed takes the lead in desired data is sent, then response speed time fast storage medium also has been ready to the data that will send, by that analogy, this kind location mode can provide for the read requests of application server at once, the data response that filled band is wide.
Description of drawings
Fig. 1 is the flow chart of a kind of file data storing provided by the invention and read method;
Fig. 2 is the structure chart that can use the storage system of a kind of file data storing provided by the invention and read method;
Fig. 3 is the flow chart of a kind of embodiment of a kind of file data storing provided by the invention and read method.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, a kind of file data storing of the present invention and the method and apparatus that reads are further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, is not intended to limit the present invention.
Due to when the reading writing harddisk, the factor of restriction performance is mainly the seek time of magnetic head and the data transmission bauds of hard disk inside.Therefore, aim of the present invention is when the server write data, can be according to response time and the read or write speed difference of various storage mediums, in conjunction with the readwrite bandwidth between application server and hard disk array, calculate the size of data that to deposit and carry out corresponding striping and process on various storage mediums, so that can be by the relay service of different storage mediums when reading, for the read requests of application server provides data response at once, that filled band is wide.This calculating and data sectional are deposited necessary address allocation procedure and are carried out on the IPSAN Control Server of storage system inside.
A kind of file data reading/writing method provided by the invention as shown in Figure 1, comprises the steps:
A. existing different types of storage medium in storage system is arranged from short to long according to response time, calculated successively the differential time of the response time of the response time of current kind storage medium and next class storage medium.
The response time of storage medium is from storage medium and receives after read write command that to the length of the time of delay of sending desired data, time of delay is longer, and response speed is slower.The response time of hard disk depends primarily on the seek time of magnetic head and the data transmission bauds of hard disk inside.
After sequence, be the first storage medium, the second storage medium, the 3rd storage medium according to all kinds of storage mediums of sequence definition
Calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium, the time difference of namely at first calculating the response time of the first storage medium and the second storage medium, obtain the first differential time; The response time of then calculating the second storage medium and the 3rd storage medium is poor, obtains the second differential time; Calculate again the time difference of the 3rd storage medium and the 4th storage medium ... when not having the 4th storage medium, do not calculate.
B. calculate successively the data volume of the wide energy transmission of filled band in described differential time, result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves, come the storage medium of last position without each data writing amount higher limit.
Bandwidth is the call parameter of calculated data amount, should set in advance each application server towards the port bandwidth of storage medium on the IPSAN Control Server.
Calculating in the first differential time, the data volume that bandwidth can transmit fixedly the time, defining this data volume is the first data volume, the first data volume=bandwidth * the first differential time; Calculate again the second data volume, the second data volume=bandwidth * the second differential time ... in the middle of practical application, also should consider the problem of valid data transmission efficiency, at this moment, the first data volume=first differential time * bandwidth * valid data transmission efficiency, the second data volume=second differential time * bandwidth * valid data transmission efficiency, the rest may be inferred, can calculate the data volume in other differential times, repeat no more.
On the IPSAN Control Server, each data volume of calculating gained is designated as each data writing amount higher limit of corresponding storage medium, and it is saved as static data, a limit standard of the data writing amount when storing data into different storage medium.So-called higher limit, the data volume that namely at every turn is written to this kind storage medium can not be more than this higher limit.
When C. write data requests occurring, the data in the file that will deposit according to each data writing amount higher limit of every kind of storage medium are deposited into storage medium successively.
Because every kind of storage medium has all been set corresponding each data writing amount higher limit, therefore file is deposited in the process of different storage mediums, also just Data Division has been become a plurality of data segments.For example, if the data volume that a file arranged greater than each data writing amount (the first data volume) higher limit of the first storage medium less than each data writing amount higher limit sum of the first storage medium and the second storage medium, in the process that writes, the data of each data writing amount higher limit equivalent of the front in this document and the first storage medium are deposited into the first storage medium, remaining data volume writes the second storage medium, and file just has been divided into two data segments and stores like this.If the data volume of certain file is larger, the each data writing amount higher limit sum greater than the first storage medium and the second storage medium is deposited into the 3rd storage medium with the remaining data volume of this document, and like this, file just has been divided into three data segments and has stored.If three kinds of storage mediums are only arranged in storage system, the 3rd storage medium is not set each data writing amount higher limit.
in the middle of practical application, in order to take full advantage of application server reading and writing data bandwidth, the data that identical file is deposited in storage medium of the same race can be deposited in different memory cell through striping, as the reading and writing data bandwidth of supposing application server is 10Gbps, and the first storage medium is the SSD hard disk, its bandwidth that connects by network is 1Gbps, the first data volume is 4.3MB as calculated, the data of the front 4.3MB of a certain file can leave 10 or more on polylith SSD hard disk (bandwidth of 10 or more 1Gbps merge reach the above bandwidth of 10Gbps) in through striping, when the transmission of data, these SSD hard disks transmit data simultaneously, make the application server readwrite bandwidth fully to be occupied and to utilize.
When D. read data request occurring, send simultaneously or concentrate the transmission reading order to the storage medium that has required file data.
When read data request occurring, should guarantee that in all kinds of storage mediums and all kinds of storage medium, each memory cell is to receive simultaneously reading order.But, in actual applications, because application server can't accomplish to send simultaneously many reading orders, can only be one by one, the concentrated area sends reading order, namely each reading order sends in succession, but the time of two adjacent order meeting machine cycles of interval.Because this machine cycle is very of short duration, can ignore, reading order almost arrives all kinds of storage mediums simultaneously.
Almost receive simultaneously under the prerequisite of read write command at all kinds of storage mediums, because the response speed of the first storage medium is the fastest, through the response time of the first storage medium, the first storage medium sends the data of the first data volume in the middle of required file, after the first differential time, the transfer of data of the first data volume is complete.At this moment, almost receive simultaneously the moment of read write command apart from all kinds of storage mediums, elapsed time length is response time and the first differential time sum of the first storage medium, and the response time of the first storage medium and the first differential time sum are the response time of the second storage medium, that is to say from the moment of receiving read write command and count, passed through the response time of the second storage medium, at this moment, the second storage medium is with the DSR of the second data volume of required file and send to application server.When the data of the second data volume are transmitted when complete, the response time of the 3rd storage medium arrives, and the 3rd storage medium sends the data of the 3rd data volume, by that analogy, until the data in file are all sent.
Therefore, after sending the order that reads or writes to the data of certain file, only through the response time of the first storage medium, needed data will wide, continuous the sending of filled band, if the performance of the first storage medium is enough superior, the response time of the first storage medium can be ignored, and namely application server can obtain wide, the continuous data of filled band at once after sending read write command.
Referring to Fig. 2, Fig. 2 is for using the schematic diagram of a kind of storage system of the present invention.By the Ethernet switch Direct Communication, be point-to-point interconnect architecture between the application server of this system and storage medium array, have fixing bandwidth.
On such system architecture, the storage medium of storage system inside can have SSD hard disk, SAS hard disk and SATA hard disk.Technical scheme of the present invention is applied to this storage system, and storage medium is defined as SATA hard disk, SAS hard disk and SSD hard disk, and as shown in Figure 3, technical scheme of the present invention comprises:
A1. SATA hard disk, SAS hard disk and SSD hard disk are arranged from short to long according to response time.
The SSD hard disk is owing to adopting semiconductor technology, there is not the seek time of magnetic head, it is very high that read or write speed can reach, it is only tens of microseconds that read-write postpones, and the rotating speed of SAS hard disk is at 10000 rev/mins to 15000 rev/mins, 2 to 4 milliseconds of the seek time average out to of magnetic head, internal transmission speed is suitable with the SATA hard disk.The SATA hard disk gets 8 to 12 milliseconds of seek time average out to left and right of magnetic head, internal data transfer speed average out to 110MB/ second.Therefore the SSD hard disk should come at first, the SAS hard disk secondly, the result of arrangement is: SSD hard disk, SAS hard disk, SATA hard disk.
A2. calculate the first differential time of the response time of the response time of described SSD hard disk and described SAS hard disk; Calculate the second differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
The response time of supposing the SAS hard disk is 4ms (millisecond), and the response time of SATA is 12ms, and the response time of SSD hard disk is ignored, the first differential time=4ms-0ms=4ms, the second differential time=12ms-4ms=8ms.
B1. set application server towards the bandwidth value that reads and writes data of the storage array of three kinds of hard disks compositions.
B2. calculate the wide data volume that can transmission of in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk filled band; The wide data volume that can transmission of calculating filled band in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
Calculate the wide data volume that can transmit of filled band in the first differential time 4ms, this data volume is the first data volume, the first data volume=4ms* bandwidth.In the middle of practical application, also should consider the problem of valid data transmission efficiency, at this moment the first data volume=4ms* bandwidth * valid data transmission efficiency.Calculate the wide data volume that can transmit of filled band in the second differential time 8ms, this data volume is the second data volume, the second data volume=8ms* bandwidth * valid data transmission efficiency.
B3. will be in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk the wide data volume that can transmission of filled band be set as each data writing amount higher limit of SSD hard disk; Will be in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk the wide data volume that can transmission of filled band be set as each data writing amount higher limit of SAS hard disk, the SATA hard disk is without each data writing amount higher limit.
Above-mentioned steps A1-B3 be for data deposit and read the preparation of doing in, the parameters such as response time of application server port bandwidth and all kinds of storage mediums can be set when system initialization by the management software by system.After calculating the data volume higher limit that all kinds of hard disks deposit at every turn, the IPSAN Control Server just can be deposited in different hard disks with file fragmentation according to this data value.
When C1. write data requests occurring, whether the data volume of the file that judgement will deposit in greater than each data writing amount higher limit of SSD hard disk, is that execution in step C2, no, execution in step C3;
Data with each data writing amount higher limit equivalent of SSD hard disk in the file that C2. will deposit in are deposited into the SSD hard disk; Remaining data in file are deposited into the SAS hard disk; If file data also has residue, the remaining data in file is deposited into the SATA hard disk.
C3. the judgement file that need to deposit in whether be read frequency higher than predeterminated frequency, be, this document is deposited into the SSD hard disk, no, be deposited into SAS hard disk and/or SATA hard disk.
When write data requests occurring, having some small documents need to deposit in, the size of the file that at first the IPSAN Control Server will deposit in and the first data volume are relatively, when the data volume of the file that will deposit in is less than or equal to the first data volume, according to the practical application scene of small documents, small documents is stored in suitable storage medium.As a kind of embodiment, small documents just can be come the deposit position of regulation small documents according to accessed frequency, what accessed frequency was high deposits in the fast storage medium of response, and the small documents that accessed frequency is low is deposited in the storage medium of low-response.
For the file of data volume greater than the first data volume, the data with the first data volume in file deposit the first storage medium in, deposit the data of the second data volume in the middle of remaining data in file in second storage medium, by that analogy.If there is not the 4th storage medium, deposit the remaining data in file in the 3rd storage medium.
When D. read data request occurring, send simultaneously or concentrate the transmission reading order to the storage medium that has required file data.
Suppose that the readwrite bandwidth between application server and hard disk array is 1Gbps, valid data transmission efficiency is 88%, within the time (approximately 4ms) of SAS hard disk tracking and internal data transfer end, can transmit the approximately data of 430KB.And within the time (approximately 12ms) of SATA hard disk tracking and internal data transfer end, can transmit the approximately data of 1.26MB.In the fastest SSD hard disk of response speed, and follow-up data 830KB (being that 4ms is to the data that should transmit between 12ms) leaves in the SAS hard disk the deposit data of the front 430KB of this document, and the remainder data of this document leaves on the SATA hard disk.
After data keep, assign the reading order of data by the Ethernet interface that connects storage system when application server, this order is stored the IPSAN Control Server of internal system and understands, and be mapped to real LBA (Logical Block Address on physical hard disk by the request that the virtualization engine that is arranged on server will read and write data, LBA), then the address after these mappings is returned to application server, application server is the address concrete according to these, with reading order to concentrate the disposable three kinds of hard disks that mail to.
SAS hard disk and SATA hard disk carry out the magnetic head tracking immediately after receiving order.Due to the outstanding data response speed of SSD hard disk, it can provide data at first.After the SSD hard disk had passed its data (430KB), the SAS hard disk had been ready to leave in the 830KB above it, and transmission immediately.After the SAS hard disk groups had passed data, the SATA hard disk also had been ready to remaining data, continued the transmission of data.
Like this, the experience that the application server end obtains is: when it will read the data of a file, order once the delay of sending through approximately tens of microseconds the data that just can want with the wide continuous acquisition of filled band.So effectively avoid transmission latency of causing because of SAS and SATA hard disc magnetic head seek time, greatly improved the performance of storage system.
The present invention also provides a kind of file data read-write equipment, comprise and set in advance module, be positioned on Control Server, be used for the existing different types of storage medium of storage system is arranged from short to long according to response time, calculate successively the differential time of the response time of the response time of current kind storage medium and next class storage medium; Calculate successively the wide data volume that can transmit of filled band in described differential time, result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves; Writing module is positioned on application server, and when being used for write data requests occurring, the data in the file that will deposit according to each data writing amount higher limit of every kind of storage medium are deposited into storage medium successively; Read module is positioned on application server, when being used for read data request occurring, sends simultaneously or concentrate the transmission reading order to the storage medium that has required file data.
As a kind of embodiment, the described module that sets in advance comprises: the differential time acquisition module, be used for the existing different types of storage medium of storage system is arranged from short to long according to response time, calculate successively the differential time of the response time of the response time of current kind storage medium and next class storage medium; Set the bandwidth module, be used for setting application server towards the bandwidth value that reads and writes data of storage medium; Computing module is used for calculating the wide data volume that can transmit of filled band in described differential time; Memory module is used for result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves.
A kind of file data storing provided by the invention and the method and apparatus that reads, the allocation ratio of various storage mediums is identical in system, namely under the prerequisite suitable with the cost of existing storage system, there is more file to obtain the lifting of read-write response speed, thereby significantly promotes the whole input and output performance of storage system.
Should be noted that at last obviously, those skilled in the art can carry out various changes and modification and not break away from the spirit and scope of the present invention the present invention.Like this, if within of the present invention these were revised and modification belongs to the scope of claim of the present invention and equivalent technologies thereof, the present invention also was intended to comprise these changes and modification.
Claims (8)
1. a file data storing and read method, is characterized in that, comprises step:
A. existing different types of storage medium in storage system is arranged from short to long according to response time, calculated successively the differential time of the response time of the response time of current kind storage medium and next class storage medium;
B. calculate successively the data volume of the wide energy transmission of filled band in described differential time, result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves, come the storage medium of last position without each data writing amount higher limit;
When C. write data requests occurring, the data in the file that will deposit according to each data writing amount higher limit of every kind of storage medium are deposited into storage medium successively;
When D. read data request occurring, send simultaneously or concentrate the transmission reading order to the storage medium that has required file data;
The response time of storage medium is for receiving after read write command to the length of the time of delay of sending desired data from storage medium.
2. method according to claim 1, is characterized in that, described step B comprises:
B1. set application server towards the bandwidth value that reads and writes data of storage medium;
B2. calculate in described differential time the data volume that can transmit during the wide transmission of data bandwidth filled band;
B3. result of calculation be set as each data writing amount higher limit of corresponding current storage medium and preserve.
3. method according to claim 1, is characterized in that, described step C comprises:
When C1. write data requests occurring, whether the data volume of the file that judgement will deposit in greater than the each data writing amount of the storage medium that ranks the first higher limit, is that execution in step C2, no, execution in step C3;
C2. according to each data writing amount higher limit of every kind of storage medium, the data of the file that will deposit in are deposited into two or more storage medium successively;
C3. the judgement file that need to deposit in whether be read frequency higher than predeterminated frequency, be, this document is deposited into the storage medium that ranks the first, no, be deposited into other storage mediums.
4. method according to claim 1, is characterized in that, when described storage medium comprised SSD hard disk, SAS hard disk and SATA hard disk, described steps A comprised:
A1. three kinds of hard disks are arranged from short to long according to response time, rank results is SSD hard disk, SAS hard disk, SATA hard disk;
A2. calculate the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk; Calculate the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
5. method according to claim 4, is characterized in that, described step B comprises:
B1. set application server towards the bandwidth value that reads and writes data of the storage array of three kinds of hard disks compositions;
B2. calculate the wide data volume that can transmission of in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk filled band; The wide data volume that can transmission of calculating filled band in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk;
B3. will be in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk the wide data volume that can transmission of filled band be set as each data writing amount higher limit of SSD hard disk; Will be in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk the wide data volume that can transmission of filled band be set as each data writing amount higher limit of SAS hard disk, the SATA hard disk is without each data writing amount higher limit.
6. method according to claim 4, is characterized in that, described step C comprises:
When C1. write data requests occurring, whether the data volume of the file that judgement will deposit in greater than each data writing amount higher limit of SSD hard disk, is that execution in step C2, no, execution in step C3;
Data with each data writing amount higher limit equivalent of SSD hard disk in the file that C2. will deposit in are deposited into the SSD hard disk; Remaining data in file are deposited into the SAS hard disk; If file data also has residue, the remaining data in file is deposited into the SATA hard disk;
C3. the judgement file that need to deposit in whether be read frequency higher than predeterminated frequency, be, this document is deposited into the SSD hard disk, no, be deposited into SAS hard disk and/or SATA hard disk.
7. a file data storing and reading device, is characterized in that, comprising:
Set in advance module, be positioned on Control Server, be used for the existing different types of storage medium of storage system is arranged from short to long according to response time, calculate successively the differential time of the response time of the response time of current kind storage medium and next class storage medium; Calculate successively the wide data volume that can transmit of filled band in described differential time, result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves;
Writing module is positioned on application server, and when being used for write data requests occurring, the data in the file that will deposit according to each data writing amount higher limit of every kind of storage medium are deposited into storage medium successively;
Read module is positioned on application server, when being used for read data request occurring, sends simultaneously or concentrate the transmission reading order to the storage medium that has required file data;
The response time of storage medium is for receiving after read write command to the length of the time of delay of sending desired data from storage medium.
8. device according to claim 7, is characterized in that, the described module that sets in advance comprises:
The differential time acquisition module, be used for the existing different types of storage medium of storage system is arranged from short to long according to response time, calculate successively the differential time of the response time of the response time of current kind storage medium and next class storage medium;
Set the bandwidth module, be used for setting application server towards the bandwidth value that reads and writes data of storage medium;
Computing module is used for calculating the wide data volume that can transmit of filled band in described differential time;
Memory module is used for result of calculation is set as each data writing amount higher limit of corresponding current storage medium and preserves.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110189501 CN102263818B (en) | 2011-07-07 | 2011-07-07 | Method for storing and reading file data, and apparatus thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110189501 CN102263818B (en) | 2011-07-07 | 2011-07-07 | Method for storing and reading file data, and apparatus thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102263818A CN102263818A (en) | 2011-11-30 |
| CN102263818B true CN102263818B (en) | 2013-06-05 |
Family
ID=45010266
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110189501 Active CN102263818B (en) | 2011-07-07 | 2011-07-07 | Method for storing and reading file data, and apparatus thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102263818B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106050A (en) * | 2013-02-22 | 2013-05-15 | 浪潮电子信息产业股份有限公司 | Method for achieving layered storage and copy of data of storage system |
| CN103605623A (en) * | 2013-10-31 | 2014-02-26 | 北京智谷睿拓技术服务有限公司 | Memory device reading-writing control method and reading-writing control device |
| CN105843819B (en) * | 2015-01-15 | 2020-07-10 | 中国移动通信集团河南有限公司 | Data export method and device |
| CN111399748B (en) * | 2019-01-02 | 2023-09-05 | 中国移动通信有限公司研究院 | Data placement method, device and computer readable storage medium |
| CN111641709B (en) * | 2020-05-29 | 2022-12-27 | 苏州浪潮智能科技有限公司 | Bandwidth control method, system, device and medium for distributed system |
| CN112331253B (en) * | 2020-10-30 | 2023-12-08 | 深圳市宏旺微电子有限公司 | Chip testing method, terminal and storage medium |
| CN115934002B (en) * | 2023-03-08 | 2023-08-04 | 阿里巴巴（中国）有限公司 | Solid state disk access method, solid state disk, storage system and cloud server |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6993624B2 (en) * | 2002-10-31 | 2006-01-31 | Hitachi Global Storage Technologies Netherlands B.V. | Method and apparatus for selecting enqueued data access operations for execution in a disk drive data storage device |
| CN1674538A (en) * | 2004-03-25 | 2005-09-28 | 联想(北京)有限公司 | Network storing system based on local network and its method for reading and writing data |
| JP4413184B2 (en) * | 2005-11-28 | 2010-02-10 | 富士通株式会社 | Data storage system and data storage control device |
| CN101118477A (en) * | 2007-08-24 | 2008-02-06 | 成都索贝数码科技股份有限公司 | Process for enhancing magnetic disc data accessing efficiency |
-
2011
- 2011-07-07 CN CN 201110189501 patent/CN102263818B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN102263818A (en) | 2011-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102263818B (en) | Method for storing and reading file data, and apparatus thereof | |
| US7949847B2 (en) | Storage extent allocation method for thin provisioning storage | |
| Balakrishnan et al. | Pelican: A building block for exascale cold data storage | |
| CN102023813B (en) | Application and tier configuration management in dynamic page realloction storage system | |
| US8284627B2 (en) | Reducing energy consumption and optimizing workload and performance in multi-tier storage systems using extent-level dynamic tiering | |
| US20180260135A1 (en) | Integrated heterogeneous solid state storage drive | |
| US11748037B2 (en) | Physical disk and virtual disk mapping in storage systems | |
| JP5363595B2 (en) | Storage system and method for rearranging data in virtual volume | |
| US20130318196A1 (en) | Storage system and storage control method for using storage area based on secondary storage as cache area | |
| CN101566927B (en) | Memory system, memory controller and data caching method | |
| CN111722786A (en) | Storage system based on NVMe equipment | |
| CN102123176B (en) | Space distribution and management method and device for network storage system | |
| US20130151762A1 (en) | Storage device | |
| US9311207B1 (en) | Data storage system optimizations in a multi-tiered environment | |
| CN103049216B (en) | Solid state hard disc and data processing method, system | |
| CN107092442A (en) | Storage system resources distribution method and device | |
| US9747040B1 (en) | Method and system for machine learning for write command selection based on technology feedback | |
| US8904119B2 (en) | Method and structures for performing a migration of a logical volume with a serial attached SCSI expander | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| US9069471B2 (en) | Passing hint of page allocation of thin provisioning with multiple virtual volumes fit to parallel data access | |
| CN101997919B (en) | Storage resource management method and device | |
| CN103827837A (en) | Method, device and system for accessing solid state disk space | |
| CN105204787B (en) | A kind of data of PCIE SSD array reads method, system and reading/writing method | |
| US9015410B2 (en) | Storage control apparatus unit and storage system comprising multiple storage control apparatus units | |
| US20180239700A1 (en) | Selection and placement of volumes in a storage system using stripes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170821 Address after: 100192, A01, room 10, block 8, science and technology wealth center, A, clear road, Haidian District, Beijing Patentee after: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Address before: 100192, Beijing, Haidian District School Road 8 (Science and technology wealth center) A block, 10 floor, South District Co-patentee before: BEIJING TEAMSUN TECHNOLOGY Co.,Ltd. Patentee before: Beijing Fortunet Information Technology Co.,Ltd. |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20111130 Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Contract record no.: X2021980013200 Denomination of invention: A method and device for storing and reading file data Granted publication date: 20130605 License type: Exclusive License Record date: 20211124 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and device for storing and reading file data Effective date of registration: 20211125 Granted publication date: 20130605 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Registration number: Y2021110000072 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20230129 Granted publication date: 20130605 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Registration number: Y2021110000072 |
|
| EC01 | Cancellation of recordation of patent licensing contract | ||
| EC01 | Cancellation of recordation of patent licensing contract |
Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Contract record no.: X2021980013200 Date of cancellation: 20230321 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20111130 Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Contract record no.: X2023110000047 Denomination of invention: A method and device for storing and reading file data Granted publication date: 20130605 License type: Exclusive License Record date: 20230323 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and device for storing and reading file data Effective date of registration: 20230327 Granted publication date: 20130605 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Registration number: Y2023110000128 |