CN102263818A - Method for storing and reading file data, and apparatus thereof - Google Patents
Method for storing and reading file data, and apparatus thereof Download PDFInfo
- Publication number
- CN102263818A CN102263818A CN2011101895015A CN201110189501A CN102263818A CN 102263818 A CN102263818 A CN 102263818A CN 2011101895015 A CN2011101895015 A CN 2011101895015A CN 201110189501 A CN201110189501 A CN 201110189501A CN 102263818 A CN102263818 A CN 102263818A
- Authority
- CN
- China
- Prior art keywords
- data
- storage medium
- hard disk
- file
- response time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000003860 storage Methods 0.000 claims abstract description 194
- 230000004044 response Effects 0.000 claims abstract description 106
- 230000005540 biological transmission Effects 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 11
- 239000012141 concentrate Substances 0.000 claims description 8
- 239000000203 mixture Substances 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 7
- 238000012546 transfer Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000000151 deposition Methods 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for storing and reading file data. The method comprises the following steps: arranging storage mediums according to response delay time, wherein an arrangement order is from short to long, and calculating differential time of the two adjacent storage mediums successively; calculating successively a data quantity transmitted under a full bandwidth in the differential time, setting a calculated result as an upper limit value of each write-in data quantity of the corresponding current storage medium and storing; successively storing data in a file to be stored to the storage medium according to the upper limit value of each write-in data quantity of each storage medium when a written data request appears; simultaneously sending or concentratedly sending a reading command to the storage medium which stores the required file data when a reading data request appears. The invention also discloses an apparatus of storing and reading the file data. The apparatus comprises a pre-set module, a write-in module and a reading module. By using the method and the apparatus of the invention, input and output performance of a storage system can be greatly raised without increasing costs.
Description
Technical field
The present invention relates to technical field of data storage, the method and apparatus that particularly relates to a kind of file data storing and read.
Background technology
Storage system is meant in the computer by the equipment (hardware) of the various memory devices of depositing program and data, control assembly and management information scheduling and the system that algorithm (software) is formed.From application point of view, a storage system should accomplish to deal with the fastest response speed user's read-write requests as possible.Therefore data write speed just seems very meaningful for weighing a performance of storage system.And the non-volatile medium of depositing such as hard disk as a kind of memory device of extensive use, has considerable influence to this performance of the speed that reads and writes data of storage system.
At present, the non-volatile medium of depositing of data mainly contains SATA (Serial Advanced Technology Attachment, Serial Advanced Technology Attachment) hard disk, SAS (Serial Attached SCSI, Serial Attached SCSI (SAS)) hard disk and SSD (Solid State Disk, solid state hard disc) hard disk etc.The rotating speed of SATA hard disk is 7200 rev/mins at present, 8 to 12 milliseconds of the seek time average out to of magnetic head, internal data transfer speed average out to 110MB/ second.And the rotating speed of SAS hard disk is at 10000 rev/mins to 15000 rev/mins, 2 to 4 milliseconds of the seek time average out to of magnetic head, and internal transmission speed is suitable with the SATA hard disk.And there is not the seek time of magnetic head in the SSD hard disk owing to adopt semiconductor technology, thereby read or write speed can reach very high, and it only is tens of microseconds that read-write postpones.
All kinds of hard disks are because self performance characteristics, the response speed difference of the order that the user is sent, and the magnetic head seek time is short more, internal data transfer is fast more, Gao Ze is short more time of delay more for rotating speed, responds soon more.The SSD hard disk is better than SAS hard disk and SATA hard disk on these performance index of read-write response, the SAS hard disk is better than the SATA hard disk.
Traditional SAN (Storage Area Network based on optical-fibre channel, storage area network) in the storage system, data write be SAN Control Server by rack inside on behalf of carrying out, the multi-section application server is shared the readwrite bandwidth between SAN Control Server and hard disk array.Because application server is not direct and hard disk array is come into contacts with, therefore can't decide the layering deposit method of file to the bandwidth of data read-write according to application server.
At the SAN storage system of this quasi-tradition, data hierarchy memory technology that storage manufacturer adopts is based on " cold " and " heat " degree of file.The file of so-called " heat " is the higher file of access frequency, and the file of " cold " is the lower file of access frequency.In the present storage system way of main flow be with the file migration of " heat " to reading and writing faster on the storage medium, as the SSD hard disk, the file of " cold " is moved on the slower storage medium of read-write, as the SATA hard disk.By this lasting migration, make active file when being read, can obtain preferential response.Thereby the performance that improves the user is experienced.
In the middle of reality, kind of document is various, the visit situation is numerous and complicated mixed and disorderly, be difficult to find comparatively reasonably access frequency " heat " and " cold " of dividing file, and, though this kind mode can obviously improve the response speed of system to the file of " heat ",, still there is long problem time of delay of response for the file of " cold ".And " cold " though the access frequency of the single file of file is not high, because short hard disk price comparison height and capacity of response time is less, enterprise is because cost consideration, and the number of files that is defined as " cold " file is a large amount of often.Therefore the number of files that adopts existing this storage mode, any time can obtain this speed of read-write requests faster is very limited.
Summary of the invention
The method and apparatus that the object of the present invention is to provide a kind of file data storing and read, solve data hierarchy method of the prior art and be merely able to promote the very problem of the read-write response speed of finite population file, make storage medium dispose under the identical condition, there is more file to obtain the lifting of read-write response speed, thereby significantly promotes the whole input and output performance of storage system.
Be a kind of file data storing and the read method of realizing that the object of the invention provides, comprise step: A. arranges the storage medium of existing and heterogeneous in the storage system from short to long according to response time, calculates the differential time of the response time of the response time of current kind storage medium and next class storage medium successively; B. calculating in described differential time the wide institute of filled band successively can data quantity transmitted, and result of calculation is set at writing the data volume higher limit at every turn and preserving of corresponding current storage medium, and the storage medium that comes the position, end does not have and writes the data volume higher limit at every turn; When C. write data requests occurring, be deposited into storage medium successively according to the data that write in the file that the data volume higher limit will deposit in of every kind of storage medium at every turn; When D. read data request occurring, send or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
Wherein, described step B comprises: B1. sets the read and write data bandwidth value of application server towards storage medium; B2. calculate in described differential time, institute can data quantity transmitted during the wide transmission of described data bandwidth filled band; B3. the data volume higher limit that at every turn writes that result of calculation is set at corresponding current storage medium is also preserved.
Wherein, described step C comprises: when write data requests appearred in C1., whether the data volume of the file that judgement will deposit in write the data volume higher limit greater than the storage medium that ranks the first at every turn, be, execution in step C2 then, not, execution in step C3 then; C2. according to every kind of storage medium write the data volume higher limit at every turn, the data of the file that will deposit in are deposited into two or more storage medium successively; C3. judge being read frequency and whether being higher than predeterminated frequency of the file that to deposit in, be, then this document is deposited into the storage medium that ranks the first, not, then be deposited into other storage mediums.
Wherein, when described storage medium comprised SSD hard disk, SAS hard disk and SATA hard disk, then described steps A comprised: A1. arranges three kinds of hard disks from short to long according to response time, and rank results is SSD hard disk, SAS hard disk, SATA hard disk; A2. calculate the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk; Calculate the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
Wherein, described step B comprises: B1. sets the read and write data bandwidth value of application server towards the storage array of three kinds of hard disks compositions; B2. calculating in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk the wide institute of filled band can data quantity transmitted; The wide institute of calculating filled band in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk can data quantity transmitted; B3. will be in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk filled band wide can data quantity transmitted be set at the SSD hard disk write the data volume higher limit at every turn; Will be in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk filled band wide can data quantity transmitted be set at the SAS hard disk write the data volume higher limit at every turn, the SATA hard disk does not have and writes the data volume higher limit at every turn.
Wherein, described step C comprises: when write data requests appears in C1., the data volume of the file that judgement will deposit in whether greater than the SSD hard disk write the data volume higher limit at every turn, be, execution in step C2 then, not, execution in step C3 then; C2. be deposited into the SSD hard disk with the data that write data volume higher limit equivalent of SSD hard disk at every turn in the file that will deposit in; Remaining data in the file are deposited into the SAS hard disk; If file data also has residue, then the remaining data in the file is deposited into the SATA hard disk; C3. judge being read frequency and whether being higher than predeterminated frequency of the file that to deposit in, be, then this document is deposited into the SSD hard disk, not, then be deposited into SAS hard disk or SATA hard disk.
The present invention also provides a kind of file data storing and reading device, comprise: set in advance module, be positioned on the Control Server, be used for the storage medium of storage system existing and heterogeneous is arranged from short to long according to response time, calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium successively; Calculate the wide institute of filled band energy data quantity transmitted in described differential time successively, the data volume higher limit that at every turn writes that result of calculation is set at corresponding current storage medium is also preserved; Writing module is positioned on the application server, when being used to write data requests occur, is deposited into storage medium successively according to the data that write in the file that the data volume higher limit will deposit in of every kind of storage medium at every turn; Read module is positioned on the application server, when being used to read data request occur, sends or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
Wherein, the described module that sets in advance comprises: the differential time acquisition module, be used for the storage medium of storage system existing and heterogeneous is arranged from short to long according to response time, calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium successively; Set the bandwidth module, be used to set the read and write data bandwidth value of application server towards storage medium; Computing module is used for calculating the wide institute of filled band energy data quantity transmitted in described differential time; Memory module is used for result of calculation is set at writing the data volume higher limit at every turn and preserving of corresponding current storage medium.
The invention has the beneficial effects as follows: a kind of file data reading/writing method and device that the present invention describes, be stored in the different storage mediums by data sementation a file, make application server after sending read-write requests simultaneously to several different storage mediums, carry out at other storage mediums in the time of work such as tracking and internal data transfer, the fastest storage medium of response speed takes the lead in desired data is sent, then response speed time fast storage medium also has been ready to the data that will send, by that analogy, this kind location mode can provide at once for the request of reading of application server, the data response that filled band is wide.
Description of drawings
Fig. 1 is the flow chart of a kind of file data storing provided by the invention and read method;
Fig. 2 is the structure chart that can use the storage system of a kind of file data storing provided by the invention and read method;
Fig. 3 is the flow chart of a kind of embodiment of a kind of file data storing provided by the invention and read method.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer,, a kind of file data storing of the present invention and the method and apparatus that reads are further elaborated below in conjunction with drawings and Examples.Should be appreciated that specific embodiment described herein only in order to explanation the present invention, and be not used in qualification the present invention.
Because when reading writing harddisk, the factor of restriction performance mainly is the seek time of magnetic head and the data transmission bauds of hard disk inside.Therefore, aim of the present invention is when the server write data, can be according to the response time and the read or write speed difference of various storage mediums, in conjunction with the readwrite bandwidth between application server and hard disk array, calculate the size of data that on various storage mediums, should deposit and carry out corresponding striping processing, so that can be when reading by the relay service of different storage mediums, for reading of application server asks to provide data response at once, that filled band is wide.This calculating and data sementation are deposited necessary address allocation procedure and are carried out on the IPSAN of storage system inside Control Server.
A kind of file data reading/writing method provided by the invention as shown in Figure 1, comprises the steps:
A. the storage medium of existing and heterogeneous in the storage system is arranged from short to long according to response time, calculated the differential time of the response time of the response time of current kind storage medium and next class storage medium successively.
The response time of storage medium is from storage medium and receives behind the read write command that to the length of the time of delay of sending desired data, time of delay is long more, and then response speed is slow more.The response time of hard disk depends primarily on the seek time of magnetic head and the data transmission bauds of hard disk inside.
After the ordering, be first storage medium, second storage medium, the 3rd storage medium according to all kinds of storage mediums of ordering definition
Calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium, the time difference of just at first calculating the response time of first storage medium and second storage medium, obtain first differential time; The response time of calculating second storage medium and the 3rd storage medium then is poor, obtains second differential time; Calculate the time difference of the 3rd storage medium and the 4th storage medium again ... when not having the 4th storage medium, then do not calculate.
B. calculating in described differential time the wide institute of filled band successively can data quantity transmitted, and result of calculation is set at writing the data volume higher limit at every turn and preserving of corresponding current storage medium, and the storage medium that comes the position, end does not have and writes the data volume higher limit at every turn.
Bandwidth is the call parameter of calculated data amount, should set in advance the port bandwidth of each application server towards storage medium on the IPSAN Control Server.
Calculating in first differential time, the data volume that can transmit during bandwidth fixed, defining this data volume is first data volume, first data volume=bandwidth * first differential time; Calculate second data volume again, second data volume=bandwidth * second differential time ... in the middle of practical application, also should consider the problem of valid data transmission efficiency, at this moment, first data volume=first differential time * bandwidth * the valid data transmission efficiency, the second data volume=second differential time * bandwidth * valid data transmission efficiency, the rest may be inferred, can calculate the data volume in other differential times, repeat no more.
On the IPSAN Control Server, write the data volume higher limit with what each data volume of calculating gained was designated as corresponding storage medium at every turn, and it is saved as static data, as the limit standard that writes data volume during to different storage medium with storage.So-called higher limit, the data volume that promptly at every turn is written to this kind storage medium can not be more than this higher limit.
When C. write data requests occurring, be deposited into storage medium successively according to the data that write in the file that the data volume higher limit will deposit in of every kind of storage medium at every turn.
Because every kind of storage medium has all been set and has been write the data volume higher limit accordingly at every turn, therefore file is deposited in the process of different storage mediums, also just data have been split into a plurality of data segments.For example, if the data volume that a file arranged writes data volume higher limit sum greater than the writing data volume (first data volume) higher limit of first storage medium at every turn less than first storage medium and second storage medium at every turn, then in the process that writes, the data that at every turn write data volume higher limit equivalent of the front in this document and first storage medium are deposited into first storage medium, remaining data volume writes second storage medium, and file just has been divided into two data segments and stores like this.If the data volume of certain file is bigger, greater than first storage medium and second storage medium write data volume higher limit sum at every turn, then the remaining data volume of this document is deposited into the 3rd storage medium, like this, file just has been divided into three data segments and has stored.If in the storage system three kinds of storage mediums are only arranged, then the 3rd storage medium is not set and is write the data volume higher limit at every turn.
In the middle of practical application, in order to make full use of application server reading and writing data bandwidth, the data that identical file is deposited in the storage medium of the same race can be deposited in different memory cell through striping, as the reading and writing data bandwidth of supposing application server is 10Gbps, and first storage medium is the SSD hard disk, it is 1Gbps by the bandwidth that network connects, first data volume is 4.3MB as calculated, then the data of the preceding 4.3MB of a certain file can leave 10 or more on the polylith SSD hard disk (bandwidth of 10 or more a plurality of 1Gbps merge reach the above bandwidth of 10Gbps) in through striping, when the transmission data, these SSD hard disks transmit data simultaneously, make the application server readwrite bandwidth fully to be occupied and to utilize.
When D. read data request occurring, send or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
When read data request occurring, should guarantee that each memory cell is to receive reading order simultaneously in all kinds of storage mediums and all kinds of storage medium.But, in actual applications, because application server can't accomplish to send simultaneously many reading orders, can only be one by one, the concentrated area sends reading order, just each reading order sends in succession, but the time of two adjacent order meeting machine cycles of interval.Because this machine cycle is very of short duration, can ignore, reading order almost arrives all kinds of storage mediums simultaneously.
Almost receive under the prerequisite of read write command simultaneously at all kinds of storage mediums, because the response speed of first storage medium is the fastest, then through the response time of first storage medium, first storage medium sends the data of first data volume in the middle of the required file, behind first differential time, the transfer of data of first data volume finishes.At this moment, apart from the moment that all kinds of storage mediums are almost received read write command simultaneously, elapsed time length is the response time and the first differential time sum of first storage medium, and the response time of first storage medium and the first differential time sum are the response time of second storage medium, that is to say from the moment of receiving read write command and count, passed through the response time of second storage medium, at this moment, second storage medium is with the DSR of second data volume of required file and send to application server.When the data of second data volume are transmitted when finishing, then the response time of the 3rd storage medium arrives, and the 3rd storage medium sends the data of the 3rd data volume, by that analogy, and up to the data in the file are all sent.
Therefore, after sending the order that reads or writes to the data of certain file, only through the response time of first storage medium, needed data will wide, continuous the sending of filled band, if the performance of first storage medium is enough superior, then the response time of first storage medium can be ignored, and promptly application server can obtain wide, the continuous data of filled band at once after sending read write command.
Referring to Fig. 2, Fig. 2 is for using the schematic diagram of a kind of storage system of the present invention.By the direct communication of Ethernet switch, be point-to-point interconnect architecture between the application server of this system and the storage medium array, have fixing bandwidth.
On such system architecture, the storage medium of storage system inside can have SSD hard disk, SAS hard disk and SATA hard disk.Technical scheme of the present invention is applied to this storage system, and then storage medium is defined as SATA hard disk, SAS hard disk and SSD hard disk, and as shown in Figure 3, technical scheme of the present invention comprises:
A1. SATA hard disk, SAS hard disk and SSD hard disk are arranged from short to long according to response time.
The SSD hard disk is owing to adopt semiconductor technology, the seek time that does not have magnetic head, it is very high that read or write speed can reach, it only is tens of microseconds that read-write postpones, and the rotating speed of SAS hard disk is at 10000 rev/mins to 15000 rev/mins, 2 to 4 milliseconds of the seek time average out to of magnetic head, internal transmission speed is suitable with the SATA hard disk.The SATA hard disk gets about 8 to 12 milliseconds of seek time average out to of magnetic head, internal data transfer speed average out to 110MB/ second.Therefore the SSD hard disk should come at first, the SAS hard disk secondly, the result of arrangement is: SSD hard disk, SAS hard disk, SATA hard disk.
A2. calculate first differential time of the response time of the response time of described SSD hard disk and described SAS hard disk; Calculate second differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
The response time of supposing the SAS hard disk is 4ms (millisecond), and the response time of SATA is 12ms, and the response time of SSD hard disk is ignored, first differential time=4ms-0ms=4ms then, second differential time=12ms-4ms=8ms.
B1. set the bandwidth value that reads and writes data of application server towards the storage array of three kinds of hard disks compositions.
B2. calculating in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk the wide institute of filled band can data quantity transmitted; The wide institute of calculating filled band in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk can data quantity transmitted.
Calculating in the first differential time 4ms the wide institute of filled band can data quantity transmitted, and this data volume is first data volume, first data volume=4ms* bandwidth.In the middle of practical application, also should consider the problem of valid data transmission efficiency, at this moment first data volume=4ms* bandwidth * valid data transmission efficiency.Calculating in the second differential time 8ms the wide institute of filled band can data quantity transmitted, and this data volume is second data volume, second data volume=8ms* bandwidth * valid data transmission efficiency.
B3. will be in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk filled band wide can data quantity transmitted be set at the SSD hard disk write the data volume higher limit at every turn; Will be in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk filled band wide can data quantity transmitted be set at the SAS hard disk write the data volume higher limit at every turn, the SATA hard disk does not have and writes the data volume higher limit at every turn.
Above-mentioned steps A1-B3 be for data deposit and read the preparation of being done in, the parameters such as response time of application server port bandwidth and all kinds of storage mediums can be set when system initialization by the management software by system.Calculate after the data volume higher limit that all kinds of hard disks deposit at every turn, the IPSAN Control Server just can be deposited in different hard disks with file fragmentation according to this data value.
When C1. write data requests occurring, the data volume of the file that judgement will deposit in whether greater than the SSD hard disk write the data volume higher limit at every turn, be, execution in step C2 then, not, execution in step C3 then;
C2. be deposited into the SSD hard disk with the data that write data volume higher limit equivalent of SSD hard disk at every turn in the file that will deposit in; Remaining data in the file are deposited into the SAS hard disk; If file data also has residue, then the remaining data in the file is deposited into the SATA hard disk.
C3. judge being read frequency and whether being higher than predeterminated frequency of the file that to deposit in, be, then this document is deposited into the SSD hard disk, not, then be deposited into SAS hard disk and/or SATA hard disk.
When write data requests occurring, having some small documents need deposit in, the size of the file that the IPSAN Control Server at first will deposit in and first data volume are relatively, when the data volume of the file that will deposit in is less than or equal to first data volume, then, small documents is stored in the suitable storage medium according to the practical application scene of small documents.As a kind of embodiment, small documents just can be come the deposit position of regulation small documents according to accessed frequency, what accessed frequency was high then deposits in the fast storage medium of response, and the small documents that accessed frequency is low is then deposited in the storage medium of low-response.
For the file of data volume greater than first data volume, then the data with first data volume in the file deposit first storage medium in, deposit the data of second data volume in the middle of the remaining data in the file in second storage medium, by that analogy.If there is not the 4th storage medium, then deposit the remaining data in the file in the 3rd storage medium.
When D. read data request occurring, send or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
Suppose that the readwrite bandwidth between application server and hard disk array is 1Gbps, valid data transmission efficiency is 88%, then in the time (about 4ms) of SAS hard disk tracking and internal data transfer end, can transmit the data of about 430KB.And in the time (about 12ms) of SATA hard disk tracking and internal data transfer end, can transmit the data of about 1.26MB.In the fastest SSD hard disk of response speed, and follow-up data 830KB (being that 4ms is to the data that should transmit between the 12ms) leaves in the SAS hard disk the deposit data of the preceding 430KB of this document, and the remainder data of this document leaves on the SATA hard disk.
After data keep, assign the reading order of data by the Ethernet interface that connects storage system when application server, this order is stored the IPSAN Control Server of internal system and understands, and be mapped on the physical hard disk real LBA (Logical Block Address by the request that the virtualization engine that is installed on the server will read and write data, LBA), then the address after these mappings is returned to application server, application server is then according to these concrete addresses, with reading order to concentrate the disposable three kinds of hard disks that mail to.
SAS hard disk and SATA hard disk carry out the magnetic head tracking immediately after receiving order.Because the outstanding data response speed of SSD hard disk, it can provide data at first.After the SSD hard disk had passed its data (430KB), the SAS hard disk had been ready to leave in the 830KB above it, and transmission immediately.After the SAS hard disk groups had passed data, the SATA hard disk also had been ready to remaining data, continued the transmission of data.
Like this, the experience that the application server end obtains is: when it will read the data of a file, order once sending, and through the delay of tens of approximately microseconds, the data that can want with the wide continuous acquisition of filled band just.So effectively avoided transmission latency of causing, improved the performance of storage system greatly because of SAS and SATA hard disc magnetic head seek time.
The present invention also provides a kind of file data read-write equipment, comprise and set in advance module, be positioned on the Control Server, be used for the storage medium of storage system existing and heterogeneous is arranged from short to long according to response time, calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium successively; Calculate the wide institute of filled band energy data quantity transmitted in described differential time successively, the data volume higher limit that at every turn writes that result of calculation is set at corresponding current storage medium is also preserved; Writing module is positioned on the application server, when being used to write data requests occur, is deposited into storage medium successively according to the data that write in the file that the data volume higher limit will deposit in of every kind of storage medium at every turn; Read module is positioned on the application server, when being used to read data request occur, sends or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
As a kind of embodiment, describedly set in advance module and comprise: the differential time acquisition module, be used for the storage medium of storage system existing and heterogeneous is arranged from short to long according to response time, calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium successively; Set the bandwidth module, be used to set the read and write data bandwidth value of application server towards storage medium; Computing module is used for calculating the wide institute of filled band energy data quantity transmitted in described differential time; Memory module is used for result of calculation is set at writing the data volume higher limit at every turn and preserving of corresponding current storage medium.
A kind of file data storing provided by the invention and the method and apparatus that reads, the allocation ratio of various storage mediums is identical in system, under the promptly suitable prerequisite with the cost of existing storage system, there is more file to obtain the lifting of read-write response speed, thereby significantly promotes the whole input and output performance of storage system.
Should be noted that at last that obviously those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, if of the present invention these revise and modification belongs within the scope of claim of the present invention and equivalent technologies thereof, then the present invention also is intended to comprise these changes and modification.
Claims (8)
1. file data storing and read method is characterized in that, comprise step:
A. the storage medium of existing and heterogeneous in the storage system is arranged from short to long according to response time, calculated the differential time of the response time of the response time of current kind storage medium and next class storage medium successively;
B. calculating in described differential time the wide institute of filled band successively can data quantity transmitted, and result of calculation is set at writing the data volume higher limit at every turn and preserving of corresponding current storage medium, and the storage medium that comes the position, end does not have and writes the data volume higher limit at every turn;
When C. write data requests occurring, be deposited into storage medium successively according to the data that write in the file that the data volume higher limit will deposit in of every kind of storage medium at every turn;
When D. read data request occurring, send or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
2. according to the described method of claim 1, it is characterized in that described step B comprises:
B1. set the bandwidth value that reads and writes data of application server towards storage medium;
B2. calculate in described differential time, institute can data quantity transmitted during the wide transmission of described data bandwidth filled band;
B3. the data volume higher limit that at every turn writes that result of calculation is set at corresponding current storage medium is also preserved.
3. method according to claim 1 is characterized in that, described step C comprises:
When C1. write data requests occurring, whether the data volume of the file that judgement will deposit in writes the data volume higher limit greater than the storage medium that ranks the first at every turn, be, execution in step C2 then, not, execution in step C3 then;
C2. according to every kind of storage medium write the data volume higher limit at every turn, the data of the file that will deposit in are deposited into two or more storage medium successively;
C3. judge being read frequency and whether being higher than predeterminated frequency of the file that to deposit in, be, then this document is deposited into the storage medium that ranks the first, not, then be deposited into other storage mediums.
4. method according to claim 1 is characterized in that, when described storage medium comprised SSD hard disk, SAS hard disk and SATA hard disk, then described steps A comprised:
A1. three kinds of hard disks are arranged from short to long according to response time, rank results is SSD hard disk, SAS hard disk, SATA hard disk;
A2. calculate the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk; Calculate the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk.
5. method according to claim 4 is characterized in that, described step B comprises:
B1. set the bandwidth value that reads and writes data of application server towards the storage array of three kinds of hard disks compositions;
B2. calculating in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk the wide institute of filled band can data quantity transmitted; The wide institute of calculating filled band in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk can data quantity transmitted;
B3. will be in the differential time of the response time of the response time of described SSD hard disk and described SAS hard disk filled band wide can data quantity transmitted be set at the SSD hard disk write the data volume higher limit at every turn; Will be in the differential time of the response time of the response time of described SAS hard disk and described SATA hard disk filled band wide can data quantity transmitted be set at the SAS hard disk write the data volume higher limit at every turn, the SATA hard disk does not have and writes the data volume higher limit at every turn.
6. method according to claim 4 is characterized in that, described step C comprises:
When C1. write data requests occurring, the data volume of the file that judgement will deposit in whether greater than the SSD hard disk write the data volume higher limit at every turn, be, execution in step C2 then, not, execution in step C3 then;
C2. be deposited into the SSD hard disk with the data that write data volume higher limit equivalent of SSD hard disk at every turn in the file that will deposit in; Remaining data in the file are deposited into the SAS hard disk; If file data also has residue, then the remaining data in the file is deposited into the SATA hard disk;
C3. judge being read frequency and whether being higher than predeterminated frequency of the file that to deposit in, be, then this document is deposited into the SSD hard disk, not, then be deposited into SAS hard disk and/or SATA hard disk.
7. file data storing and reading device is characterized in that, comprising:
Set in advance module, be positioned on the Control Server, be used for the storage medium of storage system existing and heterogeneous is arranged from short to long according to response time, calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium successively; Calculate the wide institute of filled band energy data quantity transmitted in described differential time successively, the data volume higher limit that at every turn writes that result of calculation is set at corresponding current storage medium is also preserved;
Writing module is positioned on the application server, when being used to write data requests occur, is deposited into storage medium successively according to the data that write in the file that the data volume higher limit will deposit in of every kind of storage medium at every turn;
Read module is positioned on the application server, when being used to read data request occur, sends or concentrate the transmission reading order simultaneously to the storage medium that has required file data.
8. device according to claim 7 is characterized in that, the described module that sets in advance comprises:
The differential time acquisition module, be used for the storage medium of storage system existing and heterogeneous is arranged from short to long according to response time, calculate the differential time of the response time of the response time of current kind storage medium and next class storage medium successively;
Set the bandwidth module, be used to set the read and write data bandwidth value of application server towards storage medium;
Computing module is used for calculating the wide institute of filled band energy data quantity transmitted in described differential time;
Memory module is used for result of calculation is set at writing the data volume higher limit at every turn and preserving of corresponding current storage medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110189501 CN102263818B (en) | 2011-07-07 | 2011-07-07 | Method for storing and reading file data, and apparatus thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110189501 CN102263818B (en) | 2011-07-07 | 2011-07-07 | Method for storing and reading file data, and apparatus thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102263818A true CN102263818A (en) | 2011-11-30 |
| CN102263818B CN102263818B (en) | 2013-06-05 |
Family
ID=45010266
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110189501 Active CN102263818B (en) | 2011-07-07 | 2011-07-07 | Method for storing and reading file data, and apparatus thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102263818B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106050A (en) * | 2013-02-22 | 2013-05-15 | 浪潮电子信息产业股份有限公司 | Method for achieving layered storage and copy of data of storage system |
| CN103605623A (en) * | 2013-10-31 | 2014-02-26 | 北京智谷睿拓技术服务有限公司 | Memory device reading-writing control method and reading-writing control device |
| CN105843819A (en) * | 2015-01-15 | 2016-08-10 | 中国移动通信集团河南有限公司 | Method and device for exporting data |
| CN111399748A (en) * | 2019-01-02 | 2020-07-10 | 中国移动通信有限公司研究院 | Data placement method and device and computer readable storage medium |
| CN111641709A (en) * | 2020-05-29 | 2020-09-08 | 苏州浪潮智能科技有限公司 | Bandwidth control method, system, device and medium for distributed system |
| CN112331253A (en) * | 2020-10-30 | 2021-02-05 | 深圳市宏旺微电子有限公司 | Chip testing method, terminal and storage medium |
| CN115934002A (en) * | 2023-03-08 | 2023-04-07 | 阿里巴巴（中国）有限公司 | Solid state disk access method, solid state disk, storage system and cloud server |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040088478A1 (en) * | 2002-10-31 | 2004-05-06 | International Business Machines Corporation | Method and apparatus for selecting enqueued data access operations for execution in a disk drive data storage device |
| CN1674538A (en) * | 2004-03-25 | 2005-09-28 | 联想(北京)有限公司 | Network storing system based on local network and its method for reading and writing data |
| US20070162561A1 (en) * | 2005-11-28 | 2007-07-12 | Fujitsu Limited | Data storage system and data storage control apparatus |
| CN101118477A (en) * | 2007-08-24 | 2008-02-06 | 成都索贝数码科技股份有限公司 | Process for enhancing magnetic disc data accessing efficiency |
-
2011
- 2011-07-07 CN CN 201110189501 patent/CN102263818B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040088478A1 (en) * | 2002-10-31 | 2004-05-06 | International Business Machines Corporation | Method and apparatus for selecting enqueued data access operations for execution in a disk drive data storage device |
| CN1674538A (en) * | 2004-03-25 | 2005-09-28 | 联想(北京)有限公司 | Network storing system based on local network and its method for reading and writing data |
| US20070162561A1 (en) * | 2005-11-28 | 2007-07-12 | Fujitsu Limited | Data storage system and data storage control apparatus |
| CN101118477A (en) * | 2007-08-24 | 2008-02-06 | 成都索贝数码科技股份有限公司 | Process for enhancing magnetic disc data accessing efficiency |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106050A (en) * | 2013-02-22 | 2013-05-15 | 浪潮电子信息产业股份有限公司 | Method for achieving layered storage and copy of data of storage system |
| CN103605623A (en) * | 2013-10-31 | 2014-02-26 | 北京智谷睿拓技术服务有限公司 | Memory device reading-writing control method and reading-writing control device |
| CN105843819A (en) * | 2015-01-15 | 2016-08-10 | 中国移动通信集团河南有限公司 | Method and device for exporting data |
| CN111399748A (en) * | 2019-01-02 | 2020-07-10 | 中国移动通信有限公司研究院 | Data placement method and device and computer readable storage medium |
| CN111399748B (en) * | 2019-01-02 | 2023-09-05 | 中国移动通信有限公司研究院 | Data placement method, device and computer readable storage medium |
| CN111641709A (en) * | 2020-05-29 | 2020-09-08 | 苏州浪潮智能科技有限公司 | Bandwidth control method, system, device and medium for distributed system |
| CN111641709B (en) * | 2020-05-29 | 2022-12-27 | 苏州浪潮智能科技有限公司 | Bandwidth control method, system, device and medium for distributed system |
| CN112331253A (en) * | 2020-10-30 | 2021-02-05 | 深圳市宏旺微电子有限公司 | Chip testing method, terminal and storage medium |
| CN112331253B (en) * | 2020-10-30 | 2023-12-08 | 深圳市宏旺微电子有限公司 | Chip testing method, terminal and storage medium |
| CN115934002A (en) * | 2023-03-08 | 2023-04-07 | 阿里巴巴（中国）有限公司 | Solid state disk access method, solid state disk, storage system and cloud server |
| CN115934002B (en) * | 2023-03-08 | 2023-08-04 | 阿里巴巴（中国）有限公司 | Solid state disk access method, solid state disk, storage system and cloud server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102263818B (en) | 2013-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102263818B (en) | Method for storing and reading file data, and apparatus thereof | |
| US7949847B2 (en) | Storage extent allocation method for thin provisioning storage | |
| CN102023813B (en) | Application and tier configuration management in dynamic page realloction storage system | |
| US8380928B1 (en) | Applying data access activity measurements | |
| US8706962B2 (en) | Multi-tier storage system configuration adviser | |
| US8924659B2 (en) | Performance improvement in flash memory accesses | |
| CN101566927B (en) | Memory system, memory controller and data caching method | |
| US11748037B2 (en) | Physical disk and virtual disk mapping in storage systems | |
| US20090282191A1 (en) | Operating Method for a Memory Subsystem and Devices for Executing the Operating Method | |
| US20130318196A1 (en) | Storage system and storage control method for using storage area based on secondary storage as cache area | |
| CN111722786A (en) | Storage system based on NVMe equipment | |
| CN102123176B (en) | Space distribution and management method and device for network storage system | |
| CN103229136B (en) | Disk array brush method and disk array brush device | |
| CN107092442A (en) | Storage system resources distribution method and device | |
| US9747040B1 (en) | Method and system for machine learning for write command selection based on technology feedback | |
| CN103049216B (en) | Solid state hard disc and data processing method, system | |
| CN100538617C (en) | Storage medium processing method and system | |
| CN105701029A (en) | Heterogeneous storage optimization method and apparatus | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| US9069471B2 (en) | Passing hint of page allocation of thin provisioning with multiple virtual volumes fit to parallel data access | |
| CN101997919A (en) | Storage resource management method and device | |
| CN105204787B (en) | A kind of data of PCIE SSD array reads method, system and reading/writing method | |
| US9015410B2 (en) | Storage control apparatus unit and storage system comprising multiple storage control apparatus units | |
| CN108628551A (en) | A kind of data processing method and device | |
| US11768628B2 (en) | Information processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170821 Address after: 100192, A01, room 10, block 8, science and technology wealth center, A, clear road, Haidian District, Beijing Patentee after: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Address before: 100192, Beijing, Haidian District School Road 8 (Science and technology wealth center) A block, 10 floor, South District Co-patentee before: BEIJING TEAMSUN TECHNOLOGY Co.,Ltd. Patentee before: Beijing Fortunet Information Technology Co.,Ltd. |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20111130 Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Contract record no.: X2021980013200 Denomination of invention: A method and device for storing and reading file data Granted publication date: 20130605 License type: Exclusive License Record date: 20211124 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and device for storing and reading file data Effective date of registration: 20211125 Granted publication date: 20130605 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Registration number: Y2021110000072 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20230129 Granted publication date: 20130605 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Registration number: Y2021110000072 |
|
| EC01 | Cancellation of recordation of patent licensing contract | ||
| EC01 | Cancellation of recordation of patent licensing contract |
Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Contract record no.: X2021980013200 Date of cancellation: 20230321 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20111130 Assignee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Assignor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Contract record no.: X2023110000047 Denomination of invention: A method and device for storing and reading file data Granted publication date: 20130605 License type: Exclusive License Record date: 20230323 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and device for storing and reading file data Effective date of registration: 20230327 Granted publication date: 20130605 Pledgee: CHINA TECHNOLOGY EXCHANGE Co.,Ltd. Pledgor: BEIJING TEAMSUN SOFTWARE TECHNOLOGY Co.,Ltd. Registration number: Y2023110000128 |