CN104731714A - Data access method and device - Google Patents
Data access method and device Download PDFInfo
- Publication number
- CN104731714A CN104731714A CN201510140432.7A CN201510140432A CN104731714A CN 104731714 A CN104731714 A CN 104731714A CN 201510140432 A CN201510140432 A CN 201510140432A CN 104731714 A CN104731714 A CN 104731714A
- Authority
- CN
- China
- Prior art keywords
- data block
- access
- storage medium
- data
- access speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data access method and device. The data access method comprises the steps that a data access request of an application program is received; a data block to which the application program requests access is looked up according to the data access request; if the data block to which the application program requests access is not found in a storage medium with higher access speed, then the data block to which the application program requests access is looked up in a storage medium with lower access speed, and the found data block is filled in the storage space occupied by a data block, needing to be weeded out, of the storage medium with the higher access speed for the application program to perform accessing. According to the data access method and device, the advantages of the mixed storage mediums are fully played, the storage cost is effectively lowered, and meanwhile the unit data access throughput and performance are improved.
Description
Technical field
The present invention relates to Internet technical field, particularly relate to a kind of data access method and device.
Background technology
In order to save storage, Internet firm progressively changes from the buying of original specialized configuration type to the buying of mixed type configuration type.Mixed type type can only limit unlike specialized configuration type and be applied in some specific application scenarios and business, therefore has more efficient utilization factor and more cheap business output cost, therefore also has more wide production application prospect.
In mixed type configuration type, in order to more business demand can be adapted to, be generally configured with large internal memory (128GB), solid state hard disc (Solid State Drive; Hereinafter referred to as: SSD) and Serial Advanced Technology Attachment (Serial AdvancedTechnology Attachment; Hereinafter referred to as: SATA).Like this, the application of different performance demand can mixed deployment together, use the storage medium of different medium separately.But, for storage-type application or Data Manipulation Mode application, because need mass data to store, SATA medium can only be adopted, but SSD or the next performance benefits brought of internal memory cannot be called; If all adopt SSD, be limited to again unit capacity cost high and have to increase the budget of additional machine.Therefore, handling up and performance of unit data access, and memory capacity is difficult to accomplish doulbe-sides' victory.
Summary of the invention
Object of the present invention is intended to solve one of technical matters in correlation technique at least to a certain extent.
For this reason, first object of the present invention is to propose a kind of data access method, and the method has given full play to the advantage of hybrid storage medium, significantly reduces the cost of storage, improves handling up and performance of unit data access simultaneously.
Second object of the present invention is to propose a kind of DAA.
To achieve these goals, the data access method of first aspect present invention embodiment, comprising: the data access request receiving application program; The data block of described application requests access is searched according to described data access request; If do not find the data block of described application requests access in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of described application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into described access speed storage medium faster, access for described application program.
The data access method of the embodiment of the present invention, after receiving the data access request of application program, if do not find the data block of above-mentioned application requests access in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of above-mentioned application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into above-mentioned access speed storage medium faster, access for above-mentioned application program, thus the advantage of hybrid storage medium can be given full play to, significantly reduce the cost of storage, improve handling up and performance of unit data access simultaneously.
To achieve these goals, the DAA of second aspect present invention embodiment, comprising: receiver module, for receiving the data access request of application program; Search module, the data access request for receiving according to described receiver module searches the data block of described application requests access; And when not finding the data block of described application requests access in access speed faster storage medium, in the storage medium that access speed is slower, search the data block of described application requests access; Packing module, for by described search module searches to the data block needs that the are filled into described access speed storage medium faster data block of eliminating shared by storage space in, access for described application program.
The DAA of the embodiment of the present invention, after receiver module receives the data access request of application program, if search the data block that module does not find the access of above-mentioned application requests in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of above-mentioned application requests access, then in the storage space shared by data block that the needs that the data block found is filled into above-mentioned access speed storage medium faster by packing module are eliminated, access for above-mentioned application program, thus the advantage of hybrid storage medium can be given full play to, significantly reduce the cost of storage, improve handling up and performance of unit data access simultaneously.
The aspect that the present invention adds and advantage will part provide in the following description, and part will become obvious from the following description, or be recognized by practice of the present invention.
Accompanying drawing explanation
The present invention above-mentioned and/or additional aspect and advantage will become obvious and easy understand from the following description of the accompanying drawings of embodiments, wherein:
Fig. 1 is the process flow diagram of a data access method of the present invention embodiment;
Fig. 2 is the schematic diagram of a block management data of the present invention embodiment;
Fig. 3 is the schematic diagram of the present invention's three layers of storage organization embodiment;
Fig. 4 is the structural representation of a DAA of the present invention embodiment;
Fig. 5 is the structural representation of another embodiment of DAA of the present invention.
Embodiment
Be described below in detail embodiments of the invention, the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or has element that is identical or similar functions from start to finish.Being exemplary below by the embodiment be described with reference to the drawings, only for explaining the present invention, and can not limitation of the present invention being interpreted as.On the contrary, embodiments of the invention comprise fall into attached claims spirit and intension within the scope of all changes, amendment and equivalent.
Fig. 1 is the process flow diagram of a data access method of the present invention embodiment, and as shown in Figure 1, this data access method can comprise:
Step 101, receives the data access request of application program.
Step 102, searches the data block of above-mentioned application requests access according to above-mentioned data access request.
Step 103, if do not find the data block of above-mentioned application requests access in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of above-mentioned application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into above-mentioned access speed storage medium faster, access for above-mentioned application program.
In a kind of implementation of the present embodiment, the slower storage medium of above-mentioned access speed can be remote storage (GlobalStorage); Above-mentioned access speed faster storage medium can be local disk, and wherein, local disk can comprise SSD, flash memory (flash) and hard disk (disk);
Said method also comprises: when finding the data block that needs are eliminated in local disk, the value of the metadata position that the data block of being eliminated by above-mentioned needs is corresponding is denoted as the free time (free).Like this, in step 103, in the storage medium that access speed is slower, search the data block of above-mentioned application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into above-mentioned access speed storage medium faster can be: trigger remote invocation of procedure agreement (Remote Procedure Call Protocol; Hereinafter referred to as: RPC) call remote storage, the data block of above-mentioned application requests access is searched in above-mentioned remote storage, and by the transmission of data blocks that finds to local disk, be filled in local disk the storage space shared by data block being denoted as idle metadata position correspondence.
In the another kind of implementation of the present embodiment, the slower storage medium of above-mentioned access speed can be local disk, and wherein, local disk can comprise SSD, flash memory (flash) and hard disk (Disk); Above-mentioned access speed faster storage medium can be internal memory; Like this, in step 103, in the storage medium that access speed is slower, search the data block of above-mentioned application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into above-mentioned access speed storage medium faster can be: the data block of searching the access of above-mentioned application requests in local disk; If find the data block of above-mentioned application requests access, then the data block found being filled into above-mentioned internal memory needs in the storage space shared by data block eliminated.
Further, search the data block of application requests access in above-mentioned local disk after, if do not find the data block of above-mentioned application requests access, then trigger RPC and call remote storage, the data block of above-mentioned application requests access is searched in remote storage, and by the transmission of data blocks that finds to local disk, then the data block found is dispatched to internal memory from above-mentioned local disk, being filled into internal memory needs in the storage space shared by data block eliminated.
In the present embodiment, data block can be dispatched to different storage medium according to cold and hot degree, and the cold and hot degree of data block can according to data block feature in the application: as index block, Bloom filter (Bloomfilter) block and metamessage block etc.The replacement policy of data block scheduling is open to application program definition in the design, usually adopts recent minimum use (LeastRecently Used; Hereinafter referred to as: LRU) algorithm.The unit of scheduling is the data block in applied logic, and this data block is finally still decided by the characteristic of bottom physical medium certainly.
No matter be LRU or other replacement algorithms, all can perform data erase operation, to reclaim space.For internal memory, this does not have too many expense; Such as, but for local disk, ssd, flash or disk etc., the time delay of erase operation is higher, and can cause loss physically, and then reduces the serviceable life of storage medium.Therefore, in the present embodiment, substantial data erase operation is not on a storage medium had.An abstract metadata management layer in the caching design of local disk layer, as shown in Figure 2, Fig. 2 is the schematic diagram of a block management data of the present invention embodiment, when lru algorithm finds the data block needing to eliminate, as long as the metadata position corresponding to the data block needing to eliminate is denoted as free time (free) state, if do not find the data block of application requests access in local disk, then trigger RPC and call remote storage, the data block of above-mentioned application requests access is searched in above-mentioned remote storage, and by the transmission of data blocks that finds to local disk, be filled in local disk the storage space shared by data block being denoted as idle metadata position correspondence.
The management of internal storage data block does not then need design element data management layer, because internal memory can be arbitrarily erasable.LRU acts directly in internal storage data block, if find the data block of application requests access in local disk, then the data block found directly being filled into above-mentioned internal memory needs in the storage space shared by data block eliminated; And if do not find in local disk above-mentioned application requests access data block, then trigger RPC and call remote storage, the data block of above-mentioned application requests access is searched in remote storage, and by the transmission of data blocks that finds to local disk, again the data block found is dispatched to internal memory from above-mentioned local disk, being filled into internal memory needs in the storage space shared by data block eliminated.
There will be " aging " state because data block retains in the buffer for a long time, namely there occurs inconsistent with the data in remote storage, therefore need to upgrade the data block in buffer memory.In Practical Project realizes, because the entrance of Data Update is initiated by application program, therefore need synchronously to perform buffer update while initiation write upgrades.
Particularly, the data access method that the present embodiment provides can also comprise: the Data Update request receiving application program, by Data Update request above-mentioned in storage medium slower for above-mentioned access speed ask upgrade data block be updated to the data block of carrying in above-mentioned Data Update request, and the data block after upgrading in storage medium slower for above-mentioned access speed is dispatched to above-mentioned access speed faster in storage medium, upgrade above-mentioned access speed corresponding data block in storage medium faster.That is, when application requests upgrades certain data block, first upgrade the data block in remote storage, to ensure data reliability, be updated successfully corresponding data block in rear renewal local disk simultaneously and internal memory, to keep the consistance write.
Further, if in data block renewal process, have received the read requests of above-mentioned data block, then judge whether the above-mentioned data block of above-mentioned access speed faster in storage medium is legal data block; If not, then after waiting for that the above-mentioned data block of above-mentioned access speed faster in storage medium upgrades, the above-mentioned read requests of reprocessing.That is, if in the renewal process of data block, have received the request needing to read this data block, need first to judge the legitimacy of the above-mentioned data block in internal memory or local disk, particularly, judge that the legitimacy of data block can adopt Message Digest 5 the 5th edition (Message Digest Algorithm 5; Hereinafter referred to as: MD5) scheduling algorithm, the present embodiment is to judging that the algorithm that the legitimacy of data block adopts is not construed as limiting.If determine that the above-mentioned data block of preserving in internal memory or local disk is illegal, then after first waiting for the above-mentioned data block renewal in internal memory or local disk, this read requests of reprocessing.
Above-mentioned data access method, after receiving the data access request of application program, if do not find the data block of above-mentioned application requests access in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of above-mentioned application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into above-mentioned access speed storage medium faster, access for above-mentioned application program, thus the advantage of hybrid storage medium can be given full play to, significantly reduce the cost of storage, improve handling up and performance of unit data access simultaneously.
Tertiary structure that storage medium is divided into by data access method provided by the invention from design architecture " internal memory-local disk-remote storage ".Such design has taken into full account the feature of each layer medium: (1) internal memory: access performance is best, and cost is the highest, and capacity is minimum, and can not persistence; (2) local disk: local IP access expense is little, and cost is relatively cheap, volume ratio is comparatively large, and can persistence; (3) remote storage: read-write needs through network, expense is large, and cost is cheap, and capacity is large, can persistence.Above-mentioned three layers of storage organization schematic diagram that the present invention proposes can as shown in Figure 3, and Fig. 3 is the schematic diagram of the present invention's three layers of storage organization embodiment.
Fig. 3 is an example of the present invention's three layers of storage organization, on data access method provided by the invention can be applied in " internal memory (memory)-SSD/flash-SATA " such three layers of storage organization, wherein, these three kinds of storage mediums of memory, SSD/flash and SATA can be deployed on a machine; Data access method provided by the invention also can be applied in " the advanced reduced instruction set machine of the local SSD/SATA-of memory-(Advanced Reduced Instruction SetComputer Machine; Hereinafter referred to as ARM) framework cloud dish (being applied in publicly-owned cloud product) " on such three layers of storage organization.In essence, as long as employing three kinds or the storage organization more than three kinds of storage mediums, and storage medium has the difference on performance and cost just can adopt data access method provided by the invention.
Fig. 4 is the structural representation of a DAA of the present invention embodiment, DAA in the present embodiment can realize the present invention's flow process embodiment illustrated in fig. 1, as shown in Figure 4, this DAA can comprise: receiver module 41, search module 42 and packing module 43;
Wherein, receiver module 41, for receiving the data access request of application program;
Search module 42, the data access request for receiving according to receiver module 41 searches the data block of above-mentioned application requests access; And when not finding the data block of above-mentioned application requests access in access speed faster storage medium, in the storage medium that access speed is slower, search the data block of above-mentioned application requests access;
Packing module 43, in the storage space that will search shared by data block that needs that data block that module 42 finds is filled into above-mentioned access speed storage medium faster eliminate, accesses for above-mentioned application program.
Above-mentioned DAA, after receiver module 41 receives the data access request of application program, if search the data block that module 42 does not find the access of above-mentioned application requests in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of above-mentioned application requests access, then in the storage space shared by data block that the needs that the data block found is filled into above-mentioned access speed storage medium faster by packing module 43 are eliminated, access for above-mentioned application program, thus the advantage of hybrid storage medium can be given full play to, significantly reduce the cost of storage, improve handling up and performance of unit data access simultaneously.
Fig. 5 is the structural representation of another embodiment of DAA of the present invention, compared with the DAA shown in Fig. 4, difference is, in a kind of implementation of the present embodiment, the slower storage medium of above-mentioned access speed can be remote storage, above-mentioned access speed faster storage medium can be local disk, and wherein, local disk can comprise SSD, flash memory (flash) and hard disk (disk);
In this implementation, said apparatus can also comprise:
Indicate module 44, for when finding the data block needing to eliminate in local disk, the value of the metadata position that the data block of being eliminated by above-mentioned needs is corresponding is denoted as the free time.
At this moment, search module 42, calling remote storage specifically for triggering RPC, in above-mentioned remote storage, searching the data block of above-mentioned application requests access;
Packing module 43, specifically for by searching transmission of data blocks that module 42 finds to above-mentioned local disk, is filled in local disk the storage space shared by data block being denoted as idle metadata position correspondence.
In the another kind of implementation of the present embodiment, the slower storage medium of above-mentioned access speed can be local disk, and wherein, local disk can comprise SSD, flash memory (flash) and hard disk (Disk); Above-mentioned access speed faster storage medium can be internal memory;
In this implementation, search module 42, specifically for searching the data block of application requests access in above-mentioned local disk;
Packing module 43, specifically for when searching module 42 and finding the data block of above-mentioned application requests access, the data block found being filled into above-mentioned internal memory needs in the storage space shared by data block eliminated.
Further, search module 42, also for when not finding the data block of above-mentioned application requests access in above-mentioned local disk, triggering RPC and calling remote storage, in above-mentioned remote storage, searching the data block of above-mentioned application requests access;
Packing module 43, also for searching transmission of data blocks that module 42 finds to above-mentioned local disk, then the data block found is dispatched to above-mentioned internal memory from above-mentioned local disk, being filled into above-mentioned internal memory needs in the storage space shared by data block eliminated.
In the present embodiment, data block can be dispatched to different storage medium according to cold and hot degree, and the cold and hot degree of data block can according to data block feature in the application: as index block, Bloom filter (Bloomfilter) block and metamessage block etc.The replacement policy of data block scheduling is open to application program definition in the design, usually adopts lru algorithm.The unit of scheduling is the data block in applied logic, and this data block is finally still decided by the characteristic of bottom physical medium certainly.
No matter be LRU or other replacement algorithms, all can perform data erase operation, to reclaim space.For internal memory, this does not have too many expense; Such as, but for local disk, ssd, flash or disk etc., the time delay of erase operation is higher, and can cause loss physically, and then reduces the serviceable life of storage medium.Therefore, in the present embodiment, substantial data erase operation is not on a storage medium had.An abstract metadata management layer in the caching design of local disk layer, as shown in Figure 2, when lru algorithm finds the data block needing to eliminate, as long as indicate module 44 the metadata position corresponding to the data block needing to eliminate is denoted as free time (free) state, if search the data block that module 42 does not find application requests access in local disk, then trigger RPC and call remote storage, the data block of above-mentioned application requests access is searched in above-mentioned remote storage, and by packing module 43 by the transmission of data blocks that finds to local disk, be filled in local disk the storage space shared by data block being denoted as idle metadata position correspondence.
The management of internal storage data block does not then need design element data management layer, because internal memory can be arbitrarily erasable.LRU acts directly in internal storage data block, if search the data block that module 42 finds application requests access in local disk, then the data block found directly is filled into above-mentioned internal memory by packing module 43 needs in the storage space shared by data block eliminated; And if search module 42 do not find in local disk above-mentioned application requests access data block, then trigger RPC and call remote storage, the data block of above-mentioned application requests access is searched in remote storage, and by packing module 43 by the transmission of data blocks that finds to local disk, again the data block found is dispatched to internal memory from above-mentioned local disk, being filled into internal memory needs in the storage space shared by data block eliminated.
There will be " aging " state because data block retains in the buffer for a long time, namely there occurs inconsistent with the data in remote storage, therefore need to upgrade the data block in buffer memory.In Practical Project realizes, because the entrance of Data Update is initiated by application program, therefore need synchronously to perform buffer update while initiation write upgrades.
Particularly, the DAA that the present embodiment provides can also comprise: update module 45;
At this moment, receiver module 41, also for receiving the Data Update request of above-mentioned application program;
Update module 45, for by Data Update request above-mentioned in storage medium slower for above-mentioned access speed ask upgrade data block be updated to the data block of carrying in above-mentioned Data Update request, and the data block after upgrading in storage medium slower for above-mentioned access speed is dispatched to above-mentioned access speed faster in storage medium, upgrade above-mentioned access speed corresponding data block in storage medium faster.That is, when application requests upgrades certain data block, update module 45 first upgrades the data block in remote storage, to ensure data reliability, is updated successfully corresponding data block in rear renewal local disk simultaneously and internal memory, to keep the consistance write.
Further, above-mentioned DAA can also comprise: judge module 46 and processing module 47;
Judge module 46, for when receiver module 41 is in data block renewal process, when have received the read requests of above-mentioned data block, judges whether the above-mentioned data block of above-mentioned access speed faster in storage medium is legal data block;
Processing module 47, during for determining that when judge module 46 the described data block of above-mentioned access speed faster in storage medium is not legal data block, after waiting for that the described data block of above-mentioned access speed faster in storage medium upgrades, process above-mentioned read requests.That is, if in the renewal process of data block, receiver module 41 have received the request needing to read this data block, judge module 46 needs first to judge the legitimacy of the above-mentioned data block in internal memory or local disk, particularly, judge module 46 judges that the legitimacy of data block can adopt MD5 scheduling algorithm, to judge module 46, the present embodiment judges that the algorithm that the legitimacy of data block adopts is not construed as limiting.If judge module 46 determines that the above-mentioned data block of preserving in internal memory or local disk is illegal, then after first waiting for the above-mentioned data block renewal in internal memory or local disk, this read requests of processing module 47 reprocessing.
Above-mentioned DAA can give full play to the advantage of hybrid storage medium, significantly reduces the cost of storage, improves handling up and performance of unit data access simultaneously.
It should be noted that, in describing the invention, term " first ", " second " etc. only for describing object, and can not be interpreted as instruction or hint relative importance.In addition, in describing the invention, except as otherwise noted, the implication of " multiple " is two or more.
Describe and can be understood in process flow diagram or in this any process otherwise described or method, represent and comprise one or more for realizing the module of the code of the executable instruction of the step of specific logical function or process, fragment or part, and the scope of the preferred embodiment of the present invention comprises other realization, wherein can not according to order that is shown or that discuss, comprise according to involved function by the mode while of basic or by contrary order, carry out n-back test, this should understand by embodiments of the invention person of ordinary skill in the field.
Should be appreciated that each several part of the present invention can realize with hardware, software, firmware or their combination.In the above-described embodiment, multiple step or method can with to store in memory and the software performed by suitable instruction execution system or firmware realize.Such as, if realized with hardware, the same in another embodiment, can realize by any one in following technology well known in the art or their combination: the discrete logic with the logic gates for realizing logic function to data-signal, there is the special IC of suitable combinational logic gate circuit, programmable gate array (Programmable Gate Array; Hereinafter referred to as: PGA), field programmable gate array (Field ProgrammableGate Array; Hereinafter referred to as: FPGA) etc.
Those skilled in the art are appreciated that realizing all or part of step that above-described embodiment method carries is that the hardware that can carry out instruction relevant by program completes, described program can be stored in a kind of computer-readable recording medium, this program perform time, step comprising embodiment of the method one or a combination set of.
In addition, each functional module in each embodiment of the present invention can be integrated in a processing module, also can be that the independent physics of modules exists, also can two or more module integrations in a module.Above-mentioned integrated module both can adopt the form of hardware to realize, and the form of software function module also can be adopted to realize.If described integrated module using the form of software function module realize and as independently production marketing or use time, also can be stored in a computer read/write memory medium.
The above-mentioned storage medium mentioned can be ROM (read-only memory), disk or CD etc.
In the description of this instructions, specific features, structure, material or feature that the description of reference term " embodiment ", " some embodiments ", " example ", " concrete example " or " some examples " etc. means to describe in conjunction with this embodiment or example are contained at least one embodiment of the present invention or example.In this manual, identical embodiment or example are not necessarily referred to the schematic representation of above-mentioned term.And the specific features of description, structure, material or feature can combine in an appropriate manner in any one or more embodiment or example.
Although illustrate and describe embodiments of the invention above, be understandable that, above-described embodiment is exemplary, can not be interpreted as limitation of the present invention, and those of ordinary skill in the art can change above-described embodiment within the scope of the invention, revises, replace and modification.
Claims (14)
1. a data access method, is characterized in that, comprising:
Receive the data access request of application program;
The data block of described application requests access is searched according to described data access request;
If do not find the data block of described application requests access in access speed faster storage medium, in the slower storage medium of access speed, then search the data block of described application requests access, and in the storage space shared by data block eliminated of the needs data block found being filled into described access speed storage medium faster, access for described application program.
2. method according to claim 1, is characterized in that, the slower storage medium of described access speed includes but not limited to remote storage; Described access speed faster storage medium includes but not limited to local disk;
Described method also comprises:
When finding the data block needing to eliminate in local disk, the value of the metadata position that the data block of being eliminated by described needs is corresponding is denoted as the free time.
3. method according to claim 2, it is characterized in that, search the data block of described application requests access in the described storage medium slower in access speed, and the storage space shared by data block that the needs data block found being filled into described access speed storage medium are faster eliminated comprises:
Trigger remote invocation of procedure agreement calls remote storage, the data block of described application requests access is searched in described remote storage, and by the transmission of data blocks that finds to described local disk, be filled in described local disk the storage space shared by data block being denoted as idle metadata position correspondence.
4. method according to claim 1, is characterized in that, the slower storage medium of described access speed includes but not limited to local disk; Described access speed faster storage medium comprises internal memory;
Search the data block of described application requests access in the described storage medium slower in access speed, and the storage space shared by data block that the needs data block found being filled into described access speed storage medium are faster eliminated comprises:
The data block of described application requests access is searched in described local disk;
If find the data block of described application requests access, then the data block found being filled into described internal memory needs in the storage space shared by data block eliminated.
5. method according to claim 4, is characterized in that, described search in described local disk described application requests access data block after, also comprise:
If do not find the data block of described application requests access, then trigger remote invocation of procedure agreement calls remote storage, the data block of described application requests access is searched in described remote storage, and by the transmission of data blocks that finds to described local disk, again the data block found is dispatched to described internal memory from described local disk, being filled into described internal memory needs in the storage space shared by data block eliminated.
6. the method according to claim 1-5 any one, is characterized in that, also comprises:
Receive the Data Update request of described application program;
By Data Update request described in storage medium slower for described access speed ask upgrade data block be updated to the data block of carrying in described Data Update request, and the data block after upgrading in storage medium slower for described access speed is dispatched to described access speed faster in storage medium, upgrade described access speed corresponding data block in storage medium faster.
7. method according to claim 6, is characterized in that, also comprises:
If in data block renewal process, have received the read requests of described data block, then judge whether the described data block of described access speed faster in storage medium is legal data block;
If not, then after waiting for that the described data block of described access speed faster in storage medium upgrades, read requests described in reprocessing.
8. a DAA, is characterized in that, comprising:
Receiver module, for receiving the data access request of application program;
Search module, the data access request for receiving according to described receiver module searches the data block of described application requests access; And when not finding the data block of described application requests access in access speed faster storage medium, in the storage medium that access speed is slower, search the data block of described application requests access;
Packing module, for by described search module searches to the data block needs that the are filled into described access speed storage medium faster data block of eliminating shared by storage space in, access for described application program.
9. device according to claim 8, is characterized in that, the slower storage medium of described access speed includes but not limited to remote storage; Described access speed faster storage medium includes but not limited to local disk;
Described device also comprises:
Indicate module, for when finding the data block needing to eliminate in local disk, the value of the metadata position that the data block of being eliminated by described needs is corresponding is denoted as the free time.
10. device according to claim 9, is characterized in that,
Describedly search module, call remote storage specifically for trigger remote invocation of procedure agreement, in described remote storage, search the data block of described application requests access;
Described packing module, specifically for searching transmission of data blocks that module searches arrives to described local disk by described, is filled in described local disk the storage space shared by data block being denoted as idle metadata position correspondence.
11. devices according to claim 8, is characterized in that, the slower storage medium of described access speed includes but not limited to local disk; Described access speed faster storage medium comprises internal memory;
Describedly search module, specifically for searching the data block of described application requests access in described local disk;
Described packing module, specifically for when described in search the data block that module searches accesses to described application requests time, the data block found being filled into described internal memory needs in the storage space shared by data block eliminated.
12. devices according to claim 11, is characterized in that,
Describedly search module, also for when not finding the data block of described application requests access in described local disk, trigger remote invocation of procedure agreement calls remote storage, searches the data block of described application requests access in described remote storage;
Described packing module, also for searching transmission of data blocks that module searches arrives to described local disk by described, again the data block found is dispatched to described internal memory from described local disk, being filled into described internal memory needs in the storage space shared by data block eliminated.
13. devices according to Claim 8 described in-12 any one, is characterized in that, also comprise: update module;
Described receiver module, also for receiving the Data Update request of described application program;
Described update module, for by Data Update request described in storage medium slower for described access speed ask upgrade data block be updated to the data block of carrying in described Data Update request, and the data block after upgrading in storage medium slower for described access speed is dispatched to described access speed faster in storage medium, upgrade described access speed corresponding data block in storage medium faster.
14. devices according to claim 13, is characterized in that, also comprise: judge module and processing module;
Described judge module, for when described receiver module is in data block renewal process, when have received the read requests of described data block, judges whether the described data block of described access speed faster in storage medium is legal data block;
Described processing module, during for determining that when described judge module the described data block of described access speed faster in storage medium is not legal data block, after waiting for that the described data block of described access speed faster in storage medium upgrades, process described read requests.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510140432.7A CN104731714A (en) | 2015-03-27 | 2015-03-27 | Data access method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510140432.7A CN104731714A (en) | 2015-03-27 | 2015-03-27 | Data access method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104731714A true CN104731714A (en) | 2015-06-24 |
Family
ID=53455622
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510140432.7A Pending CN104731714A (en) | 2015-03-27 | 2015-03-27 | Data access method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104731714A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108418877A (en) * | 2018-02-22 | 2018-08-17 | 上海思华科技股份有限公司 | Polymorphic type storage scheduling plug-flow method, system, data-updating method |
| CN110244911A (en) * | 2019-06-20 | 2019-09-17 | 北京奇艺世纪科技有限公司 | A kind of data processing method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101398744A (en) * | 2007-09-26 | 2009-04-01 | 新奥特硅谷视频技术有限责任公司 | Method and system for implementing network data process by utilizing virtual file system technology |
| CN102355495A (en) * | 2011-09-27 | 2012-02-15 | 中国联合网络通信集团有限公司 | Data processing method, device and system |
| US20120297158A1 (en) * | 2011-05-19 | 2012-11-22 | Chih-Yao Hua | Mass storage device capable of accessing a network storage |
-
2015
- 2015-03-27 CN CN201510140432.7A patent/CN104731714A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101398744A (en) * | 2007-09-26 | 2009-04-01 | 新奥特硅谷视频技术有限责任公司 | Method and system for implementing network data process by utilizing virtual file system technology |
| US20120297158A1 (en) * | 2011-05-19 | 2012-11-22 | Chih-Yao Hua | Mass storage device capable of accessing a network storage |
| CN102355495A (en) * | 2011-09-27 | 2012-02-15 | 中国联合网络通信集团有限公司 | Data processing method, device and system |
Non-Patent Citations (1)
| Title |
|---|
| 王婷婷等: "《微型计算机原理、单片机原理与接口技术》", 28 February 2014 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108418877A (en) * | 2018-02-22 | 2018-08-17 | 上海思华科技股份有限公司 | Polymorphic type storage scheduling plug-flow method, system, data-updating method |
| CN110244911A (en) * | 2019-06-20 | 2019-09-17 | 北京奇艺世纪科技有限公司 | A kind of data processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11960441B2 (en) | Retention management for data streams | |
| EP3195104B1 (en) | Efficient data movement within file system volumes | |
| US10242022B1 (en) | Systems and methods for managing delayed allocation on clustered file systems | |
| US11687446B2 (en) | Namespace change propagation in non-volatile memory devices | |
| US10860493B2 (en) | Method and apparatus for data storage system | |
| US8347050B2 (en) | Append-based shared persistent storage | |
| US11593272B2 (en) | Method, apparatus and computer program product for managing data access | |
| CN108153849B (en) | Database table segmentation method, device, system and medium | |
| US11245774B2 (en) | Cache storage for streaming data | |
| US20140351515A1 (en) | Providing data attributes to a storage manager to use to select a storage tier to use for a data set management operation | |
| US20140052892A1 (en) | Methods and apparatus for providing acceleration of virtual machines in virtual environments | |
| US20140351505A1 (en) | Providing data attributes to a storage manager to use to select a storage tier to use for a data set management operation | |
| CN114327280B (en) | Message storage method and system based on cold and hot separation storage | |
| US9778927B2 (en) | Storage control device to control storage devices of a first type and a second type | |
| KR20180086120A (en) | Tail latency aware foreground garbage collection algorithm | |
| JP6228374B2 (en) | System for dynamically adapting cache storage | |
| CN103617133B (en) | Virtual memory compression method and device in a kind of Windows systems | |
| US10423580B2 (en) | Storage and compression of an aggregation file | |
| CN103927136A (en) | Identification method and device for input and output IO types | |
| CN102298556A (en) | Data stream recognition method and device | |
| US20210103520A1 (en) | System and method for inline tiering of write data | |
| CN104731714A (en) | Data access method and device | |
| US11315028B2 (en) | Method and apparatus for increasing the accuracy of predicting future IO operations on a storage system | |
| US20220261354A1 (en) | Data access method and apparatus and storage medium | |
| CN105278873B (en) | A kind of distribution method and device of disk block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150624 |