CN101398821A - Network storage method and system based on local cache and virtual file systems technology - Google Patents
Network storage method and system based on local cache and virtual file systems technology Download PDFInfo
- Publication number
- CN101398821A CN101398821A CNA2007101224457A CN200710122445A CN101398821A CN 101398821 A CN101398821 A CN 101398821A CN A2007101224457 A CNA2007101224457 A CN A2007101224457A CN 200710122445 A CN200710122445 A CN 200710122445A CN 101398821 A CN101398821 A CN 101398821A
- Authority
- CN
- China
- Prior art keywords
- local cache
- network storage
- file
- network
- upper layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the field of network data memory technique and in particular relates to a network memory method and a system on the basis of local cache and virtual file system technique; the method assumes one or more pieces of hard disk equipment on the equipment layer of an video/audio workstation operation system; the virtual hard disk equipment is arranged between an upper layer software module and a local disk; the upper software module operates the virtual hard disks in the form of visiting normal drivers; the virtual hard disk equipment is connected with network memory equipment by data; when the upper layer software carries out write-file operation, one copy of the file is written respectively in the cache of the local workstation and the network memory equipment; when the upper layer software carries out the read-file operation, the data memorized in the local cache is preferably used. The method and the system reduce the usage stress of the network disc array, reduce the requirement on the network bandwidth, break no program production when limit condition occurs to the system, and ensure the reliable running of the system at high speed.
Description
Technical field
The invention belongs to the network data technical field of memory, be specifically related to a kind of network storage method and storage system based on virtual file system technique.This system can be applied to the audio-video network storage of radio, TV and film industries, and other to the network bandwidth, security request data than higher various field of data storage.
Background technology
In recent years, in the design and implementation process of the non-netting network of radio, TV and film industries project, consider that maximum problems is the stable and bandwidth at a high speed that how to guarantee network, stable, high speed is the basic demand of non-volume to document storage system.Non-volume is very high to the requirement of bandwidth, and bandwidth has fluctuation will cause the discontinuous of broadcasting pictures slightly, its consequence or be that the program quality of making does not pass a test, or cause the collapse of non-volume system, even directly cause the broadcast accident.
How using lower cost to set up a high definition network on a large scale is the another one problem that radio, TV and film industries will face, and the bandwidth requirement of high definition network can be more than 2 times of present bandwidth, and this is a challenge to present technology.
Because the singularity of television broadcasting, audio-video network can often run into the following problem at present: collapsed as the dish battle array and how to have handled, how switch tackles if having damaged, server has been delayed machine and how to have been solved or the like, and this just needs a series of about the emergence treatment scheme of system under limit situation.Because the dish battle array once probability of bad two above hard disks is minimum, so conventional solution generally is can above dual controller, last Dual-switch allows server support HA.
More than these solutions except cost is doubled and redoubled, the system that can not guarantee is really perfectly safe.Generally speaking, above-mentioned serial solution can guarantee that data can not lose, but can not guarantee that system can move always continuously, but always has one section unavailable time, and this section period be can not determine length.For program network systems such as news productions, system's unavailable time more than 1 hour, just might the program that be about to broadcast be impacted, thus seek one cost-saved and have more the emergency plan of practicality, be the target that the radio, TV and film industries person skilled is pursued always.
At last, non-ly be organized in that the occupied bandwidth resource is the highest in the non-netting network, the main application that non-volume takies these bandwidth comprises: material collection, program making, material are downloaded, packing is synthetic, program examination etc.In these operating process, except the open-and-shut news category program of part the time ratio of making link cost is less, should program making time of cost be the longest, and the bandwidth that program making takies is also maximum, the network of especially making type is just more obvious.The dish battle array is in the operation of bearing " reading " more the time as can be seen from analyze, if minimizing dish battle array " reading " operation as far as possible, this part work is put into other places, thereby makes the dish battle array bear the operation of more " writing ", also just supported more non-volume website.
Summary of the invention
The objective of the invention is at the existing defective of the emergence treatment scheme of system under limit situation in the present audio-video network technology, a kind of network storage method and system based on local cache and virtual file system technique is provided, thereby the operation format of reasonable configuration network disk battle array more, supporting more non-volume website, the assurance system can be reliably, operation at a high speed.
Technical scheme of the present invention is as follows: a kind of network storage method based on local cache and virtual file system technique, this method is at the virtual one or more hard disc apparatus of the mechanical floor of looking audio workstation operating system, when upper layer software (applications) carries out operating writing-file, in the Cache of workstation this locality and the network storage equipment, respectively write portion, when upper layer software (applications) is read file operation, preferentially use the data of storing among the local Cache.
Aforesaid network storage method based on local cache and virtual file system technique, wherein, when upper layer software (applications) is read file operation, if there is not related data in the Cache of workstation this locality, then reading of data returns to upper layer application from the network storage equipment, and keeps a identical data file when passing to the upper strata in the Cache of this locality.
Aforesaid network storage method based on local cache and virtual file system technique wherein, when upper layer software (applications) carries out operating writing-file, under the not enough situation of network disk battle array speed, preferentially writes file among the local Cache.
Further, in above-mentioned network storage method, when upper layer software (applications) is made amendment file operation, related data is modified in the Cache of workstation this locality and the network storage equipment on the pairing file simultaneously based on local cache and virtual file system technique; If there is not associated data files in the Cache of workstation this locality, then obtain the size of corresponding document in the network storage equipment, in the Cache of this locality, create an onesize empty file, the data that write are write in these two files simultaneously, write down the position that writes simultaneously, during the free time other data are written to local Cache in system.
Further, in above-mentioned network storage method, when upper layer software (applications) carries out delete file operation, delete the associated documents in the network storage equipment earlier, and then delete the associated documents among the local Cache based on local cache and virtual file system technique; In delete procedure, if deletion network file success, then no matter whether the file among the local Cache is successfully deleted, and all returns to the information of upper layer application success.
Further, in above-mentioned network storage method based on local cache and virtual file system technique, when upper layer software (applications) carries out the inquiry file operation, for application model, associated documents in the preferential use network storage equipment return Query Result, if fault is given birth in the network disk paroxysm, then use local Cache to carry out the listed files inquiry; For application models such as remote storage, preferentially use the associated documents among the local Cache to return Query Result.
A kind of network store system based on local cache and virtual file system technique, comprise and look the audio workstation and the network storage equipment, wherein, between upper layer software (applications) module of looking audio workstation and local disk, the virtual hard disc apparatus of one cover is set, the upper layer software (applications) module is operated virtual hard disc apparatus with the form of visit generic drive, carries out data between virtual hard disc apparatus and the network storage equipment and is connected.
Aforesaid network store system based on local cache and virtual file system technique, wherein, virtual hard disc apparatus is one or more virtual hard disks.
Aforesaid network store system based on local cache and virtual file system technique wherein, in the Cache and the network storage equipment of local disk, all stores relevant video/audio file.
Beneficial effect of the present invention is as follows: the present invention overlaps hard disc apparatus by virtual file system technique at the mechanical floor of looking audio workstation virtual, and between virtual system and network disk battle array, set up data and get in touch, this locality of non-volume workstation deposit a and dish gust on the same material, to be put in the Virtual File System " reading " operation of data file in the network system and be achieved, thereby, reduced the working pressure of network disk battle array, reduced demand to the network bandwidth, and can occur under the limit situation in system, do not interrupt program production, the assurance system can be reliable, operation at a high speed.
Description of drawings
Fig. 1 is a system architecture synoptic diagram of the present invention.
Process flow diagram when Fig. 2 carries out data read operation for the present invention.
Process flow diagram when Fig. 3 carries out data write operation for the present invention.
Process flow diagram when Fig. 4 carries out the data modification operation for the present invention.
Embodiment
The present invention is described in further detail below in conjunction with drawings and Examples.
As shown in Figure 1, comprise based on the network store system of local cache and virtual file system technique and to look the audio workstation and the network storage equipment--the network disk battle array, between upper layer software (applications) module of looking audio workstation and local disk, by the VFS virtual file system technique the virtual hard disc apparatus of one cover is set, the upper layer software (applications) module is operated virtual hard disc apparatus with the form of visit generic drive, carries out data between virtual hard disc apparatus and the network disk battle array and is connected.
The VFS technology is a kind of known technology of this area, and in the present invention, VFS need handle following several main things or function:
● at the mechanical floor virtual hard disk equipment of operating system.
● when upper strata during, need in the local Cache (cache memory) and the network storage equipment, respectively write portion at written document.
● when the upper strata when reading file, preferentially use data among the local Cache, if there is not related data among the local Cache, then from the network storage equipment, read, in local Cache, keep a when passing to the upper strata again.
● support multi-link redundancy, when a MDC or NAS head fail, enable other reserve link automatically the network storage equipment.
● according to the space operating position of local Cache, and the frequency of utilization of cache file, safeguard the life cycle of file in the local cache automatically.
● provide simple function to upper layer application, be used for confirming whether local Cache is arranged, and the file that may use carried out Cache in advance operate.Here it needs to be noted that the API that upper layer application can be never called any VFS just can be benefited from the basic function of VFS.
The present invention can realize in the operating system platform arbitrarily, and is irrelevant with concrete Platform Type.With Windows operating system is example, at first develop a virtual hard disk module--file virtual level, this is the basis of Virtual File System, can in the Windows system, find the virtual drive that comes out by this module user, can in this drive, carry out common file operation, not have what difference with using common hard disk.
Last layer operation to file system has following several substantially: read, write (newly-built, modification), deletion, inquiry, these operations handling procedures for the file virtual level are as follows:
(1) read operation
When upper layer software (applications) is read file operation, the preferential data of storing among the local Cache of using, if there is not related data in the Cache of workstation this locality, then reading of data returns to upper layer application from the network storage equipment, and keeps a identical data file when passing to the upper strata in the Cache of this locality.Idiographic flow is seen Fig. 2.
(2) write operation (newly-built, modification)
When upper layer software (applications) carries out operating writing-file, in the Cache of workstation this locality and the network storage equipment, respectively write portion, under the not enough situation of network disk battle array speed, file is preferentially write among the local Cache.Idiographic flow is seen Fig. 3.
When upper layer software (applications) is made amendment file operation, related data is modified in the Cache of workstation this locality and the network storage equipment on the pairing file simultaneously; If there is not associated data files in the Cache of workstation this locality, then obtain the size of corresponding document in the network storage equipment, in the Cache of this locality, create an onesize empty file, the data that write are write in these two files simultaneously, write down the position that writes simultaneously, during the free time other data are written to local Cache in system.Idiographic flow is seen Fig. 4.
(3) deletion action
When upper layer software (applications) carries out delete file operation, the material on local and the network disk battle array deleted simultaneously get final product, be noted that first deletion dish battle array material but delete, and then delete local material, so that the consistance of maintenance data.In delete procedure, if the success of deletion network file then no matter whether local Cache file deletes success, all returns to the information of upper layer application success.
(4) query manipulation
When upper layer software (applications) carries out the inquiry file operation, just obtain listed files, reduce this demand of bandwidth pressure (or application model) for local Cache, the preferential network disk battle array of using of inquiry, if fault is given birth in the network disk paroxysm, will use local Cache to carry out the listed files inquiry; For application models such as remote storage,, preferentially use local Cache to return Query Result in order to improve response speed to upper layer application.
Consider the description of this invention disclosed herein and special embodiment, other embodiment of the present invention are conspicuous for a person skilled in the art.These explanations and embodiment only consider as an example that they all belong to by within the indicated protection scope of the present invention and spirit of claims.
Claims (12)
1. network storage method based on local cache and virtual file system technique, it is characterized in that: this method is at the virtual one or more hard disc apparatus of the mechanical floor of looking audio workstation operating system, when upper layer software (applications) carries out operating writing-file, in the Cache of workstation this locality and the network storage equipment, respectively write portion, when upper layer software (applications) is read file operation, preferentially use the data of storing among the local Cache.
2. the network storage method based on local cache and virtual file system technique as claimed in claim 1, it is characterized in that: when upper layer software (applications) is read file operation, if there is not related data in the Cache of workstation this locality, then reading of data returns to upper layer application from the network storage equipment, and keeps a identical data file when passing to the upper strata in the Cache of this locality.
3. the network storage method based on local cache and virtual file system technique as claimed in claim 1 or 2, it is characterized in that: when upper layer software (applications) carries out operating writing-file, under the not enough situation of network disk battle array speed, file is preferentially write among the local Cache.
4. the network storage method based on local cache and virtual file system technique as claimed in claim 1, it is characterized in that: when upper layer software (applications) is made amendment file operation, related data is modified in the Cache of workstation this locality and the network storage equipment on the pairing file simultaneously.
5. the network storage method based on local cache and virtual file system technique as claimed in claim 4, it is characterized in that: if there is not associated data files in the Cache of workstation this locality, then obtain the size of corresponding document in the network storage equipment, in the Cache of this locality, create an onesize empty file, the data that write are write in these two files simultaneously, write down the position that writes simultaneously, during the free time other data are written to local Cache in system.
6. the network storage method based on local cache and virtual file system technique as claimed in claim 1, it is characterized in that: when upper layer software (applications) carries out delete file operation, delete the associated documents in the network storage equipment earlier, and then delete the associated documents among the local Cache.
7. the network storage method based on local cache and virtual file system technique as claimed in claim 6, it is characterized in that: in delete procedure, if deletion network file success, then no matter whether the file among the local Cache is successfully deleted, and all returns to the information of upper layer application success.
8. the network storage method based on local cache and virtual file system technique as claimed in claim 1, it is characterized in that: when upper layer software (applications) carries out the inquiry file operation, for application model, associated documents in the preferential use network storage equipment return Query Result, if fault is given birth in the network disk paroxysm, then use local Cache to carry out the listed files inquiry.
9. the network storage method based on local cache and virtual file system technique as claimed in claim 1, it is characterized in that: when upper layer software (applications) carries out the inquiry file operation, for the remote storage isotype, preferentially use the associated documents among the local Cache to return Query Result.
10. network store system based on local cache and virtual file system technique, comprise and look the audio workstation and the network storage equipment, it is characterized in that: between upper layer software (applications) module of looking audio workstation and local disk, the virtual hard disc apparatus of one cover is set, the upper layer software (applications) module is operated virtual hard disc apparatus with the form of visit generic drive, carries out data between virtual hard disc apparatus and the network storage equipment and is connected.
11. the network store system based on local cache and virtual file system technique as claimed in claim 10 is characterized in that: virtual hard disc apparatus comprises one or more virtual hard disks.
12., it is characterized in that: in the Cache and the network storage equipment of local disk, all store relevant video/audio file as claim 10 or 11 described network store systems based on local cache and virtual file system technique.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2007101224457A CN101398821A (en) | 2007-09-26 | 2007-09-26 | Network storage method and system based on local cache and virtual file systems technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2007101224457A CN101398821A (en) | 2007-09-26 | 2007-09-26 | Network storage method and system based on local cache and virtual file systems technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101398821A true CN101398821A (en) | 2009-04-01 |
Family
ID=40517384
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2007101224457A Pending CN101398821A (en) | 2007-09-26 | 2007-09-26 | Network storage method and system based on local cache and virtual file systems technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101398821A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521330A (en) * | 2011-12-07 | 2012-06-27 | 华中科技大学 | Mirror distributed storage method under desktop virtual environment |
| CN103020199A (en) * | 2012-12-05 | 2013-04-03 | 上海创图网络科技发展有限公司 | Virtual file system suitable for different operating system platforms and application thereof |
| WO2013131443A1 (en) * | 2012-03-09 | 2013-09-12 | 腾讯科技（深圳）有限公司 | Data storage method and device |
| WO2016197838A1 (en) * | 2015-06-08 | 2016-12-15 | 阿里巴巴集团控股有限公司 | Access method and apparatus |
| CN106850825A (en) * | 2017-02-23 | 2017-06-13 | 中南大学 | Client's end block level cache optimization method under a kind of mobile lucidification disposal environment |
| CN108052536A (en) * | 2017-11-16 | 2018-05-18 | 中南大学 | A kind of file system of IoT equipment |
-
2007
- 2007-09-26 CN CNA2007101224457A patent/CN101398821A/en active Pending
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521330A (en) * | 2011-12-07 | 2012-06-27 | 华中科技大学 | Mirror distributed storage method under desktop virtual environment |
| WO2013131443A1 (en) * | 2012-03-09 | 2013-09-12 | 腾讯科技（深圳）有限公司 | Data storage method and device |
| CN103020199A (en) * | 2012-12-05 | 2013-04-03 | 上海创图网络科技发展有限公司 | Virtual file system suitable for different operating system platforms and application thereof |
| KR20180016488A (en) * | 2015-06-08 | 2018-02-14 | 알리바바 그룹 홀딩 리미티드 | Access methods and devices |
| CN106302609A (en) * | 2015-06-08 | 2017-01-04 | 阿里巴巴集团控股有限公司 | A kind of access method and device |
| WO2016197838A1 (en) * | 2015-06-08 | 2016-12-15 | 阿里巴巴集团控股有限公司 | Access method and apparatus |
| CN106302609B (en) * | 2015-06-08 | 2020-02-28 | 阿里巴巴集团控股有限公司 | Access method and device |
| KR102256890B1 (en) | 2015-06-08 | 2021-05-31 | 어드밴스드 뉴 테크놀로지스 씨오., 엘티디. | Access method and device |
| US11221997B2 (en) | 2015-06-08 | 2022-01-11 | Advanced New Technologies Co., Ltd. | On-demand creation and access of a virtual file system |
| CN106850825A (en) * | 2017-02-23 | 2017-06-13 | 中南大学 | Client's end block level cache optimization method under a kind of mobile lucidification disposal environment |
| CN106850825B (en) * | 2017-02-23 | 2020-08-07 | 中南大学 | Client block-level cache optimization method in mobile transparent computing environment |
| CN108052536A (en) * | 2017-11-16 | 2018-05-18 | 中南大学 | A kind of file system of IoT equipment |
| CN108052536B (en) * | 2017-11-16 | 2020-12-11 | 中南大学 | File system of IoT (Internet of things) equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101398744B (en) | Method and system for implementing network data process by utilizing virtual file system technology | |
| US9733862B1 (en) | Systems and methods for reverse point-in-time copy management in a storage system | |
| CN102955720B (en) | A kind of method improving ext file system stability | |
| CN101501623B (en) | Filesystem-aware block storage system, apparatus, and method | |
| CN102033786B (en) | Method for repairing consistency of copies in object storage system | |
| US7844643B2 (en) | Storage management system with integrated continuous data protection and remote copy | |
| US20160357446A1 (en) | Virtual disk drive system and method | |
| US7213116B2 (en) | Method and apparatus for mirroring objects between storage systems | |
| CN101661414B (en) | Computer system and backup method therefor | |
| US9996557B2 (en) | Database storage system based on optical disk and method using the system | |
| CN101398821A (en) | Network storage method and system based on local cache and virtual file systems technology | |
| KR101678868B1 (en) | Apparatus for flash address translation apparatus and method thereof | |
| CN101008886A (en) | Storage virtual computer system and management data snapshot image method in storage system | |
| CN102405460A (en) | Virtualized storage system and method of operating it | |
| US8560884B2 (en) | Application recovery in a file system | |
| CN101258473A (en) | Data consolidation and garbage collection in direct data file storage memories | |
| CN103106047A (en) | Storage system based on object and storage method thereof | |
| CN103838830A (en) | Data management method and system of HBase database | |
| US8495010B2 (en) | Method and system for adaptive metadata replication | |
| CN102541691A (en) | Log check point recovery method applied to memory data base OLTP (online transaction processing) | |
| CN101727298B (en) | Method and device for realizing independent redundant arrays of inexpensive disks | |
| CN106326229A (en) | Method and device for file storage of embedded system | |
| CN102982182A (en) | Data storage planning method and device | |
| CN100454302C (en) | File management system and method | |
| KR101604222B1 (en) | Recording apparatus and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20090401 |
|
| C20 | Patent right or utility model deemed to be abandoned or is abandoned |