CN106407409A - A virtual file system based on DAS architecture storage servers and a file management method thereof - Google Patents
A virtual file system based on DAS architecture storage servers and a file management method thereof Download PDFInfo
- Publication number
- CN106407409A CN106407409A CN201610843245.XA CN201610843245A CN106407409A CN 106407409 A CN106407409 A CN 106407409A CN 201610843245 A CN201610843245 A CN 201610843245A CN 106407409 A CN106407409 A CN 106407409A
- Authority
- CN
- China
- Prior art keywords
- file
- task
- subsystem
- data
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/188—Virtual file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a virtual file system based on DAS architecture storage servers and a file management method thereof. The virtual file system comprises a task management sub-system and a cache sub-system. The task management sub-system is used for receiving file operation tasks from request terminals and sending the tasks to the cache sub-system. If the task type of the file operation tasks is file reading tasks, the cache sub-system acquires to-be-read file data corresponding to the file operation tasks from a cache space, and sends the to-be-read file data to the task management sub-system; or if the task type of the file operation tasks is file writing tasks, the cache sub-system writes to-be-written file data corresponding to the file operation tasks into the cache space. The task management sub-system receives the to-be-read file data and sends the to-be-read file data to the request terminals. The virtual file system is provided with the cache sub-system which can process file operation tasks, so that repeated reading of disks or disk arrays can be prevented and the IO performance of the disks or disk arrays can be guaranteed.
Description
Technical field
The present invention relates to Virtual File System field, more particularly, to a kind of virtual literary composition based on DAS framework storage server
Part system and its file management method.
Background technology
With developing rapidly of the Internet and video technique, people's continuous lifting to video quality demands, lead to video counts
It is continuously increased according to transmission quantity, require video storage server to improve constantly video read-write speed therewith, timely respond to use to meet
The purpose of family video read-write requests.Current disk storage server mainly includes tri- kinds of frameworks of DAS, NAS and SAN.Wherein, DAS
(Direct Access Storage, i.e. Direct Attached Storage) refers to for storage device to pass through scsi interface or optical-fibre channel is straight
It is connected in succession on a computer.NAS (Network Attached Storage, i.e. network connection storage) refers to set storage
For by the network topology structure (such as Ethernet) of standard, it is connected on a group computer.SAN(Storage Area
Network, i.e. storage area network) refer to be connected on a group computer by optical-fibre channel.
In actual application, the medium-sized and small enterprises such as hotel, apartment, local tv station are cost limited, typically will not
The storage server of the complex and high cost NAS framework of selected technology or SAN framework carries out video data storage it will usually adopt
The storage server of DAS framework and then video data storage, the storage server of DAS framework relies primarily on the master of storage server
Machine operating system carries out read-write operation and the storage system maintenance management of data.The storage server of DAS framework there is low cost, knot
Structure simply it is not necessary to the advantages of special technical staff safeguards, stores and the IO performance to read-write operation in general data
When (Input/Output performance, i.e. input and output performance) is less demanding, primary demand can be met.But needing big concurrent reading and writing
During operation, the storage server of DAS framework can only be by configuring RAID (Redundant Arrays of Independent
Disks, i.e. independent redundancy disk array) card, improves the overall read-write speed of the disk array being connected with storage server.Existing
The storage server of DAS framework, even if being configured with RAID card, if during the same video resource of multiple user's central access, also can shape
Become the IO performance bottleneck of whole storage server, lead to the disk storing this video resource nervous, the IO of impact storage server
Performance.Even and if, multiple access request is not to access same video resource but the corresponding video resource of multiple access request to exist
It is also possible to lead to disk nervous on same disk, the IO performance of impact storage server.
Content of the invention
The technical problem to be solved in the present invention is, for the defect of prior art, provides based on DAS framework storage service
The Virtual File System of device and its file management method, in order to improve the read-write speed of storage server.
The technical solution adopted for the present invention to solve the technical problems is:A kind of void based on DAS framework storage server
Intend file system, including task management subsystem and cache subsystem;
Described task management subsystem, for receiving the file operation task from requesting terminal, in described file operation
When the task type of task is reading and writing of files task, described file operation task is sent to described cache subsystem;
Described cache subsystem, for receiving described file operation task;Task type in described file operation task
During for reading file task, from spatial cache, obtain the to be read file data corresponding with described file operation task, and will
Described file data to be read is sent to described task management subsystem;Or, in the task type of described file operation task
During for written document task, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem, for receiving described file data to be read, and by described file data to be read
It is sent to described requesting terminal.
Preferably, described Virtual File System also includes storage subsystem;
Described cache subsystem, for judging to receive the task type of described file operation task as reading file task also
It is written document task;
Described cache subsystem, during for being to read file task in the task type receiving described file operation task,
Judge to whether there is the to be read file data corresponding with described file operation task in spatial cache;If existing, update
File data in described spatial cache, and described file data to be read is sent to described task management subsystem;If no
Exist, then described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, obtains corresponding with described file operation task
File data to be read, and described file data to be read is sent to described cache subsystem;
Described cache subsystem, for receiving the file data described to be read from described storage subsystem, will be described
File data to be read is added in described spatial cache, updates the file data in described spatial cache;And continue described
File data is taken to be sent to described task management subsystem;
Described cache subsystem, during for being written document task in the task type receiving described file operation task,
Described file operation task includes described file data to be written;Described file data to be written is added to described spatial cache
Interior, update the file data of described spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, and stores described file data to be written.
Preferably, described Virtual File System also includes setting up the catalogue subsystem of internal memory mapping relations with host operating system
System;
Described directory subsystem, for the number of files being stored in using preset data structure management in described storage subsystem
According to corresponding file directory information;
Described storage subsystem, deposits for disk is divided into memory paragraph, memory block and memory page three-level according to space size
Storage structure, and described file data is stored at least one described memory page;Described file directory information records each institute
State file data at least one storage address corresponding;
Or, described storage subsystem, for will be stored in the file data in described storage subsystem be divided into some
Sub-file data, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information
Record the corresponding disk address of each described sub-file data.
Preferably, also include:Described task management subsystem, is pipe for the task type in described file operation task
During reason file task, described file operation task is sent to described directory subsystem;Described management file task includes creating
File task, deletion file task, modification file task or locating file task;
Described directory subsystem, for receiving and executing described file operation task.
Preferably, described directory subsystem, is establishment file task for the task type in described file operation task
When, judge whether the file size of described file operation task file to be created has determined that;
Described directory subsystem, for when described file size has determined that, by described task management subsystem to institute
State storage subsystem and send application spatial cache request, receive the distribution spatial cache of described storage subsystem;Based on described literary composition
Part operation task generates an establishment file directory information, and described establishment file directory information is related to described distribution spatial cache
Connection, and described establishment file directory information is added in described preset data structure;
Described directory subsystem, for when described file size does not determine, generating one based on described file operation task
Establishment file directory information, and described establishment file directory information is added in described preset data structure.
The present invention also provides a kind of file management method of the Virtual File System based on DAS framework storage server, institute
State Virtual File System and include task management subsystem and cache subsystem;
Described task management subsystem receives the file operation task from requesting terminal, in described file operation task
When task type is reading and writing of files task, described file operation task is sent to described cache subsystem;
Described cache subsystem receives described file operation task;It is to read literary composition in the task type of described file operation task
During part task, obtain the to be read file data corresponding with described file operation task from spatial cache, and treat described
Read file data and be sent to described task management subsystem;Or, it is to write literary composition in the task type of described file operation task
During part task, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem receives described file data to be read, and described file data to be read is sent to
Described requesting terminal.
Preferably, described Virtual File System also includes storage subsystem;
Described cache subsystem judges that receiving the task type of described file operation task still writes as reading file task
File task;
Described cache subsystem, when the task type receiving described file operation task is to read file task, judges slow
Deposit and in space, whether there is the to be read file data corresponding with described file operation task;If existing, update described slow
Deposit the file data in space, and described file data to be read is sent to described task management subsystem;If not existing,
Described file operation task is sent to described storage subsystem;
Described storage subsystem receives described file operation task, obtains and described corresponding the continuing of file operation task
Take file data, and described file data to be read is sent to described cache subsystem;
Described cache subsystem receives the file data described to be read from described storage subsystem, will be described to be read
File data is added in described spatial cache, updates the file data in described spatial cache;And by described file to be read
Data is activation gives described task management subsystem;
Described cache subsystem receive described file operation task task type be written document task when, described literary composition
Part operation task includes described file data to be written;Described file data to be written is added in described spatial cache, more
The newly file data of described spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem receives described file operation task, and stores described file data to be written.
Preferably, described Virtual File System also includes setting up the catalogue subsystem of internal memory mapping relations with host operating system
System;The corresponding literary composition of file data that described directory subsystem is stored in described storage subsystem using preset data structure management
Part directory information;
Disk is divided into memory paragraph, memory block and memory page tertiary storage knot according to space size by described storage subsystem
Structure, and described file data is stored at least one described memory page;Described file directory information records each described literary composition
Number of packages is according at least one storage address corresponding;
Or, the file data that described storage subsystem will be stored in described storage subsystem is divided into some subfiles
Data, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information record is every
The corresponding disk address of sub-file data described in.
Preferably, also include:
Described task management subsystem, will when judging the task type of described file operation task for management file task
Described file operation task is sent to described directory subsystem;Described management file task includes establishment file task, deletes literary composition
Part task, modification file task or locating file task;
Described directory subsystem receives and executes described file operation task.
Preferably, also include:Described directory subsystem is appointed for establishment file in the task type of described file operation task
During business, judge whether the file size of described file operation task file to be created has determined that;
Described directory subsystem when described file size has determined that, by described task management subsystem to described storage
Subsystem sends application spatial cache request, receives the distribution spatial cache of described storage subsystem;Based on described file operation
Task generates an establishment file directory information, and described establishment file directory information is associated with described distribution spatial cache, and
Described establishment file directory information is added in described preset data structure;
Described directory subsystem, when described file size does not determine, generates one based on described file operation task and creates literary composition
Part directory information, and described establishment file directory information is added in described preset data structure.
The present invention compared with prior art has the advantage that:Provided by the present invention based on DAS framework storage server
Virtual File System and its file management method, task management subsystem receive task type be reading and writing of files task literary composition
During part operation task, file operation task is sent to cache subsystem;Cache subsystem is in the task class of file operation task
When type is to read file task, obtains the to be read file data corresponding with file operation task from spatial cache and be sent to times
Business management subsystem;When the task type of file operation task is written document task, will be corresponding with file operation task
File data to be written writes spatial cache.In this Virtual File System setting cache subsystem to file operation task at
Reason, maskable to disk or disk array repeat read, to reduce the read-write to disk or disk array for the file operation task
Pressure, reduces the IO expenditure of disk or disk array, to ensure the IO performance of disk or disk array.
Brief description
Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:
Fig. 1 is the theory diagram of the Virtual File System in the embodiment of the present invention 1 based on DAS framework storage server.
Fig. 2 is the file management method of the Virtual File System in the embodiment of the present invention 2 based on DAS framework storage server
A flow chart.
In figure:11st, task management subsystem;12nd, cache subsystem;13rd, storage subsystem;14th, directory subsystem.
Specific embodiment
In order to be more clearly understood to the technical characteristic of the present invention, purpose and effect, now comparison accompanying drawing describes in detail
The specific embodiment of the present invention.
Embodiment 1
Fig. 1 illustrates the theory diagram of the Virtual File System based on DAS framework in the present embodiment.Based on DAS framework
In storage server, storage server is joined directly together by scsi interface with disk or disk array, sets in storage server
Have for realizing the file data on disk or disk array is carried out with IO read-write and storage system maintenance management, data backup and resume
The host operating system of the functions such as requirement;Wherein, disk array includes at least two disks.The present embodiment provided based on
In the Virtual File System of DAS framework storage server, Virtual File System and host operating system pre-build that internal memory maps
Relation, specifically includes task management subsystem 11, cache subsystem 12, directory subsystem 14 and storage subsystem 13;Wherein, mesh
Record subsystem 14 sets up internal memory mapping relations with host operating system.
Directory subsystem 14 is used for each number of files being stored in Virtual File System using preset data structure management
According to corresponding file directory information.Specifically, the file directory information in directory subsystem 14 is stored in internal memory mapping mode
In host operating system so that the file directory information in directory subsystem 14 can real-time update, and then ensure file directory
The safety of information and effective in real time.Because directory subsystem 14 and host operating system set up internal memory mapping relations so that every time
When host operating system starts, it is both needed to load directory subsystem 14.In the present embodiment, directory subsystem 14 is used for managing virtual literary composition
All of file data in part system, the file directory information (including file size, storage address etc.) of log file data, real
Existing establishment file, deletion file, the modification management file task such as file and locating file.Directory subsystem 14 institute in the present embodiment
Using preset data structure be red-black data tree structure so that execution establishment file, deleting file, modification file and search literary composition
The management such as part file task is positively retained under relatively low time complexity.It is to be appreciated that keeping relatively low time complexity
In the case of, preset data structure can also be other data structures.
Task management subsystem 11 is responsible for receiving and manage the file behaviour of the access Virtual File System of requesting terminal transmission
Make task, the task type of file operation task includes reading and writing of files task and management file task;Wherein, reading and writing of files task
Including reading file task and written document task;Management file task includes establishment file task, deletes file task, modification file
Task and locating file task.Each file operation task correspond to a task data block, this task data block be used for storage with
The related mission bit stream of task, including but not limited to task type, file name, file size, creation time, tasks carrying shape
State, code check, video type, compressed format etc..In addition, task data block does not store specific file data, but and file data
Storage address be associated.Specifically, when the task type in file operation task is to read file task, by its corresponding task
Data block is associated with the storage address of the file data to be read getting;It is to write literary composition in the task type of file operation task
During part task, its corresponding task data block is associated with the storage address of file data to be written.In Virtual File System
In, the communication between each subsystem of Virtual File System is realized based on the task data block of file operation task and calls.Task pipe
File operation task can be sent to by reason subsystem 11 according to the task type on the task data block of each file operation task
Corresponding subsystem is processed, and result is returned to the requesting terminal sending file operation task.In the present embodiment,
Task management subsystem 11, when the task type of the file operation task receiving is reading and writing of files task, file operation is appointed
Business is sent to cache subsystem 12 and is processed;It is management file task in the task type of the file operation task receiving
When, file operation task is sent to directory subsystem 14 and is processed.
Cache subsystem 12 is used for managing the All Files data of caching in spatial cache, receives task management subsystem 11
The task type sending is the file operation task of reading and writing of files task, and is reading and writing of files task to the task type receiving
File operation task processed.Specifically, cache subsystem 12 receives task type is the file behaviour reading file task
When making task, first check in spatial cache and whether there is the to be read file data corresponding with file operation task, if existing
File data to be read, then directly return to task management subsystem 11 by the file data to be read of caching, and passes through task
Management subsystem 11 is sent to requesting terminal;If there is not file data to be read, file operation task is sent to storage
Subsystem 13, obtains file data to be read from storage subsystem 13 and returns to task management subsystem 11.Cache subsystem
12 receive the file operation task that task type is written document task (now comprises number of files to be written in file operation task
According to) when, first the file data to be written that file operation task is comprised can be stored in spatial cache, update spatial cache
File data, is then written into file data and sends storage subsystem 13, store disk or disk by storage subsystem 13
On array.
It is to be appreciated that cache subsystem 12 is in advance to host operating system application spatial cache, as virtual file system
The caching of system, and it is independently operated, the 2 of the cache subsystem spatial caches that application is arrived in units of page size, it is divided into multiple
Caching page.Cache subsystem 12 is responsible for the caching page status of each caching page in spatial cache, to determine that caching page is in sky
Not busy state or use state.If being in idle condition, can write direct new file data caching to caching page;If place
In use state, i.e. storage address in storage subsystem 13 for the file data of caching, this storage address in record buffer memory page
Including storage sector address in disk or disk array for the file data, MBA memory block address and storage page address, by this storage
Address can directly position corresponding file data in internal memory.
Further, cache subsystem 12 to the file data to be read being got based on file operation task (from depositing
Storage subsystem 13) or file data to be written (from file operation task), it is both needed to cache portion in spatial cache, stored
On the caching page of idle condition, form the file data of spatial cache, be to read file in order to receive task type again
During the file operation task of task, can directly return obtaining file data to be read accordingly in the file data of spatial cache
To task management subsystem 11, file operation task need not be sent to storage subsystem 13 to obtain file to be read accordingly
Data, thus filtering out a part of task type is the file operation task reading file task so that only in spatial cache
Do not exist in file data accordingly file data to be read when just can obtain file task to be read from storage subsystem 13, with
Mitigate the IO pressure of disk.It is to be appreciated that when file data to be read or file data to be written are added spatial cache,
Need to first check whether spatial cache is full;If spatial cache is full, does not exist in spatial cache and can cache number of files to be read
According to or file data to be written the caching page being in idle condition, now need using default life cycle algorithm, eliminate old slow
Deposit the file data of caching on page.This caching life cycle algorithm includes but is not limited to RAND algorithm, FIFO algorithm, LFU algorithm, LRU
Algorithm, OPT algorithm.
Storage subsystem 13 is responsible for disk or disk array, and coordinates and complete file operation task.Specifically, store
Subsystem 13 is responsible for disk or disk array in storage server, and disk or disk array are divided into according to space size
Memory paragraph, memory block and memory page tertiary storage structure (disk or disk array will be divided into some storages according to space size
Section, each memory page is divided into some memory blocks, and each memory block is divided into some memory pages), and file data is stored in
On at least one memory page.Storage subsystem 13 goes back definition data structure to manage the state of memory paragraph, memory block and memory page
Information, and determine that storage organizations at different levels (including memory paragraph, memory block and memory page) are in idle condition or use state.If
Then represent that corresponding disk space is sky for idle condition, can write direct file data;If use state then represents corresponding
Disk space occupied, not directly write file data.Storage subsystem 13 also sets up Task-list, is derived from for receiving
The task type of cache subsystem 12 is the file operation task of reading and writing of files task;And the task type in file operation task
During for reading file task, file data to be read being obtained from disk or disk array and returns to cache subsystem 12, now
Do not change the file data on disk or disk array;When the task type of file operation task is written document task, will treat
Write file data write disk or disk array, the file data on modification disk or disk array, and it is written into file
The file directory information of data is added in directory subsystem 14.
In a kind of specific embodiment, disk or disk array are divided into storage according to space size by storage subsystem 13
Section, memory block and memory page tertiary storage structure, and file data is stored at least one memory page;Correspondingly, catalogue
File directory information in subsystem 14 records the corresponding storage address of each file data, and this storage address includes memory paragraph ground
Location, MBA memory block address and storage page address.Because disk or disk array are divided into memory paragraph, memory block and memory page three-level
Storage organization, when file data is stored in disk or disk array, can be stored on W memory page, W memory page may belong to
M memory block, M memory block may belong to N number of memory paragraph, and N number of memory paragraph may belong to different disks.Correspondingly, catalogue
The storage sector address of N number of memory paragraph, N number of memory paragraph ground is recorded in the corresponding file directory information of this document data in subsystem 14
M MBA memory block address is recorded, W storage page address of record under M MBA memory block address under location.It is to be appreciated that each memory page
Space size be generally the integral multiple of physical disk page so that disk or disk array are obtaining file data to be read or writing
Enter during file data to be written and just to correspond to physical disk page, unnecessary to avoid producing for the non-integral multiple of physical disk page
Read-write operation.If memory page is 1.5 times of physical disk pages, need to be to physics when task type is reading and writing of files task executing
Disk page is read and write 2 times.Physical disk page is usually 512K or 1024K, correspondingly, defines the storage subsystem of Virtual File System
The disk space size of 13 memory page is 512K or 1024K.
In another specific embodiment, the file data that storage subsystem 13 will be stored in storage subsystem 13 divides
Become some sub-file data, and some sub-file data are uniformly distributed and are stored on all disks;File directory information is remembered
Record the corresponding disk address of each sub-file data.Disk in the present embodiment is storage disk and non-system disk.In this tool
In body embodiment, disk or disk array need not be divided into memory paragraph, memory block and memory page tertiary storage structure, but
In the file data distribution disk space to disk to be written or disk array, first file data is cut into N number of subfile
Data, more N number of sub-file data is uniformly distributed and is stored on all disks.If file data to be written is XXX.ts, deposit
File data is cut into XXX_0.ts by Chu Shixu, the sub-file data such as XXX_1.ts, XXX_2.ts, and by above-mentioned Ziwen number of packages
According to being uniformly distributed and be stored on all disks.In this specific embodiment, disk worker thread is not direct read/write disk,
And be intended to open corresponding sub-file data, and complete reading and writing of files task.Correspondingly, the file directory in directory subsystem 14
Information record is not the storage address that the corresponding memory paragraph of file data, memory block and memory page are formed, but record literary composition
Number of packages evidence is cut into which sub-file data and the corresponding disk address of each sub-file data.
No matter it is to be appreciated that being that disk or disk array are carried out with memory paragraph, memory block and memory page tertiary storage knot
Structure divides, and file data is stored at least one memory page;Or file data is divided into some sub-file data,
And some sub-file data are uniformly distributed and are stored on all disks, a file data all can be made to be stored in different disks
On, substantially reduce the probability that user accesses file data on same disk.Because file data is respectively stored in different disk
On, when file data is conducted interviews can using different disks, thus realize big concurrently access disk when, visit capacity is point
It is fitted on different disks, this avoid the impact to overall performance for the monolithic disk I/O short slab.
It is to be appreciated that when being started based on DAS framework storage server, the startup of its host operating system, correspondingly, with
The Virtual File System that host operating system sets up internal memory mapping relations also starts, i.e. task management subsystem 11 and caching subsystem
System 12, storage subsystem 13 and directory subsystem 14 start simultaneously.Wherein, directory subsystem 14 loads host service function system when starting
The corresponding file directory information of file data of system, using RBTree data structure managing file directory information, this document catalogue
Information is Memory Mapping File, when directory subsystem 14 changes the file directory information of internal memory, the literary composition in host operating system
Part directory information also can synchronized update.Task management subsystem 11 completes the initialization of data structure when starting, for receiving
File operation task from requesting terminal and according to task type by file operation task be sent respectively to cache subsystem 12 or
Directory subsystem 14.Can first apply for empty as caching in a bulk of internal memory when cache subsystem 12 starts to storage subsystem 13
Between, and internal memory is divided into multiple caching pages, set up the caching laminated of all caching pages on data structure managing spatial cache
State.It is to be appreciated that when cache subsystem 12 starts, on spatial cache, all of caching page is in idle condition.Storage
Obtain, from storage server, disk or the disk array that can be used to storage file data when subsystem 13 starts, getting magnetic
When disk or disk array, each disk is numbered, is that each disk creates a disk worker thread, disk worker thread is used
In the disk belonging to oneself is operated as big file (will disk or disk array as a big file, magnetic
Disk worker thread operation when direct read/write disk, and need not on disk establishment file), and will big file according to disk space big
Little be divided into memory paragraph, memory block and memory page tertiary storage structure, set up data structure managing memory paragraph, memory block and storage
The status information of page.Storage subsystem 13 also creates Task-list corresponding worker thread when starting, and is responsible for receiving file operation
File operation task is simultaneously distributed to each disk worker thread, and the result of execution file operation task is returned by task.
The present embodiment provides a kind of Virtual File System based on DAS framework storage server, for realizing to being stored in
File data in storage subsystem 13 conducts interviews (inclusion read-write operation), that is, be used for being appointed for reading and writing of files according to task type
The file operation task of business obtains file data to be read or is written into file data and is stored in storage subsystem 13.Specifically
Ground, should be included based on the Virtual File System of DAS framework storage server:
Task management subsystem 11, for receiving the file operation task from requesting terminal, in file operation task
When task type is reading and writing of files task, file operation task is sent to cache subsystem 12.I.e. task management subsystem 11
When receiving file operation task, appointing of file operation task, need to be determined according to file operation task corresponding task data block
Service type.Specifically, the task type of file operation task includes reading and writing of files task and management file task;Wherein, read and write
File task includes reading file task and written document task;Management file task include establishment file task, delete file task,
Modification file task and locating file task.
Cache subsystem 12, for receiving file operation task;It is to read file to appoint in the task type of file operation task
During business, from spatial cache, obtain the to be read file data corresponding with file operation task, and by file data to be read
It is sent to task management subsystem 11;Or, when the task type of file operation task is written document task, by file operation
The corresponding file data to be written of task writes spatial cache.It is to be appreciated that the task type in file operation task is to write
During file task, file operation task comprises file data to be written, is written into file data write spatial cache and caches,
So that when cache subsystem 12 receives next task type for reading file task, can be accordingly to be read from spatial cache acquisition
File data, and it is sent to task management subsystem 11.
Task management subsystem 11, for receiving file data to be read, and File data is activation to be read is given
Requesting terminal, so that requesting terminal obtains to be read file data corresponding with file operation task.
In this specific embodiment, task management subsystem 11 is receiving the file that task type is reading and writing of files task
During operation task, this document operation task is sent to cache subsystem 12;Cache subsystem 12 appointing in file operation task
When service type is to read file task, obtains the to be read file data corresponding with file operation task from spatial cache and send
To task management subsystem 11;When the task type of file operation task is written document task, will be with file operation task phase
Corresponding file data to be written writes spatial cache.In this Virtual File System, setting cache subsystem 12 is appointed to file operation
Business is processed, maskable to disk or disk array repeat read, to reduce file operation task to disk or disk battle array
The read-write pressure of row, reduces the IO expenditure of disk or disk array, to ensure the IO performance of disk or disk array.
Further, the Virtual File System based on DAS framework storage server, is being to read file task to task type
File operation task when being processed, maskable to same file data to be read repeat read.
Specifically, cache subsystem 12, during for being to read file task in the task type receiving file operation task,
Judge to whether there is the to be read file data corresponding with file operation task in spatial cache.If in spatial cache exist with
The corresponding file data to be read of file operation task, then update the file data in spatial cache, and by file to be read
Data is activation is to task management subsystem 11.If there is not the to be read number of files corresponding with file operation task in spatial cache
According to then file operation task being sent to storage subsystem 13.
Storage subsystem 13, for receiving file operation task, obtains the to be read literary composition corresponding with file operation task
Number of packages evidence, and file data to be read is sent to cache subsystem 12.
Cache subsystem 12, for receiving the file data to be read from storage subsystem 13, by number of files to be read
According to being added in spatial cache, update the file data in spatial cache;And file data to be read is sent to task management
Subsystem 11.
The file operation task that is based on cache subsystem 12 reads file data to be read and includes random read take and order reading
Two kinds of situations.Original position and the file size of file data to be read in random read take, need to be indicated, calculate original position
In the concrete caching page of spatial cache, begin look for spatial cache with the presence or absence of relative with file operation task from concrete caching page
The file data to be read answered, if there is file data to be read accordingly in spatial cache, updates spatial cache caching
File data, is eliminated with preventing file data to be read to be buffered the default life cycle algorithm of subsystem 12, and by file to be read
Data is activation is to task management subsystem 11;If there is not file data to be read accordingly in spatial cache, file is grasped
It is sent to storage subsystem 13 as task, file data to be read is read by storage subsystem 13.When order reads, may specify
The original position of file data to be read, it would however also be possible to employ the original position of acquiescence, and according to original position and file size meter
Calculate the concrete caching page in spatial cache for the original position, begin look for spatial cache from concrete caching page and whether there is and file behaviour
Make the corresponding file data to be read of task, if finding, file data to be read being replicated and returns to task management subsystem
11;If not finding, it is submitted to storage subsystem 13, read operation is completed by storage subsystem 13.
It is to be appreciated that when cache subsystem 12 receives the file operation task that task type is reading file task, first
Whether there is the to be read file data corresponding with file operation task from caching spatial lookup, exist in spatial cache and continue
When taking file data, file data to be read is sent to task management subsystem 11, to complete this document operation task, need not
File operation task is sent to storage subsystem 13, identical file data to be read from shielding is to disk or disk array
Repeat read, lead to disk or disk array nervous, the IO performance of impact disk disk array.Due to file data to be read
It is stored at least one memory page, or file data to be read is divided into some sub-file data and is stored in different disks
On, when storage subsystem 13 is based on file operation task and accesses disk or disk array, file operation task can be assigned to
On different disks or disk array, to reach pressure equalization, to reduce the IO pressure of disk or disk array.From caching
In system 12 direct access file data to be read or receive from storage subsystem 13 file data to be read when, be both needed to more
The file data of new spatial cache, is eliminated with avoiding file data to be read to be buffered the default life cycle algorithm of subsystem 12, no
Method realizes shielding the file operation task duplication reading to identical file data to be read in cache subsystem 12.
In a specific embodiment, based on the Virtual File System of DAS framework storage server, realizing to task class
Type is the file operation task of written document task when being processed, to shield subsequently this document operation task is comprised to be written
Enter file data repeats reading.
Specifically, cache subsystem 12, during for being written document task in the task type receiving file operation task,
File operation task includes file data to be written;It is written into file data to be added in spatial cache, update spatial cache
File data;And file operation task is sent to storage subsystem 13.It is to be appreciated that each task type is written document
The file operation task of task all includes file data to be written, when cache subsystem 12 receives this document operation task, needs
Corresponding for file operation task file data to be written being write spatial cache, forming the file data of spatial cache so that delaying
Deposit subsystem 12 when subsequently received task type is the file operation task reading file data, can be by newly from spatial cache
The file data of write spatial cache is sent to task management subsystem 11 as file data to be read accordingly, right to shield
In disk or disk array, identical file data to be read repeats reading, it is to avoid disk or disk array are nervous, thus shadow
Ring the IO performance of disk or disk array.
Storage subsystem 13, for receiving file operation task, and stores file data to be written.Specifically, store son
Disk or disk array are divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by system 13, and will
File data is stored at least one memory page.Or, storage subsystem 13 will be stored in the file in storage subsystem 13
Data is divided into some sub-file data, and some sub-file data are evenly distributed on disk.It is to be appreciated that will be to be written
Enter file data to be stored at least one memory page, or be written into file data and be divided into some sub-file data storages
On different disks, file data is conducted interviews (include reading and writing of files task) when can be using different disks, thus real
Now big when concurrently accessing disk, visit capacity is assigned on different disks so that the IO reduced pressure of each disk, so
Avoid the impact to overall performance for the monolithic disk I/O short slab.
Cache subsystem 12 is based on file operation task write file data to be written and includes random writing and be sequentially written in
Two kinds of situations.When being sequentially written in, the writing position being written into that file data writes spatial cache need not be specified, cache subsystem
System 12, according to the file size of file data to be written, is written into the file data in file data write current cache space
Afterbody, and be written into file data and be submitted to storage subsystem 13, file data write is written into by storage subsystem 13
On concrete disk, complete write operation.In the case of random writing, file data to be written need to be specified in the write of spatial cache
Position, cache subsystem 12, according to original position, determines concrete caching page, and is written into the file data concrete caching page of write
In.Meanwhile, task type is sent to storage subsystem 13 for the file operation task of written document task by cache subsystem 12, by
Storage subsystem 13 is written into file data and writes on concrete disk, completes write operation.
The present embodiment provides a kind of Virtual File System based on DAS framework storage server, is used for realizing task type
File operation task for file management tasks is processed, and this document management role includes establishment file task, deletes file
Task, modification file task or locating file task.Specifically, it is somebody's turn to do the Virtual File System based on DAS framework storage server
Including:
Task management subsystem 11, for when the task type of file operation task is for management file task, by file
Operation task is sent to directory subsystem 14.Wherein, management file task includes establishment file task, deletes file task, repaiies
Change file task or locating file task.
Directory subsystem 14, for receiving and executing file operation task.Using red-black data tree structure in the present embodiment
The corresponding file directory information of each file data in management storage subsystem 13, directory subsystem 14 receives task type
For manage file task file operation task when, the establishment to file can be realized respectively, delete, repair according to file operation task
Change or locating function.
Further, when the task type of file operation task is establishment file task, should be based on DAS framework storage clothes
The Virtual File System of business device also includes:
Directory subsystem 14, for when the task type of file operation task is establishment file task, judging that file is grasped
Whether the file size making task file to be created has determined that.It is to be appreciated that directory subsystem 14 is receiving task
When type is the file operation task of establishment file task, this document is determined according to file operation task corresponding task data block
Whether the file size of operation task file to be created has determined that.
Directory subsystem 14, for when file size has determined that, by task management subsystem 11 to storage subsystem
13 send application spatial cache request, receive the distribution spatial cache of storage subsystem 13;Generate one based on file operation task
Establishment file directory information, establishment file directory information is associated with distribution spatial cache, and by establishment file directory information
It is added in preset data structure.Under the scenes such as VOD order video, the file size of file operation task file to be created
Have determined that.Specifically, directory subsystem 14 need to first pass through task management subsystem 11 and send application X size to storage subsystem 13
The spatial cache request in space, to call storage subsystem 13 interface.After storage subsystem 13 receives spatial cache request, meter
Calculate number Y of the corresponding memory paragraph in X size space, and Y memory paragraph is divided equally on all disks, empty to form distribution caching
Between.Corresponding for the distribution distributing spatial cache segment data structure is returned to directory subsystem 14, hop count by storage subsystem 13
The storage block message being comprised according to record storage section in structure.Directory subsystem 14 is based on file operation task and generates an establishment literary composition
Part directory information, and this establishment file directory information is associated with receiving distribution spatial cache corresponding segment data structure.
Establishment file directory information is added in red-black data tree structure directory subsystem 14, completes a file size fixed
The establishment of file.It is to be appreciated that thinner in order to get, also X size space can be divided for unit by memory block, be calculated
The quantity of memory block, and all memory blocks are divided equally on all disks, to form distribution spatial cache;Correspondingly, store son
System 13 returns block data structure, the storage page information that in block data structure, record storage block is comprised to directory subsystem 14.
Directory subsystem 14, for when file size does not determine, generating an establishment file mesh based on file operation task
Record information, and establishment file directory information is added in preset data structure.In file operation task file to be created
In the case of file size is undetermined, only an establishment file directory information need to be generated based on file operation task, then literary composition will be created
Part directory information is added in red-black data tree structure, you can complete the establishment of the undetermined file of file size, will
Provisional application spatial cache again during the file data to be written write undetermined file of file size, each provisional application can be applied for
The spatial cache of one memory paragraph, is written into the spatial cache that file data write application is arrived, and discharges spatial cache (storage
Section) not have use memory block.
The Virtual File System based on DAS framework storage server that the present embodiment is provided, by setting up storage subsystem
System 13, when making multiple request terminal accesses identical file data, actual access is file data in spatial cache, not right
Disk or disk array cause to read and write pressure, reduce the IO expenditure of disk or disk array.In addition, storage subsystem 13 is by disk
Or disk array is divided into memory paragraph, memory block and memory page tertiary storage structure according to space size, and file data is stored
On at least one memory page;Or, if the file data that storage subsystem 13 will be stored in storage subsystem 13 is divided into
Dry sub-file data, and some sub-file data are evenly distributed on all disks.Above two storage mode, all can make
One file data is stored on different disks, reduces the probability that user accesses the file data on same disk, concurrent greatly
When accessing disk, visit capacity is distributed to all disks, it is to avoid monolithic disk I/O short slab impacts to whole performance.
Embodiment 2
The present embodiment provides a kind of file management method of the Virtual File System based on DAS framework storage server, uses
In realizing the file data being stored in storage subsystem 13 is conducted interviews (inclusion read-write operation), that is, it is used for according to task class
Type is that the file operation task of reading and writing of files task obtains file data to be read or is written into file data and is stored in storage
Subsystem 13.Specifically, the file management method being somebody's turn to do the Virtual File System based on DAS framework storage server includes:
S10:Task management subsystem 11 receives the file operation task from requesting terminal, and determines file operation task
Task type.Specifically, the task type of file operation task includes reading and writing of files task and management file task;Wherein,
Reading and writing of files task includes reading file task and written document task;Management file task includes establishment file task, deletes file
Task, modification file task and locating file task.
S20:Task management subsystem 11, when the task type of file operation task is reading and writing of files task, file is grasped
It is sent to cache subsystem 12 as task.I.e. task management subsystem 11, need to be according to file when receiving file operation task
Operation task corresponding task data block determines the task type of file operation task.
S30:Cache subsystem 12 receives file operation task;It is to read file task in the task type of file operation task
When, obtain the to be read file data corresponding with file operation task from spatial cache, and file data to be read is sent out
Give task management subsystem 11;Or, when the task type of file operation task is written document task, file operation is appointed
Corresponding file data to be written of being engaged in writes spatial cache.It is to be appreciated that the task type in file operation task is to write literary composition
During part task, file operation task comprises file data to be written, is written into file data write spatial cache and caches, with
When making cache subsystem 12 receive next task type for reading file task, can be from the literary composition accordingly to be read of spatial cache acquisition
Number of packages evidence, and it is sent to task management subsystem 11.
S40:Task management subsystem 11 receives file data to be read, and by File data is activation to be read give please
Seek terminal, so that requesting terminal obtains to be read file data corresponding with file operation task.
S50:Task management subsystem 11, when the task type of file operation task is management file task, file is grasped
It is sent to directory subsystem 14 as task.Wherein, management file task includes establishment file task, deletes file task, modification
File task or locating file task.
S60:Directory subsystem 14 receives and executes file operation task.Using red-black data tree structure pipe in the present embodiment
The corresponding file directory information of each file data in reason storage subsystem 13, directory subsystem 14 receives task type and is
During the file operation task of management file task, the establishment to file, deletion, modification can be realized respectively according to file operation task
Or locating function.
In this specific embodiment, task management subsystem 11 is receiving the file that task type is reading and writing of files task
During operation task, this document operation task is sent to cache subsystem 12;Cache subsystem 12 appointing in file operation task
When service type is to read file task, obtains the to be read file data corresponding with file operation task from spatial cache and send
To task management subsystem 11;When the task type of file operation task is written document task, will be with file operation task phase
Corresponding file data to be written writes spatial cache.In this Virtual File System, setting cache subsystem 12 is appointed to file operation
Business is processed, maskable to disk or disk array repeat read, to reduce file operation task to disk or disk battle array
The read-write pressure of row, reduces the IO expenditure of disk or disk array, to ensure the IO performance of disk or disk array.
Further, the file management method of the Virtual File System based on DAS framework storage server, to task class
Type is the file operation task reading file task when being processed, maskable to same file data to be read repeat read,
I.e. step S20 specifically includes:
S21:Cache subsystem 12, when the task type receiving file operation task is to read file task, judges caching
Whether there is the to be read file data corresponding with file operation task in space.If existing and file operation in spatial cache
The corresponding file data to be read of task, then update the file data in spatial cache, and file data to be read sent
To task management subsystem 11.If there is not the to be read file data corresponding with file operation task in spatial cache, will
File operation task is sent to storage subsystem 13.
S22:Storage subsystem 13 receives file operation task, obtains the to be read file corresponding with file operation task
Data, and file data to be read is sent to cache subsystem 12.
S23:Cache subsystem 12 receives the file data to be read from storage subsystem 13, by file data to be read
It is added in spatial cache, update the file data in spatial cache;And file data to be read is sent to task management
System 11.
The file operation task that is based on cache subsystem 12 reads file data to be read and includes random read take and order reading
Two kinds of situations.Original position and the file size of file data to be read in random read take, need to be indicated, calculate original position
In the concrete caching page of spatial cache, begin look for spatial cache with the presence or absence of relative with file operation task from concrete caching page
The file data to be read answered, if there is file data to be read accordingly in spatial cache, updates spatial cache caching
File data, is eliminated with preventing file data to be read to be buffered the default life cycle algorithm of subsystem 12, and by file to be read
Data is activation is to task management subsystem 11;If there is not file data to be read accordingly in spatial cache, file is grasped
It is sent to storage subsystem 13 as task, file data to be read is read by storage subsystem 13.When order reads, may specify
The original position of file data to be read, it would however also be possible to employ the original position of acquiescence, and according to original position and file size meter
Calculate the concrete caching page in spatial cache for the original position, begin look for spatial cache from concrete caching page and whether there is and file behaviour
Make the corresponding file data to be read of task, if finding, file data to be read being replicated and returns to task management subsystem
11;If not finding, it is submitted to storage subsystem 13, read operation is completed by storage subsystem 13.
It is to be appreciated that when cache subsystem 12 receives the file operation task that task type is reading file task, first
Whether there is the to be read file data corresponding with file operation task from caching spatial lookup, exist in spatial cache and continue
When taking file data, file data to be read is sent to task management subsystem 11, to complete this document operation task, need not
File operation task is sent to storage subsystem 13, identical file data to be read from shielding is to disk or disk array
Repeat read, lead to disk or disk array nervous, the IO performance of impact disk disk array.Due to file data to be read
It is stored at least one memory page, or file data to be read is divided into some sub-file data and is stored in different disks
On, when storage subsystem 13 is based on file operation task and accesses disk or disk array, file operation task can be assigned to
On different disks or disk array, to reach pressure equalization, to reduce the IO pressure of disk or disk array.From caching
In system 12 direct access file data to be read or receive from storage subsystem 13 file data to be read when, be both needed to more
The file data of new spatial cache, is eliminated with avoiding file data to be read to be buffered the default life cycle algorithm of subsystem 12, no
Method realizes shielding the file operation task duplication reading to identical file data to be read in cache subsystem 12.
In a specific embodiment, the file management method of the Virtual File System based on DAS framework storage server,
When realizing task type is processed for the file operation task of written document task, appointed with shielding follow-up this document is operated
Be engaged in comprised file data to be written repeat read, that is, step S20 also includes:
S24:Cache subsystem 12 receive file operation task task type be written document task when, file operation
Task includes file data to be written;It is written into file data to be added in spatial cache, update the number of files of spatial cache
According to;And file operation task is sent to storage subsystem 13.It is to be appreciated that each task type is the literary composition of written document task
Part operation task all includes file data to be written, when cache subsystem 12 receives this document operation task, need to grasp file
Make the corresponding file data to be written of task and write spatial cache, form the file data of spatial cache so that cache subsystem
12, when subsequently received task type is the file operation task reading file data, will can newly write caching from spatial cache
The file data in space is sent to task management subsystem 11 as file data to be read accordingly, to shield to disk or magnetic
In disk array identical file data to be read repeat read, it is to avoid disk or disk array are nervous, thus affect disk or
The IO performance of disk array.
S25:Storage subsystem 13 receives file operation task, and stores file data to be written.Specifically, store subsystem
Disk or disk array 13 are divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by system, and by literary composition
Number of packages evidence is stored at least one memory page.Or, storage subsystem 13 will be stored in the number of files in storage subsystem 13
According to being divided into some sub-file data, and some sub-file data are evenly distributed on disk.It is to be appreciated that being written into
File data is stored at least one memory page, or is written into file data and is divided into some sub-file data being stored in
On different disks, file data is conducted interviews (include reading and writing of files task) when can be using different disks, thus realizing
In big concurrently access disk, visit capacity is assigned on different disks so that the IO reduced pressure of each disk, so keeps away
Exempt from the impact to overall performance for the monolithic disk I/O short slab.
Cache subsystem 12 is based on file operation task write file data to be written and includes random writing and be sequentially written in
Two kinds of situations.When being sequentially written in, the writing position being written into that file data writes spatial cache need not be specified, cache subsystem
System 12, according to the file size of file data to be written, is written into the file data in file data write current cache space
Afterbody, and be written into file data and be submitted to storage subsystem 13, file data write is written into by storage subsystem 13
On concrete disk, complete write operation.In the case of random writing, file data to be written need to be specified in the write of spatial cache
Position, cache subsystem 12, according to original position, determines concrete caching page, and is written into the file data concrete caching page of write
In.Meanwhile, task type is sent to storage subsystem 13 for the file operation task of written document task by cache subsystem 12, by
Storage subsystem 13 is written into file data and writes on concrete disk, completes write operation.
The present embodiment provides a kind of file management method of the Virtual File System based on DAS framework storage server, uses
Processed in realizing the file operation task that task type is file management tasks, this document management role includes establishment file
Task, deletion file task, modification file task or locating file task.
Further, when the task type of file operation task is establishment file task, should be based on DAS framework storage clothes
Step S60 of the file management method of Virtual File System of business device specifically includes:
S61:Directory subsystem 14, when the task type of file operation task is establishment file task, judges file operation
Whether the file size of task file to be created has determined that.It is to be appreciated that directory subsystem 14 is receiving task class
When type is the file operation task of establishment file task, determine that this document is grasped according to file operation task corresponding task data block
Whether the file size making task file to be created has determined that.
S62:Directory subsystem 14 when file size has determined that, by task management subsystem 11 to storage subsystem 13
Send application spatial cache request, receive the distribution spatial cache of storage subsystem 13;One wound is generated based on file operation task
Build file directory information, establishment file directory information is associated with distribution spatial cache, and establishment file directory information is added
It is added in preset data structure.Under the scenes such as VOD order video, the file size of file operation task file to be created is
Determine.Specifically, directory subsystem 14 need to first pass through task management subsystem 11 and send application X size sky to storage subsystem 13
Between spatial cache request, to call storage subsystem 13 interface.After storage subsystem 13 receives spatial cache request, calculate
Number Y of the corresponding memory paragraph in X size space, and Y memory paragraph is divided equally on all disks, empty to form distribution caching
Between.Corresponding for the distribution distributing spatial cache segment data structure is returned to directory subsystem 14, hop count by storage subsystem 13
The storage block message being comprised according to record storage section in structure.Directory subsystem 14 is based on file operation task and generates an establishment literary composition
Part directory information, and this establishment file directory information is associated with receiving distribution spatial cache corresponding segment data structure.
Establishment file directory information is added in red-black data tree structure directory subsystem 14, completes a file size fixed
The establishment of file.It is to be appreciated that thinner in order to get, also X size space can be divided for unit by memory block, be calculated
The quantity of memory block, and all memory blocks are divided equally on all disks, to form distribution spatial cache;Correspondingly, store son
System 13 returns block data structure, the storage page information that in block data structure, record storage block is comprised to directory subsystem 14.
S63:Directory subsystem 14, when file size does not determine, generates an establishment file catalogue based on file operation task
Information, and establishment file directory information is added in preset data structure.Literary composition in file operation task file to be created
In the case of part size is undetermined, only an establishment file directory information need to be generated based on file operation task, then by establishment file
Directory information is added in red-black data tree structure, you can completes the establishment of the undetermined file of file size, will treat
Provisional application spatial cache again during the write file data write undetermined file of file size, each provisional application can apply for one
The spatial cache of individual memory paragraph, is written into the spatial cache that file data write application is arrived, and discharges spatial cache (storage
Section) not have use memory block.
The file management method of the Virtual File System based on DAS framework storage server that the present embodiment is provided, leads to
Cross and set up storage subsystem 13, when making multiple request terminal accesses identical file data, actual access is in spatial cache
File data, disk or disk array are not caused to read and write pressure, reduce the IO expenditure of disk or disk array.In addition, depositing
Disk or disk array are divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by storage subsystem 13,
And file data is stored at least one memory page;Or, storage subsystem 13 will be stored in storage subsystem 13
File data is divided into some sub-file data, and some sub-file data are evenly distributed on all disks.Above two
Storage mode, all can make a file data be stored on different disks, reduce user and access the number of files on same disk
According to probability, big concurrently access disk when, visit capacity is distributed to all disks, it is to avoid monolithic disk I/O short slab is to whole property
Can impact.
The present invention is illustrated by several specific embodiments, it will be appreciated by those skilled in the art that, without departing from
In the case of the scope of the invention, the present invention can also be carried out with various conversion and equivalent substitute.In addition, being directed to particular condition or tool
Body situation, can make various modifications to the present invention, without deviating from the scope of the present invention.Therefore, disclosed in the present invention is not limited to
Specific embodiment, and the whole embodiments falling within the scope of the appended claims should be included.
Claims (10)
1. a kind of Virtual File System based on DAS framework storage server it is characterised in that include task management subsystem and
Cache subsystem;
Described task management subsystem, for receiving the file operation task from requesting terminal, in described file operation task
Task type be reading and writing of files task when, described file operation task is sent to described cache subsystem;
Described cache subsystem, for receiving described file operation task;It is to read in the task type of described file operation task
During file task, from spatial cache, obtain the to be read file data corresponding with described file operation task, and will be described
File data to be read is sent to described task management subsystem;Or, it is to write in the task type of described file operation task
During file task, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem, for receiving described file data to be read, and described file data to be read is sent
To described requesting terminal.
2. the Virtual File System based on DAS framework storage server according to claim 1 is it is characterised in that described
Virtual File System also includes storage subsystem;
Described cache subsystem, for judging that receiving the task type of described file operation task still writes as reading file task
File task;
Described cache subsystem, during for being to read file task in the task type receiving described file operation task, judges
Whether there is the to be read file data corresponding with described file operation task in spatial cache;If existing, update described
File data in spatial cache, and described file data to be read is sent to described task management subsystem;If not existing,
Then described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, obtains and described corresponding the treating of file operation task
Read file data, and described file data to be read is sent to described cache subsystem;
Described cache subsystem, for receiving the file data described to be read from described storage subsystem, continues described
Take file data to be added in described spatial cache, update the file data in described spatial cache;And by described literary composition to be read
Number of packages evidence is sent to described task management subsystem;
Described cache subsystem, during for being written document task in the task type receiving described file operation task, described
File operation task includes described file data to be written;Described file data to be written is added in described spatial cache,
Update the file data of described spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, and stores described file data to be written.
3. the Virtual File System based on DAS framework storage server according to claim 1 is it is characterised in that described
Virtual File System also includes setting up the directory subsystem of internal memory mapping relations with host operating system;
Described directory subsystem, for the file data pair being stored in using preset data structure management in described storage subsystem
The file directory information answered;
Described storage subsystem, for being divided into memory paragraph, memory block and memory page tertiary storage knot by disk according to space size
Structure, and described file data is stored at least one described memory page;Described file directory information records each described literary composition
Number of packages is according at least one storage address corresponding;
Or, described storage subsystem, it is divided into some Ziwens for will be stored in the file data in described storage subsystem
Number of packages evidence, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information record
The corresponding disk address of each described sub-file data.
4. the Virtual File System based on DAS framework storage server according to claim 3 is it is characterised in that also wrap
Include:
Described task management subsystem, for when the task type of described file operation task is for management file task, by institute
State file operation task and be sent to described directory subsystem;Described management file task includes establishment file task, deletes file
Task, modification file task or locating file task;
Described directory subsystem, for receiving and executing described file operation task.
5. the Virtual File System based on DAS framework storage server according to claim 4 is it is characterised in that also wrap
Include:
Described directory subsystem, for when the task type of described file operation task is establishment file task, judging described
Whether the file size of file operation task file to be created has determined that;
Described directory subsystem, for when described file size has determined that, being deposited to described by described task management subsystem
Storage subsystem sends application spatial cache request, receives the distribution spatial cache of described storage subsystem;Based on described file behaviour
Make task and generate an establishment file directory information, described establishment file directory information is associated with described distribution spatial cache,
And described establishment file directory information is added in described preset data structure;
Described directory subsystem, for when described file size does not determine, generating an establishment based on described file operation task
File directory information, and described establishment file directory information is added in described preset data structure.
6. a kind of file management method of the Virtual File System based on DAS framework storage server is it is characterised in that described void
Intend file system and include task management subsystem and cache subsystem;
Described task management subsystem receives the file operation task from requesting terminal, in the task of described file operation task
When type is reading and writing of files task, described file operation task is sent to described cache subsystem;
Described cache subsystem receives described file operation task;It is to read file to appoint in the task type of described file operation task
During business, from spatial cache, obtain the to be read file data corresponding with described file operation task, and will be described to be read
File data is sent to described task management subsystem;Or, appoint for written document in the task type of described file operation task
During business, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem receives described file data to be read, and described file data to be read is sent to described
Requesting terminal.
7. the file management method of the Virtual File System based on DAS framework storage server according to claim 6, its
It is characterised by, described Virtual File System also includes storage subsystem;
Described cache subsystem judges to receive the task type of described file operation task as reading file task or written document
Task;
Described cache subsystem, when the task type receiving described file operation task is to read file task, judges that caching is empty
Between in the presence or absence of the to be read file data corresponding with described file operation task;If existing, update described caching empty
Interior file data, and described file data to be read is sent to described task management subsystem;If not existing, by institute
State file operation task and be sent to described storage subsystem;
Described storage subsystem receives described file operation task, obtains the to be read literary composition corresponding with described file operation task
Number of packages evidence, and described file data to be read is sent to described cache subsystem;
Described cache subsystem receives the file data described to be read from described storage subsystem, by described file to be read
Data is added in described spatial cache, updates the file data in described spatial cache;And by described file data to be read
It is sent to described task management subsystem;
Described cache subsystem when the task type receiving described file operation task is written document task, grasp by described file
Include described file data to be written as task;Described file data to be written is added in described spatial cache, updates institute
State the file data of spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem receives described file operation task, and stores described file data to be written.
8. the file management method of the Virtual File System based on DAS framework storage server according to claim 6, its
It is characterised by, described Virtual File System also includes setting up the directory subsystem of internal memory mapping relations with host operating system;Institute
State the file data corresponding file mesh that directory subsystem is stored in described storage subsystem using preset data structure management
Record information;
Disk is divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by described storage subsystem, and
Described file data is stored at least one described memory page;Described file directory information records each described file data
At least one storage address corresponding;
Or, the file data that described storage subsystem will be stored in described storage subsystem is divided into some Ziwen numbers of packages
According to, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information records each
The corresponding disk address of described sub-file data.
9. the file management method of the Virtual File System based on DAS framework storage server according to claim 6, its
It is characterised by, also include:
Described task management subsystem, will be described when judging the task type of described file operation task for management file task
File operation task is sent to described directory subsystem;Described management file task includes establishment file task, deletion file is appointed
Business, modification file task or locating file task;
Described directory subsystem receives and executes described file operation task.
10. the file management method of the Virtual File System based on DAS framework storage server according to claim 9,
It is characterized in that, also include:
Described directory subsystem, when the task type of described file operation task is establishment file task, judges described file behaviour
Whether the file size making task file to be created has determined that;
Described directory subsystem when described file size has determined that, by described task management subsystem to described storage subsystem
System sends application spatial cache request, receives the distribution spatial cache of described storage subsystem;Based on described file operation task
Generate an establishment file directory information, described establishment file directory information is associated with described distribution spatial cache, and by institute
State establishment file directory information to be added in described preset data structure;
Described directory subsystem, when described file size does not determine, generates an establishment file mesh based on described file operation task
Record information, and described establishment file directory information is added in described preset data structure.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610843245.XA CN106407409A (en) | 2016-09-22 | 2016-09-22 | A virtual file system based on DAS architecture storage servers and a file management method thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610843245.XA CN106407409A (en) | 2016-09-22 | 2016-09-22 | A virtual file system based on DAS architecture storage servers and a file management method thereof |
Publications (1)
Publication Number | Publication Date |
---|---|
CN106407409A true CN106407409A (en) | 2017-02-15 |
Family
ID=57997704
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610843245.XA Pending CN106407409A (en) | 2016-09-22 | 2016-09-22 | A virtual file system based on DAS architecture storage servers and a file management method thereof |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106407409A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106973109A (en) * | 2017-03-31 | 2017-07-21 | 努比亚技术有限公司 | Broadcast message pushes control method and device |
CN113569277A (en) * | 2021-06-25 | 2021-10-29 | 北京鼎普科技股份有限公司 | Method and device for checking security file data and electronic equipment |
CN116049131A (en) * | 2022-06-10 | 2023-05-02 | 荣耀终端有限公司 | File management method, system, electronic equipment and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101398744A (en) * | 2007-09-26 | 2009-04-01 | 新奥特硅谷视频技术有限责任公司 | Method and system for implementing network data process by utilizing virtual file system technology |
CN101639848A (en) * | 2009-06-01 | 2010-02-03 | 北京四维图新科技股份有限公司 | Spatial data engine and method applying management spatial data thereof |
CN102004760A (en) * | 2010-11-03 | 2011-04-06 | 深圳市同洲电子股份有限公司 | Multimedia file storing and applying method, related device and system |
US20130339786A1 (en) * | 2012-06-19 | 2013-12-19 | Lsi Corporation | Smart active-active high availability das systems |
-
2016
- 2016-09-22 CN CN201610843245.XA patent/CN106407409A/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101398744A (en) * | 2007-09-26 | 2009-04-01 | 新奥特硅谷视频技术有限责任公司 | Method and system for implementing network data process by utilizing virtual file system technology |
CN101639848A (en) * | 2009-06-01 | 2010-02-03 | 北京四维图新科技股份有限公司 | Spatial data engine and method applying management spatial data thereof |
CN102004760A (en) * | 2010-11-03 | 2011-04-06 | 深圳市同洲电子股份有限公司 | Multimedia file storing and applying method, related device and system |
US20130339786A1 (en) * | 2012-06-19 | 2013-12-19 | Lsi Corporation | Smart active-active high availability das systems |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106973109A (en) * | 2017-03-31 | 2017-07-21 | 努比亚技术有限公司 | Broadcast message pushes control method and device |
CN106973109B (en) * | 2017-03-31 | 2020-06-09 | 南京秦淮紫云创益企业服务有限公司 | Broadcast message push control method and device |
CN113569277A (en) * | 2021-06-25 | 2021-10-29 | 北京鼎普科技股份有限公司 | Method and device for checking security file data and electronic equipment |
CN113569277B (en) * | 2021-06-25 | 2024-01-26 | 北京鼎普科技股份有限公司 | Method and device for checking security document data and electronic equipment |
CN116049131A (en) * | 2022-06-10 | 2023-05-02 | 荣耀终端有限公司 | File management method, system, electronic equipment and storage medium |
CN116049131B (en) * | 2022-06-10 | 2023-10-13 | 荣耀终端有限公司 | File management method, system, electronic equipment and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11593319B2 (en) | Virtualized data storage system architecture | |
CN104899156B (en) | A kind of diagram data storage and querying method towards extensive social networks | |
US9836419B2 (en) | Efficient data movement within file system volumes | |
US9152332B2 (en) | Storage system and method for reducing energy consumption | |
CN105095094B (en) | EMS memory management process and equipment | |
CN101840308B (en) | Hierarchical memory system and logical volume management method thereof | |
US9037828B2 (en) | Transferring storage resources between snapshot storage pools and volume storage pools in a data storage system | |
CN104765575B (en) | information storage processing method | |
CN107193758A (en) | The mapping table management method and solid state hard disc of a kind of solid state hard disc | |
JP5944587B2 (en) | Computer system and control method | |
CN105549905A (en) | Method for multiple virtual machines to access distributed object storage system | |
JP2003162377A (en) | Disk array system and method for taking over logical unit among controllers | |
CN106201335B (en) | Storage system | |
KR20070024573A (en) | Method for managing files for optimal performance | |
CN103067433A (en) | Method, device and system of data migration of distributed type storage system | |
JP2010541060A (en) | Method and apparatus for migration and cancellation of thin provisioning | |
JP2017134817A (en) | Systems and methods for adaptive partitioning in distributed cache memories | |
CN110196818A (en) | Data cached method, buffer memory device and storage system | |
CN100530069C (en) | Virtualizing system and method for non-homogeny storage device | |
US20140089562A1 (en) | Efficient i/o processing in storage system | |
US20180307426A1 (en) | Storage apparatus and storage control method | |
CN106407409A (en) | A virtual file system based on DAS architecture storage servers and a file management method thereof | |
CN109582658A (en) | A kind of distributed file system realizes the method and device of data consistency | |
CN111488125A (en) | Cache Tier Cache optimization method based on Ceph cluster | |
WO2024152614A1 (en) | Data request method and apparatus, and device and non-volatile readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170215 |
|
RJ01 | Rejection of invention patent application after publication |