CN106407409A - A virtual file system based on DAS architecture storage servers and a file management method thereof - Google Patents

A virtual file system based on DAS architecture storage servers and a file management method thereof Download PDF

Info

Publication number
CN106407409A
CN106407409A CN201610843245.XA CN201610843245A CN106407409A CN 106407409 A CN106407409 A CN 106407409A CN 201610843245 A CN201610843245 A CN 201610843245A CN 106407409 A CN106407409 A CN 106407409A
Authority
CN
China
Prior art keywords
file
task
subsystem
data
read
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610843245.XA
Other languages
Chinese (zh)
Inventor
李涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TCL Corp
Original Assignee
TCL Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TCL Corp filed Critical TCL Corp
Priority to CN201610843245.XA priority Critical patent/CN106407409A/en
Publication of CN106407409A publication Critical patent/CN106407409A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/11File system administration, e.g. details of archiving or snapshots
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/188Virtual file systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a virtual file system based on DAS architecture storage servers and a file management method thereof. The virtual file system comprises a task management sub-system and a cache sub-system. The task management sub-system is used for receiving file operation tasks from request terminals and sending the tasks to the cache sub-system. If the task type of the file operation tasks is file reading tasks, the cache sub-system acquires to-be-read file data corresponding to the file operation tasks from a cache space, and sends the to-be-read file data to the task management sub-system; or if the task type of the file operation tasks is file writing tasks, the cache sub-system writes to-be-written file data corresponding to the file operation tasks into the cache space. The task management sub-system receives the to-be-read file data and sends the to-be-read file data to the request terminals. The virtual file system is provided with the cache sub-system which can process file operation tasks, so that repeated reading of disks or disk arrays can be prevented and the IO performance of the disks or disk arrays can be guaranteed.

Description

Virtual File System based on DAS framework storage server and its file management method
Technical field
The present invention relates to Virtual File System field, more particularly, to a kind of virtual literary composition based on DAS framework storage server Part system and its file management method.
Background technology
With developing rapidly of the Internet and video technique, people's continuous lifting to video quality demands, lead to video counts It is continuously increased according to transmission quantity, require video storage server to improve constantly video read-write speed therewith, timely respond to use to meet The purpose of family video read-write requests.Current disk storage server mainly includes tri- kinds of frameworks of DAS, NAS and SAN.Wherein, DAS (Direct Access Storage, i.e. Direct Attached Storage) refers to for storage device to pass through scsi interface or optical-fibre channel is straight It is connected in succession on a computer.NAS (Network Attached Storage, i.e. network connection storage) refers to set storage For by the network topology structure (such as Ethernet) of standard, it is connected on a group computer.SAN(Storage Area Network, i.e. storage area network) refer to be connected on a group computer by optical-fibre channel.
In actual application, the medium-sized and small enterprises such as hotel, apartment, local tv station are cost limited, typically will not The storage server of the complex and high cost NAS framework of selected technology or SAN framework carries out video data storage it will usually adopt The storage server of DAS framework and then video data storage, the storage server of DAS framework relies primarily on the master of storage server Machine operating system carries out read-write operation and the storage system maintenance management of data.The storage server of DAS framework there is low cost, knot Structure simply it is not necessary to the advantages of special technical staff safeguards, stores and the IO performance to read-write operation in general data When (Input/Output performance, i.e. input and output performance) is less demanding, primary demand can be met.But needing big concurrent reading and writing During operation, the storage server of DAS framework can only be by configuring RAID (Redundant Arrays of Independent Disks, i.e. independent redundancy disk array) card, improves the overall read-write speed of the disk array being connected with storage server.Existing The storage server of DAS framework, even if being configured with RAID card, if during the same video resource of multiple user's central access, also can shape Become the IO performance bottleneck of whole storage server, lead to the disk storing this video resource nervous, the IO of impact storage server Performance.Even and if, multiple access request is not to access same video resource but the corresponding video resource of multiple access request to exist It is also possible to lead to disk nervous on same disk, the IO performance of impact storage server.
Content of the invention
The technical problem to be solved in the present invention is, for the defect of prior art, provides based on DAS framework storage service The Virtual File System of device and its file management method, in order to improve the read-write speed of storage server.
The technical solution adopted for the present invention to solve the technical problems is:A kind of void based on DAS framework storage server Intend file system, including task management subsystem and cache subsystem;
Described task management subsystem, for receiving the file operation task from requesting terminal, in described file operation When the task type of task is reading and writing of files task, described file operation task is sent to described cache subsystem;
Described cache subsystem, for receiving described file operation task;Task type in described file operation task During for reading file task, from spatial cache, obtain the to be read file data corresponding with described file operation task, and will Described file data to be read is sent to described task management subsystem;Or, in the task type of described file operation task During for written document task, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem, for receiving described file data to be read, and by described file data to be read It is sent to described requesting terminal.
Preferably, described Virtual File System also includes storage subsystem;
Described cache subsystem, for judging to receive the task type of described file operation task as reading file task also It is written document task;
Described cache subsystem, during for being to read file task in the task type receiving described file operation task, Judge to whether there is the to be read file data corresponding with described file operation task in spatial cache;If existing, update File data in described spatial cache, and described file data to be read is sent to described task management subsystem;If no Exist, then described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, obtains corresponding with described file operation task File data to be read, and described file data to be read is sent to described cache subsystem;
Described cache subsystem, for receiving the file data described to be read from described storage subsystem, will be described File data to be read is added in described spatial cache, updates the file data in described spatial cache;And continue described File data is taken to be sent to described task management subsystem;
Described cache subsystem, during for being written document task in the task type receiving described file operation task, Described file operation task includes described file data to be written;Described file data to be written is added to described spatial cache Interior, update the file data of described spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, and stores described file data to be written.
Preferably, described Virtual File System also includes setting up the catalogue subsystem of internal memory mapping relations with host operating system System;
Described directory subsystem, for the number of files being stored in using preset data structure management in described storage subsystem According to corresponding file directory information;
Described storage subsystem, deposits for disk is divided into memory paragraph, memory block and memory page three-level according to space size Storage structure, and described file data is stored at least one described memory page;Described file directory information records each institute State file data at least one storage address corresponding;
Or, described storage subsystem, for will be stored in the file data in described storage subsystem be divided into some Sub-file data, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information Record the corresponding disk address of each described sub-file data.
Preferably, also include:Described task management subsystem, is pipe for the task type in described file operation task During reason file task, described file operation task is sent to described directory subsystem;Described management file task includes creating File task, deletion file task, modification file task or locating file task;
Described directory subsystem, for receiving and executing described file operation task.
Preferably, described directory subsystem, is establishment file task for the task type in described file operation task When, judge whether the file size of described file operation task file to be created has determined that;
Described directory subsystem, for when described file size has determined that, by described task management subsystem to institute State storage subsystem and send application spatial cache request, receive the distribution spatial cache of described storage subsystem;Based on described literary composition Part operation task generates an establishment file directory information, and described establishment file directory information is related to described distribution spatial cache Connection, and described establishment file directory information is added in described preset data structure;
Described directory subsystem, for when described file size does not determine, generating one based on described file operation task Establishment file directory information, and described establishment file directory information is added in described preset data structure.
The present invention also provides a kind of file management method of the Virtual File System based on DAS framework storage server, institute State Virtual File System and include task management subsystem and cache subsystem;
Described task management subsystem receives the file operation task from requesting terminal, in described file operation task When task type is reading and writing of files task, described file operation task is sent to described cache subsystem;
Described cache subsystem receives described file operation task;It is to read literary composition in the task type of described file operation task During part task, obtain the to be read file data corresponding with described file operation task from spatial cache, and treat described Read file data and be sent to described task management subsystem;Or, it is to write literary composition in the task type of described file operation task During part task, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem receives described file data to be read, and described file data to be read is sent to Described requesting terminal.
Preferably, described Virtual File System also includes storage subsystem;
Described cache subsystem judges that receiving the task type of described file operation task still writes as reading file task File task;
Described cache subsystem, when the task type receiving described file operation task is to read file task, judges slow Deposit and in space, whether there is the to be read file data corresponding with described file operation task;If existing, update described slow Deposit the file data in space, and described file data to be read is sent to described task management subsystem;If not existing, Described file operation task is sent to described storage subsystem;
Described storage subsystem receives described file operation task, obtains and described corresponding the continuing of file operation task Take file data, and described file data to be read is sent to described cache subsystem;
Described cache subsystem receives the file data described to be read from described storage subsystem, will be described to be read File data is added in described spatial cache, updates the file data in described spatial cache;And by described file to be read Data is activation gives described task management subsystem;
Described cache subsystem receive described file operation task task type be written document task when, described literary composition Part operation task includes described file data to be written;Described file data to be written is added in described spatial cache, more The newly file data of described spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem receives described file operation task, and stores described file data to be written.
Preferably, described Virtual File System also includes setting up the catalogue subsystem of internal memory mapping relations with host operating system System;The corresponding literary composition of file data that described directory subsystem is stored in described storage subsystem using preset data structure management Part directory information;
Disk is divided into memory paragraph, memory block and memory page tertiary storage knot according to space size by described storage subsystem Structure, and described file data is stored at least one described memory page;Described file directory information records each described literary composition Number of packages is according at least one storage address corresponding;
Or, the file data that described storage subsystem will be stored in described storage subsystem is divided into some subfiles Data, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information record is every The corresponding disk address of sub-file data described in.
Preferably, also include:
Described task management subsystem, will when judging the task type of described file operation task for management file task Described file operation task is sent to described directory subsystem;Described management file task includes establishment file task, deletes literary composition Part task, modification file task or locating file task;
Described directory subsystem receives and executes described file operation task.
Preferably, also include:Described directory subsystem is appointed for establishment file in the task type of described file operation task During business, judge whether the file size of described file operation task file to be created has determined that;
Described directory subsystem when described file size has determined that, by described task management subsystem to described storage Subsystem sends application spatial cache request, receives the distribution spatial cache of described storage subsystem;Based on described file operation Task generates an establishment file directory information, and described establishment file directory information is associated with described distribution spatial cache, and Described establishment file directory information is added in described preset data structure;
Described directory subsystem, when described file size does not determine, generates one based on described file operation task and creates literary composition Part directory information, and described establishment file directory information is added in described preset data structure.
The present invention compared with prior art has the advantage that:Provided by the present invention based on DAS framework storage server Virtual File System and its file management method, task management subsystem receive task type be reading and writing of files task literary composition During part operation task, file operation task is sent to cache subsystem;Cache subsystem is in the task class of file operation task When type is to read file task, obtains the to be read file data corresponding with file operation task from spatial cache and be sent to times Business management subsystem;When the task type of file operation task is written document task, will be corresponding with file operation task File data to be written writes spatial cache.In this Virtual File System setting cache subsystem to file operation task at Reason, maskable to disk or disk array repeat read, to reduce the read-write to disk or disk array for the file operation task Pressure, reduces the IO expenditure of disk or disk array, to ensure the IO performance of disk or disk array.
Brief description
Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:
Fig. 1 is the theory diagram of the Virtual File System in the embodiment of the present invention 1 based on DAS framework storage server.
Fig. 2 is the file management method of the Virtual File System in the embodiment of the present invention 2 based on DAS framework storage server A flow chart.
In figure:11st, task management subsystem;12nd, cache subsystem;13rd, storage subsystem;14th, directory subsystem.
Specific embodiment
In order to be more clearly understood to the technical characteristic of the present invention, purpose and effect, now comparison accompanying drawing describes in detail The specific embodiment of the present invention.
Embodiment 1
Fig. 1 illustrates the theory diagram of the Virtual File System based on DAS framework in the present embodiment.Based on DAS framework In storage server, storage server is joined directly together by scsi interface with disk or disk array, sets in storage server Have for realizing the file data on disk or disk array is carried out with IO read-write and storage system maintenance management, data backup and resume The host operating system of the functions such as requirement;Wherein, disk array includes at least two disks.The present embodiment provided based on In the Virtual File System of DAS framework storage server, Virtual File System and host operating system pre-build that internal memory maps Relation, specifically includes task management subsystem 11, cache subsystem 12, directory subsystem 14 and storage subsystem 13;Wherein, mesh Record subsystem 14 sets up internal memory mapping relations with host operating system.
Directory subsystem 14 is used for each number of files being stored in Virtual File System using preset data structure management According to corresponding file directory information.Specifically, the file directory information in directory subsystem 14 is stored in internal memory mapping mode In host operating system so that the file directory information in directory subsystem 14 can real-time update, and then ensure file directory The safety of information and effective in real time.Because directory subsystem 14 and host operating system set up internal memory mapping relations so that every time When host operating system starts, it is both needed to load directory subsystem 14.In the present embodiment, directory subsystem 14 is used for managing virtual literary composition All of file data in part system, the file directory information (including file size, storage address etc.) of log file data, real Existing establishment file, deletion file, the modification management file task such as file and locating file.Directory subsystem 14 institute in the present embodiment Using preset data structure be red-black data tree structure so that execution establishment file, deleting file, modification file and search literary composition The management such as part file task is positively retained under relatively low time complexity.It is to be appreciated that keeping relatively low time complexity In the case of, preset data structure can also be other data structures.
Task management subsystem 11 is responsible for receiving and manage the file behaviour of the access Virtual File System of requesting terminal transmission Make task, the task type of file operation task includes reading and writing of files task and management file task;Wherein, reading and writing of files task Including reading file task and written document task;Management file task includes establishment file task, deletes file task, modification file Task and locating file task.Each file operation task correspond to a task data block, this task data block be used for storage with The related mission bit stream of task, including but not limited to task type, file name, file size, creation time, tasks carrying shape State, code check, video type, compressed format etc..In addition, task data block does not store specific file data, but and file data Storage address be associated.Specifically, when the task type in file operation task is to read file task, by its corresponding task Data block is associated with the storage address of the file data to be read getting;It is to write literary composition in the task type of file operation task During part task, its corresponding task data block is associated with the storage address of file data to be written.In Virtual File System In, the communication between each subsystem of Virtual File System is realized based on the task data block of file operation task and calls.Task pipe File operation task can be sent to by reason subsystem 11 according to the task type on the task data block of each file operation task Corresponding subsystem is processed, and result is returned to the requesting terminal sending file operation task.In the present embodiment, Task management subsystem 11, when the task type of the file operation task receiving is reading and writing of files task, file operation is appointed Business is sent to cache subsystem 12 and is processed;It is management file task in the task type of the file operation task receiving When, file operation task is sent to directory subsystem 14 and is processed.
Cache subsystem 12 is used for managing the All Files data of caching in spatial cache, receives task management subsystem 11 The task type sending is the file operation task of reading and writing of files task, and is reading and writing of files task to the task type receiving File operation task processed.Specifically, cache subsystem 12 receives task type is the file behaviour reading file task When making task, first check in spatial cache and whether there is the to be read file data corresponding with file operation task, if existing File data to be read, then directly return to task management subsystem 11 by the file data to be read of caching, and passes through task Management subsystem 11 is sent to requesting terminal;If there is not file data to be read, file operation task is sent to storage Subsystem 13, obtains file data to be read from storage subsystem 13 and returns to task management subsystem 11.Cache subsystem 12 receive the file operation task that task type is written document task (now comprises number of files to be written in file operation task According to) when, first the file data to be written that file operation task is comprised can be stored in spatial cache, update spatial cache File data, is then written into file data and sends storage subsystem 13, store disk or disk by storage subsystem 13 On array.
It is to be appreciated that cache subsystem 12 is in advance to host operating system application spatial cache, as virtual file system The caching of system, and it is independently operated, the 2 of the cache subsystem spatial caches that application is arrived in units of page size, it is divided into multiple Caching page.Cache subsystem 12 is responsible for the caching page status of each caching page in spatial cache, to determine that caching page is in sky Not busy state or use state.If being in idle condition, can write direct new file data caching to caching page;If place In use state, i.e. storage address in storage subsystem 13 for the file data of caching, this storage address in record buffer memory page Including storage sector address in disk or disk array for the file data, MBA memory block address and storage page address, by this storage Address can directly position corresponding file data in internal memory.
Further, cache subsystem 12 to the file data to be read being got based on file operation task (from depositing Storage subsystem 13) or file data to be written (from file operation task), it is both needed to cache portion in spatial cache, stored On the caching page of idle condition, form the file data of spatial cache, be to read file in order to receive task type again During the file operation task of task, can directly return obtaining file data to be read accordingly in the file data of spatial cache To task management subsystem 11, file operation task need not be sent to storage subsystem 13 to obtain file to be read accordingly Data, thus filtering out a part of task type is the file operation task reading file task so that only in spatial cache Do not exist in file data accordingly file data to be read when just can obtain file task to be read from storage subsystem 13, with Mitigate the IO pressure of disk.It is to be appreciated that when file data to be read or file data to be written are added spatial cache, Need to first check whether spatial cache is full;If spatial cache is full, does not exist in spatial cache and can cache number of files to be read According to or file data to be written the caching page being in idle condition, now need using default life cycle algorithm, eliminate old slow Deposit the file data of caching on page.This caching life cycle algorithm includes but is not limited to RAND algorithm, FIFO algorithm, LFU algorithm, LRU Algorithm, OPT algorithm.
Storage subsystem 13 is responsible for disk or disk array, and coordinates and complete file operation task.Specifically, store Subsystem 13 is responsible for disk or disk array in storage server, and disk or disk array are divided into according to space size Memory paragraph, memory block and memory page tertiary storage structure (disk or disk array will be divided into some storages according to space size Section, each memory page is divided into some memory blocks, and each memory block is divided into some memory pages), and file data is stored in On at least one memory page.Storage subsystem 13 goes back definition data structure to manage the state of memory paragraph, memory block and memory page Information, and determine that storage organizations at different levels (including memory paragraph, memory block and memory page) are in idle condition or use state.If Then represent that corresponding disk space is sky for idle condition, can write direct file data;If use state then represents corresponding Disk space occupied, not directly write file data.Storage subsystem 13 also sets up Task-list, is derived from for receiving The task type of cache subsystem 12 is the file operation task of reading and writing of files task;And the task type in file operation task During for reading file task, file data to be read being obtained from disk or disk array and returns to cache subsystem 12, now Do not change the file data on disk or disk array;When the task type of file operation task is written document task, will treat Write file data write disk or disk array, the file data on modification disk or disk array, and it is written into file The file directory information of data is added in directory subsystem 14.
In a kind of specific embodiment, disk or disk array are divided into storage according to space size by storage subsystem 13 Section, memory block and memory page tertiary storage structure, and file data is stored at least one memory page;Correspondingly, catalogue File directory information in subsystem 14 records the corresponding storage address of each file data, and this storage address includes memory paragraph ground Location, MBA memory block address and storage page address.Because disk or disk array are divided into memory paragraph, memory block and memory page three-level Storage organization, when file data is stored in disk or disk array, can be stored on W memory page, W memory page may belong to M memory block, M memory block may belong to N number of memory paragraph, and N number of memory paragraph may belong to different disks.Correspondingly, catalogue The storage sector address of N number of memory paragraph, N number of memory paragraph ground is recorded in the corresponding file directory information of this document data in subsystem 14 M MBA memory block address is recorded, W storage page address of record under M MBA memory block address under location.It is to be appreciated that each memory page Space size be generally the integral multiple of physical disk page so that disk or disk array are obtaining file data to be read or writing Enter during file data to be written and just to correspond to physical disk page, unnecessary to avoid producing for the non-integral multiple of physical disk page Read-write operation.If memory page is 1.5 times of physical disk pages, need to be to physics when task type is reading and writing of files task executing Disk page is read and write 2 times.Physical disk page is usually 512K or 1024K, correspondingly, defines the storage subsystem of Virtual File System The disk space size of 13 memory page is 512K or 1024K.
In another specific embodiment, the file data that storage subsystem 13 will be stored in storage subsystem 13 divides Become some sub-file data, and some sub-file data are uniformly distributed and are stored on all disks;File directory information is remembered Record the corresponding disk address of each sub-file data.Disk in the present embodiment is storage disk and non-system disk.In this tool In body embodiment, disk or disk array need not be divided into memory paragraph, memory block and memory page tertiary storage structure, but In the file data distribution disk space to disk to be written or disk array, first file data is cut into N number of subfile Data, more N number of sub-file data is uniformly distributed and is stored on all disks.If file data to be written is XXX.ts, deposit File data is cut into XXX_0.ts by Chu Shixu, the sub-file data such as XXX_1.ts, XXX_2.ts, and by above-mentioned Ziwen number of packages According to being uniformly distributed and be stored on all disks.In this specific embodiment, disk worker thread is not direct read/write disk, And be intended to open corresponding sub-file data, and complete reading and writing of files task.Correspondingly, the file directory in directory subsystem 14 Information record is not the storage address that the corresponding memory paragraph of file data, memory block and memory page are formed, but record literary composition Number of packages evidence is cut into which sub-file data and the corresponding disk address of each sub-file data.
No matter it is to be appreciated that being that disk or disk array are carried out with memory paragraph, memory block and memory page tertiary storage knot Structure divides, and file data is stored at least one memory page;Or file data is divided into some sub-file data, And some sub-file data are uniformly distributed and are stored on all disks, a file data all can be made to be stored in different disks On, substantially reduce the probability that user accesses file data on same disk.Because file data is respectively stored in different disk On, when file data is conducted interviews can using different disks, thus realize big concurrently access disk when, visit capacity is point It is fitted on different disks, this avoid the impact to overall performance for the monolithic disk I/O short slab.
It is to be appreciated that when being started based on DAS framework storage server, the startup of its host operating system, correspondingly, with The Virtual File System that host operating system sets up internal memory mapping relations also starts, i.e. task management subsystem 11 and caching subsystem System 12, storage subsystem 13 and directory subsystem 14 start simultaneously.Wherein, directory subsystem 14 loads host service function system when starting The corresponding file directory information of file data of system, using RBTree data structure managing file directory information, this document catalogue Information is Memory Mapping File, when directory subsystem 14 changes the file directory information of internal memory, the literary composition in host operating system Part directory information also can synchronized update.Task management subsystem 11 completes the initialization of data structure when starting, for receiving File operation task from requesting terminal and according to task type by file operation task be sent respectively to cache subsystem 12 or Directory subsystem 14.Can first apply for empty as caching in a bulk of internal memory when cache subsystem 12 starts to storage subsystem 13 Between, and internal memory is divided into multiple caching pages, set up the caching laminated of all caching pages on data structure managing spatial cache State.It is to be appreciated that when cache subsystem 12 starts, on spatial cache, all of caching page is in idle condition.Storage Obtain, from storage server, disk or the disk array that can be used to storage file data when subsystem 13 starts, getting magnetic When disk or disk array, each disk is numbered, is that each disk creates a disk worker thread, disk worker thread is used In the disk belonging to oneself is operated as big file (will disk or disk array as a big file, magnetic Disk worker thread operation when direct read/write disk, and need not on disk establishment file), and will big file according to disk space big Little be divided into memory paragraph, memory block and memory page tertiary storage structure, set up data structure managing memory paragraph, memory block and storage The status information of page.Storage subsystem 13 also creates Task-list corresponding worker thread when starting, and is responsible for receiving file operation File operation task is simultaneously distributed to each disk worker thread, and the result of execution file operation task is returned by task.
The present embodiment provides a kind of Virtual File System based on DAS framework storage server, for realizing to being stored in File data in storage subsystem 13 conducts interviews (inclusion read-write operation), that is, be used for being appointed for reading and writing of files according to task type The file operation task of business obtains file data to be read or is written into file data and is stored in storage subsystem 13.Specifically Ground, should be included based on the Virtual File System of DAS framework storage server:
Task management subsystem 11, for receiving the file operation task from requesting terminal, in file operation task When task type is reading and writing of files task, file operation task is sent to cache subsystem 12.I.e. task management subsystem 11 When receiving file operation task, appointing of file operation task, need to be determined according to file operation task corresponding task data block Service type.Specifically, the task type of file operation task includes reading and writing of files task and management file task;Wherein, read and write File task includes reading file task and written document task;Management file task include establishment file task, delete file task, Modification file task and locating file task.
Cache subsystem 12, for receiving file operation task;It is to read file to appoint in the task type of file operation task During business, from spatial cache, obtain the to be read file data corresponding with file operation task, and by file data to be read It is sent to task management subsystem 11;Or, when the task type of file operation task is written document task, by file operation The corresponding file data to be written of task writes spatial cache.It is to be appreciated that the task type in file operation task is to write During file task, file operation task comprises file data to be written, is written into file data write spatial cache and caches, So that when cache subsystem 12 receives next task type for reading file task, can be accordingly to be read from spatial cache acquisition File data, and it is sent to task management subsystem 11.
Task management subsystem 11, for receiving file data to be read, and File data is activation to be read is given Requesting terminal, so that requesting terminal obtains to be read file data corresponding with file operation task.
In this specific embodiment, task management subsystem 11 is receiving the file that task type is reading and writing of files task During operation task, this document operation task is sent to cache subsystem 12;Cache subsystem 12 appointing in file operation task When service type is to read file task, obtains the to be read file data corresponding with file operation task from spatial cache and send To task management subsystem 11;When the task type of file operation task is written document task, will be with file operation task phase Corresponding file data to be written writes spatial cache.In this Virtual File System, setting cache subsystem 12 is appointed to file operation Business is processed, maskable to disk or disk array repeat read, to reduce file operation task to disk or disk battle array The read-write pressure of row, reduces the IO expenditure of disk or disk array, to ensure the IO performance of disk or disk array.
Further, the Virtual File System based on DAS framework storage server, is being to read file task to task type File operation task when being processed, maskable to same file data to be read repeat read.
Specifically, cache subsystem 12, during for being to read file task in the task type receiving file operation task, Judge to whether there is the to be read file data corresponding with file operation task in spatial cache.If in spatial cache exist with The corresponding file data to be read of file operation task, then update the file data in spatial cache, and by file to be read Data is activation is to task management subsystem 11.If there is not the to be read number of files corresponding with file operation task in spatial cache According to then file operation task being sent to storage subsystem 13.
Storage subsystem 13, for receiving file operation task, obtains the to be read literary composition corresponding with file operation task Number of packages evidence, and file data to be read is sent to cache subsystem 12.
Cache subsystem 12, for receiving the file data to be read from storage subsystem 13, by number of files to be read According to being added in spatial cache, update the file data in spatial cache;And file data to be read is sent to task management Subsystem 11.
The file operation task that is based on cache subsystem 12 reads file data to be read and includes random read take and order reading Two kinds of situations.Original position and the file size of file data to be read in random read take, need to be indicated, calculate original position In the concrete caching page of spatial cache, begin look for spatial cache with the presence or absence of relative with file operation task from concrete caching page The file data to be read answered, if there is file data to be read accordingly in spatial cache, updates spatial cache caching File data, is eliminated with preventing file data to be read to be buffered the default life cycle algorithm of subsystem 12, and by file to be read Data is activation is to task management subsystem 11;If there is not file data to be read accordingly in spatial cache, file is grasped It is sent to storage subsystem 13 as task, file data to be read is read by storage subsystem 13.When order reads, may specify The original position of file data to be read, it would however also be possible to employ the original position of acquiescence, and according to original position and file size meter Calculate the concrete caching page in spatial cache for the original position, begin look for spatial cache from concrete caching page and whether there is and file behaviour Make the corresponding file data to be read of task, if finding, file data to be read being replicated and returns to task management subsystem 11;If not finding, it is submitted to storage subsystem 13, read operation is completed by storage subsystem 13.
It is to be appreciated that when cache subsystem 12 receives the file operation task that task type is reading file task, first Whether there is the to be read file data corresponding with file operation task from caching spatial lookup, exist in spatial cache and continue When taking file data, file data to be read is sent to task management subsystem 11, to complete this document operation task, need not File operation task is sent to storage subsystem 13, identical file data to be read from shielding is to disk or disk array Repeat read, lead to disk or disk array nervous, the IO performance of impact disk disk array.Due to file data to be read It is stored at least one memory page, or file data to be read is divided into some sub-file data and is stored in different disks On, when storage subsystem 13 is based on file operation task and accesses disk or disk array, file operation task can be assigned to On different disks or disk array, to reach pressure equalization, to reduce the IO pressure of disk or disk array.From caching In system 12 direct access file data to be read or receive from storage subsystem 13 file data to be read when, be both needed to more The file data of new spatial cache, is eliminated with avoiding file data to be read to be buffered the default life cycle algorithm of subsystem 12, no Method realizes shielding the file operation task duplication reading to identical file data to be read in cache subsystem 12.
In a specific embodiment, based on the Virtual File System of DAS framework storage server, realizing to task class Type is the file operation task of written document task when being processed, to shield subsequently this document operation task is comprised to be written Enter file data repeats reading.
Specifically, cache subsystem 12, during for being written document task in the task type receiving file operation task, File operation task includes file data to be written;It is written into file data to be added in spatial cache, update spatial cache File data;And file operation task is sent to storage subsystem 13.It is to be appreciated that each task type is written document The file operation task of task all includes file data to be written, when cache subsystem 12 receives this document operation task, needs Corresponding for file operation task file data to be written being write spatial cache, forming the file data of spatial cache so that delaying Deposit subsystem 12 when subsequently received task type is the file operation task reading file data, can be by newly from spatial cache The file data of write spatial cache is sent to task management subsystem 11 as file data to be read accordingly, right to shield In disk or disk array, identical file data to be read repeats reading, it is to avoid disk or disk array are nervous, thus shadow Ring the IO performance of disk or disk array.
Storage subsystem 13, for receiving file operation task, and stores file data to be written.Specifically, store son Disk or disk array are divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by system 13, and will File data is stored at least one memory page.Or, storage subsystem 13 will be stored in the file in storage subsystem 13 Data is divided into some sub-file data, and some sub-file data are evenly distributed on disk.It is to be appreciated that will be to be written Enter file data to be stored at least one memory page, or be written into file data and be divided into some sub-file data storages On different disks, file data is conducted interviews (include reading and writing of files task) when can be using different disks, thus real Now big when concurrently accessing disk, visit capacity is assigned on different disks so that the IO reduced pressure of each disk, so Avoid the impact to overall performance for the monolithic disk I/O short slab.
Cache subsystem 12 is based on file operation task write file data to be written and includes random writing and be sequentially written in Two kinds of situations.When being sequentially written in, the writing position being written into that file data writes spatial cache need not be specified, cache subsystem System 12, according to the file size of file data to be written, is written into the file data in file data write current cache space Afterbody, and be written into file data and be submitted to storage subsystem 13, file data write is written into by storage subsystem 13 On concrete disk, complete write operation.In the case of random writing, file data to be written need to be specified in the write of spatial cache Position, cache subsystem 12, according to original position, determines concrete caching page, and is written into the file data concrete caching page of write In.Meanwhile, task type is sent to storage subsystem 13 for the file operation task of written document task by cache subsystem 12, by Storage subsystem 13 is written into file data and writes on concrete disk, completes write operation.
The present embodiment provides a kind of Virtual File System based on DAS framework storage server, is used for realizing task type File operation task for file management tasks is processed, and this document management role includes establishment file task, deletes file Task, modification file task or locating file task.Specifically, it is somebody's turn to do the Virtual File System based on DAS framework storage server Including:
Task management subsystem 11, for when the task type of file operation task is for management file task, by file Operation task is sent to directory subsystem 14.Wherein, management file task includes establishment file task, deletes file task, repaiies Change file task or locating file task.
Directory subsystem 14, for receiving and executing file operation task.Using red-black data tree structure in the present embodiment The corresponding file directory information of each file data in management storage subsystem 13, directory subsystem 14 receives task type For manage file task file operation task when, the establishment to file can be realized respectively, delete, repair according to file operation task Change or locating function.
Further, when the task type of file operation task is establishment file task, should be based on DAS framework storage clothes The Virtual File System of business device also includes:
Directory subsystem 14, for when the task type of file operation task is establishment file task, judging that file is grasped Whether the file size making task file to be created has determined that.It is to be appreciated that directory subsystem 14 is receiving task When type is the file operation task of establishment file task, this document is determined according to file operation task corresponding task data block Whether the file size of operation task file to be created has determined that.
Directory subsystem 14, for when file size has determined that, by task management subsystem 11 to storage subsystem 13 send application spatial cache request, receive the distribution spatial cache of storage subsystem 13;Generate one based on file operation task Establishment file directory information, establishment file directory information is associated with distribution spatial cache, and by establishment file directory information It is added in preset data structure.Under the scenes such as VOD order video, the file size of file operation task file to be created Have determined that.Specifically, directory subsystem 14 need to first pass through task management subsystem 11 and send application X size to storage subsystem 13 The spatial cache request in space, to call storage subsystem 13 interface.After storage subsystem 13 receives spatial cache request, meter Calculate number Y of the corresponding memory paragraph in X size space, and Y memory paragraph is divided equally on all disks, empty to form distribution caching Between.Corresponding for the distribution distributing spatial cache segment data structure is returned to directory subsystem 14, hop count by storage subsystem 13 The storage block message being comprised according to record storage section in structure.Directory subsystem 14 is based on file operation task and generates an establishment literary composition Part directory information, and this establishment file directory information is associated with receiving distribution spatial cache corresponding segment data structure. Establishment file directory information is added in red-black data tree structure directory subsystem 14, completes a file size fixed The establishment of file.It is to be appreciated that thinner in order to get, also X size space can be divided for unit by memory block, be calculated The quantity of memory block, and all memory blocks are divided equally on all disks, to form distribution spatial cache;Correspondingly, store son System 13 returns block data structure, the storage page information that in block data structure, record storage block is comprised to directory subsystem 14.
Directory subsystem 14, for when file size does not determine, generating an establishment file mesh based on file operation task Record information, and establishment file directory information is added in preset data structure.In file operation task file to be created In the case of file size is undetermined, only an establishment file directory information need to be generated based on file operation task, then literary composition will be created Part directory information is added in red-black data tree structure, you can complete the establishment of the undetermined file of file size, will Provisional application spatial cache again during the file data to be written write undetermined file of file size, each provisional application can be applied for The spatial cache of one memory paragraph, is written into the spatial cache that file data write application is arrived, and discharges spatial cache (storage Section) not have use memory block.
The Virtual File System based on DAS framework storage server that the present embodiment is provided, by setting up storage subsystem System 13, when making multiple request terminal accesses identical file data, actual access is file data in spatial cache, not right Disk or disk array cause to read and write pressure, reduce the IO expenditure of disk or disk array.In addition, storage subsystem 13 is by disk Or disk array is divided into memory paragraph, memory block and memory page tertiary storage structure according to space size, and file data is stored On at least one memory page;Or, if the file data that storage subsystem 13 will be stored in storage subsystem 13 is divided into Dry sub-file data, and some sub-file data are evenly distributed on all disks.Above two storage mode, all can make One file data is stored on different disks, reduces the probability that user accesses the file data on same disk, concurrent greatly When accessing disk, visit capacity is distributed to all disks, it is to avoid monolithic disk I/O short slab impacts to whole performance.
Embodiment 2
The present embodiment provides a kind of file management method of the Virtual File System based on DAS framework storage server, uses In realizing the file data being stored in storage subsystem 13 is conducted interviews (inclusion read-write operation), that is, it is used for according to task class Type is that the file operation task of reading and writing of files task obtains file data to be read or is written into file data and is stored in storage Subsystem 13.Specifically, the file management method being somebody's turn to do the Virtual File System based on DAS framework storage server includes:
S10:Task management subsystem 11 receives the file operation task from requesting terminal, and determines file operation task Task type.Specifically, the task type of file operation task includes reading and writing of files task and management file task;Wherein, Reading and writing of files task includes reading file task and written document task;Management file task includes establishment file task, deletes file Task, modification file task and locating file task.
S20:Task management subsystem 11, when the task type of file operation task is reading and writing of files task, file is grasped It is sent to cache subsystem 12 as task.I.e. task management subsystem 11, need to be according to file when receiving file operation task Operation task corresponding task data block determines the task type of file operation task.
S30:Cache subsystem 12 receives file operation task;It is to read file task in the task type of file operation task When, obtain the to be read file data corresponding with file operation task from spatial cache, and file data to be read is sent out Give task management subsystem 11;Or, when the task type of file operation task is written document task, file operation is appointed Corresponding file data to be written of being engaged in writes spatial cache.It is to be appreciated that the task type in file operation task is to write literary composition During part task, file operation task comprises file data to be written, is written into file data write spatial cache and caches, with When making cache subsystem 12 receive next task type for reading file task, can be from the literary composition accordingly to be read of spatial cache acquisition Number of packages evidence, and it is sent to task management subsystem 11.
S40:Task management subsystem 11 receives file data to be read, and by File data is activation to be read give please Seek terminal, so that requesting terminal obtains to be read file data corresponding with file operation task.
S50:Task management subsystem 11, when the task type of file operation task is management file task, file is grasped It is sent to directory subsystem 14 as task.Wherein, management file task includes establishment file task, deletes file task, modification File task or locating file task.
S60:Directory subsystem 14 receives and executes file operation task.Using red-black data tree structure pipe in the present embodiment The corresponding file directory information of each file data in reason storage subsystem 13, directory subsystem 14 receives task type and is During the file operation task of management file task, the establishment to file, deletion, modification can be realized respectively according to file operation task Or locating function.
In this specific embodiment, task management subsystem 11 is receiving the file that task type is reading and writing of files task During operation task, this document operation task is sent to cache subsystem 12;Cache subsystem 12 appointing in file operation task When service type is to read file task, obtains the to be read file data corresponding with file operation task from spatial cache and send To task management subsystem 11;When the task type of file operation task is written document task, will be with file operation task phase Corresponding file data to be written writes spatial cache.In this Virtual File System, setting cache subsystem 12 is appointed to file operation Business is processed, maskable to disk or disk array repeat read, to reduce file operation task to disk or disk battle array The read-write pressure of row, reduces the IO expenditure of disk or disk array, to ensure the IO performance of disk or disk array.
Further, the file management method of the Virtual File System based on DAS framework storage server, to task class Type is the file operation task reading file task when being processed, maskable to same file data to be read repeat read, I.e. step S20 specifically includes:
S21:Cache subsystem 12, when the task type receiving file operation task is to read file task, judges caching Whether there is the to be read file data corresponding with file operation task in space.If existing and file operation in spatial cache The corresponding file data to be read of task, then update the file data in spatial cache, and file data to be read sent To task management subsystem 11.If there is not the to be read file data corresponding with file operation task in spatial cache, will File operation task is sent to storage subsystem 13.
S22:Storage subsystem 13 receives file operation task, obtains the to be read file corresponding with file operation task Data, and file data to be read is sent to cache subsystem 12.
S23:Cache subsystem 12 receives the file data to be read from storage subsystem 13, by file data to be read It is added in spatial cache, update the file data in spatial cache;And file data to be read is sent to task management System 11.
The file operation task that is based on cache subsystem 12 reads file data to be read and includes random read take and order reading Two kinds of situations.Original position and the file size of file data to be read in random read take, need to be indicated, calculate original position In the concrete caching page of spatial cache, begin look for spatial cache with the presence or absence of relative with file operation task from concrete caching page The file data to be read answered, if there is file data to be read accordingly in spatial cache, updates spatial cache caching File data, is eliminated with preventing file data to be read to be buffered the default life cycle algorithm of subsystem 12, and by file to be read Data is activation is to task management subsystem 11;If there is not file data to be read accordingly in spatial cache, file is grasped It is sent to storage subsystem 13 as task, file data to be read is read by storage subsystem 13.When order reads, may specify The original position of file data to be read, it would however also be possible to employ the original position of acquiescence, and according to original position and file size meter Calculate the concrete caching page in spatial cache for the original position, begin look for spatial cache from concrete caching page and whether there is and file behaviour Make the corresponding file data to be read of task, if finding, file data to be read being replicated and returns to task management subsystem 11;If not finding, it is submitted to storage subsystem 13, read operation is completed by storage subsystem 13.
It is to be appreciated that when cache subsystem 12 receives the file operation task that task type is reading file task, first Whether there is the to be read file data corresponding with file operation task from caching spatial lookup, exist in spatial cache and continue When taking file data, file data to be read is sent to task management subsystem 11, to complete this document operation task, need not File operation task is sent to storage subsystem 13, identical file data to be read from shielding is to disk or disk array Repeat read, lead to disk or disk array nervous, the IO performance of impact disk disk array.Due to file data to be read It is stored at least one memory page, or file data to be read is divided into some sub-file data and is stored in different disks On, when storage subsystem 13 is based on file operation task and accesses disk or disk array, file operation task can be assigned to On different disks or disk array, to reach pressure equalization, to reduce the IO pressure of disk or disk array.From caching In system 12 direct access file data to be read or receive from storage subsystem 13 file data to be read when, be both needed to more The file data of new spatial cache, is eliminated with avoiding file data to be read to be buffered the default life cycle algorithm of subsystem 12, no Method realizes shielding the file operation task duplication reading to identical file data to be read in cache subsystem 12.
In a specific embodiment, the file management method of the Virtual File System based on DAS framework storage server, When realizing task type is processed for the file operation task of written document task, appointed with shielding follow-up this document is operated Be engaged in comprised file data to be written repeat read, that is, step S20 also includes:
S24:Cache subsystem 12 receive file operation task task type be written document task when, file operation Task includes file data to be written;It is written into file data to be added in spatial cache, update the number of files of spatial cache According to;And file operation task is sent to storage subsystem 13.It is to be appreciated that each task type is the literary composition of written document task Part operation task all includes file data to be written, when cache subsystem 12 receives this document operation task, need to grasp file Make the corresponding file data to be written of task and write spatial cache, form the file data of spatial cache so that cache subsystem 12, when subsequently received task type is the file operation task reading file data, will can newly write caching from spatial cache The file data in space is sent to task management subsystem 11 as file data to be read accordingly, to shield to disk or magnetic In disk array identical file data to be read repeat read, it is to avoid disk or disk array are nervous, thus affect disk or The IO performance of disk array.
S25:Storage subsystem 13 receives file operation task, and stores file data to be written.Specifically, store subsystem Disk or disk array 13 are divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by system, and by literary composition Number of packages evidence is stored at least one memory page.Or, storage subsystem 13 will be stored in the number of files in storage subsystem 13 According to being divided into some sub-file data, and some sub-file data are evenly distributed on disk.It is to be appreciated that being written into File data is stored at least one memory page, or is written into file data and is divided into some sub-file data being stored in On different disks, file data is conducted interviews (include reading and writing of files task) when can be using different disks, thus realizing In big concurrently access disk, visit capacity is assigned on different disks so that the IO reduced pressure of each disk, so keeps away Exempt from the impact to overall performance for the monolithic disk I/O short slab.
Cache subsystem 12 is based on file operation task write file data to be written and includes random writing and be sequentially written in Two kinds of situations.When being sequentially written in, the writing position being written into that file data writes spatial cache need not be specified, cache subsystem System 12, according to the file size of file data to be written, is written into the file data in file data write current cache space Afterbody, and be written into file data and be submitted to storage subsystem 13, file data write is written into by storage subsystem 13 On concrete disk, complete write operation.In the case of random writing, file data to be written need to be specified in the write of spatial cache Position, cache subsystem 12, according to original position, determines concrete caching page, and is written into the file data concrete caching page of write In.Meanwhile, task type is sent to storage subsystem 13 for the file operation task of written document task by cache subsystem 12, by Storage subsystem 13 is written into file data and writes on concrete disk, completes write operation.
The present embodiment provides a kind of file management method of the Virtual File System based on DAS framework storage server, uses Processed in realizing the file operation task that task type is file management tasks, this document management role includes establishment file Task, deletion file task, modification file task or locating file task.
Further, when the task type of file operation task is establishment file task, should be based on DAS framework storage clothes Step S60 of the file management method of Virtual File System of business device specifically includes:
S61:Directory subsystem 14, when the task type of file operation task is establishment file task, judges file operation Whether the file size of task file to be created has determined that.It is to be appreciated that directory subsystem 14 is receiving task class When type is the file operation task of establishment file task, determine that this document is grasped according to file operation task corresponding task data block Whether the file size making task file to be created has determined that.
S62:Directory subsystem 14 when file size has determined that, by task management subsystem 11 to storage subsystem 13 Send application spatial cache request, receive the distribution spatial cache of storage subsystem 13;One wound is generated based on file operation task Build file directory information, establishment file directory information is associated with distribution spatial cache, and establishment file directory information is added It is added in preset data structure.Under the scenes such as VOD order video, the file size of file operation task file to be created is Determine.Specifically, directory subsystem 14 need to first pass through task management subsystem 11 and send application X size sky to storage subsystem 13 Between spatial cache request, to call storage subsystem 13 interface.After storage subsystem 13 receives spatial cache request, calculate Number Y of the corresponding memory paragraph in X size space, and Y memory paragraph is divided equally on all disks, empty to form distribution caching Between.Corresponding for the distribution distributing spatial cache segment data structure is returned to directory subsystem 14, hop count by storage subsystem 13 The storage block message being comprised according to record storage section in structure.Directory subsystem 14 is based on file operation task and generates an establishment literary composition Part directory information, and this establishment file directory information is associated with receiving distribution spatial cache corresponding segment data structure. Establishment file directory information is added in red-black data tree structure directory subsystem 14, completes a file size fixed The establishment of file.It is to be appreciated that thinner in order to get, also X size space can be divided for unit by memory block, be calculated The quantity of memory block, and all memory blocks are divided equally on all disks, to form distribution spatial cache;Correspondingly, store son System 13 returns block data structure, the storage page information that in block data structure, record storage block is comprised to directory subsystem 14.
S63:Directory subsystem 14, when file size does not determine, generates an establishment file catalogue based on file operation task Information, and establishment file directory information is added in preset data structure.Literary composition in file operation task file to be created In the case of part size is undetermined, only an establishment file directory information need to be generated based on file operation task, then by establishment file Directory information is added in red-black data tree structure, you can completes the establishment of the undetermined file of file size, will treat Provisional application spatial cache again during the write file data write undetermined file of file size, each provisional application can apply for one The spatial cache of individual memory paragraph, is written into the spatial cache that file data write application is arrived, and discharges spatial cache (storage Section) not have use memory block.
The file management method of the Virtual File System based on DAS framework storage server that the present embodiment is provided, leads to Cross and set up storage subsystem 13, when making multiple request terminal accesses identical file data, actual access is in spatial cache File data, disk or disk array are not caused to read and write pressure, reduce the IO expenditure of disk or disk array.In addition, depositing Disk or disk array are divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by storage subsystem 13, And file data is stored at least one memory page;Or, storage subsystem 13 will be stored in storage subsystem 13 File data is divided into some sub-file data, and some sub-file data are evenly distributed on all disks.Above two Storage mode, all can make a file data be stored on different disks, reduce user and access the number of files on same disk According to probability, big concurrently access disk when, visit capacity is distributed to all disks, it is to avoid monolithic disk I/O short slab is to whole property Can impact.
The present invention is illustrated by several specific embodiments, it will be appreciated by those skilled in the art that, without departing from In the case of the scope of the invention, the present invention can also be carried out with various conversion and equivalent substitute.In addition, being directed to particular condition or tool Body situation, can make various modifications to the present invention, without deviating from the scope of the present invention.Therefore, disclosed in the present invention is not limited to Specific embodiment, and the whole embodiments falling within the scope of the appended claims should be included.

Claims (10)

1. a kind of Virtual File System based on DAS framework storage server it is characterised in that include task management subsystem and Cache subsystem;
Described task management subsystem, for receiving the file operation task from requesting terminal, in described file operation task Task type be reading and writing of files task when, described file operation task is sent to described cache subsystem;
Described cache subsystem, for receiving described file operation task;It is to read in the task type of described file operation task During file task, from spatial cache, obtain the to be read file data corresponding with described file operation task, and will be described File data to be read is sent to described task management subsystem;Or, it is to write in the task type of described file operation task During file task, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem, for receiving described file data to be read, and described file data to be read is sent To described requesting terminal.
2. the Virtual File System based on DAS framework storage server according to claim 1 is it is characterised in that described Virtual File System also includes storage subsystem;
Described cache subsystem, for judging that receiving the task type of described file operation task still writes as reading file task File task;
Described cache subsystem, during for being to read file task in the task type receiving described file operation task, judges Whether there is the to be read file data corresponding with described file operation task in spatial cache;If existing, update described File data in spatial cache, and described file data to be read is sent to described task management subsystem;If not existing, Then described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, obtains and described corresponding the treating of file operation task Read file data, and described file data to be read is sent to described cache subsystem;
Described cache subsystem, for receiving the file data described to be read from described storage subsystem, continues described Take file data to be added in described spatial cache, update the file data in described spatial cache;And by described literary composition to be read Number of packages evidence is sent to described task management subsystem;
Described cache subsystem, during for being written document task in the task type receiving described file operation task, described File operation task includes described file data to be written;Described file data to be written is added in described spatial cache, Update the file data of described spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem, for receiving described file operation task, and stores described file data to be written.
3. the Virtual File System based on DAS framework storage server according to claim 1 is it is characterised in that described Virtual File System also includes setting up the directory subsystem of internal memory mapping relations with host operating system;
Described directory subsystem, for the file data pair being stored in using preset data structure management in described storage subsystem The file directory information answered;
Described storage subsystem, for being divided into memory paragraph, memory block and memory page tertiary storage knot by disk according to space size Structure, and described file data is stored at least one described memory page;Described file directory information records each described literary composition Number of packages is according at least one storage address corresponding;
Or, described storage subsystem, it is divided into some Ziwens for will be stored in the file data in described storage subsystem Number of packages evidence, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information record The corresponding disk address of each described sub-file data.
4. the Virtual File System based on DAS framework storage server according to claim 3 is it is characterised in that also wrap Include:
Described task management subsystem, for when the task type of described file operation task is for management file task, by institute State file operation task and be sent to described directory subsystem;Described management file task includes establishment file task, deletes file Task, modification file task or locating file task;
Described directory subsystem, for receiving and executing described file operation task.
5. the Virtual File System based on DAS framework storage server according to claim 4 is it is characterised in that also wrap Include:
Described directory subsystem, for when the task type of described file operation task is establishment file task, judging described Whether the file size of file operation task file to be created has determined that;
Described directory subsystem, for when described file size has determined that, being deposited to described by described task management subsystem Storage subsystem sends application spatial cache request, receives the distribution spatial cache of described storage subsystem;Based on described file behaviour Make task and generate an establishment file directory information, described establishment file directory information is associated with described distribution spatial cache, And described establishment file directory information is added in described preset data structure;
Described directory subsystem, for when described file size does not determine, generating an establishment based on described file operation task File directory information, and described establishment file directory information is added in described preset data structure.
6. a kind of file management method of the Virtual File System based on DAS framework storage server is it is characterised in that described void Intend file system and include task management subsystem and cache subsystem;
Described task management subsystem receives the file operation task from requesting terminal, in the task of described file operation task When type is reading and writing of files task, described file operation task is sent to described cache subsystem;
Described cache subsystem receives described file operation task;It is to read file to appoint in the task type of described file operation task During business, from spatial cache, obtain the to be read file data corresponding with described file operation task, and will be described to be read File data is sent to described task management subsystem;Or, appoint for written document in the task type of described file operation task During business, corresponding for described file operation task file data to be written is write described spatial cache;
Described task management subsystem receives described file data to be read, and described file data to be read is sent to described Requesting terminal.
7. the file management method of the Virtual File System based on DAS framework storage server according to claim 6, its It is characterised by, described Virtual File System also includes storage subsystem;
Described cache subsystem judges to receive the task type of described file operation task as reading file task or written document Task;
Described cache subsystem, when the task type receiving described file operation task is to read file task, judges that caching is empty Between in the presence or absence of the to be read file data corresponding with described file operation task;If existing, update described caching empty Interior file data, and described file data to be read is sent to described task management subsystem;If not existing, by institute State file operation task and be sent to described storage subsystem;
Described storage subsystem receives described file operation task, obtains the to be read literary composition corresponding with described file operation task Number of packages evidence, and described file data to be read is sent to described cache subsystem;
Described cache subsystem receives the file data described to be read from described storage subsystem, by described file to be read Data is added in described spatial cache, updates the file data in described spatial cache;And by described file data to be read It is sent to described task management subsystem;
Described cache subsystem when the task type receiving described file operation task is written document task, grasp by described file Include described file data to be written as task;Described file data to be written is added in described spatial cache, updates institute State the file data of spatial cache;And described file operation task is sent to described storage subsystem;
Described storage subsystem receives described file operation task, and stores described file data to be written.
8. the file management method of the Virtual File System based on DAS framework storage server according to claim 6, its It is characterised by, described Virtual File System also includes setting up the directory subsystem of internal memory mapping relations with host operating system;Institute State the file data corresponding file mesh that directory subsystem is stored in described storage subsystem using preset data structure management Record information;
Disk is divided into memory paragraph, memory block and memory page tertiary storage structure according to space size by described storage subsystem, and Described file data is stored at least one described memory page;Described file directory information records each described file data At least one storage address corresponding;
Or, the file data that described storage subsystem will be stored in described storage subsystem is divided into some Ziwen numbers of packages According to, and described some sub-file data are uniformly distributed and are stored on all disks;Described file directory information records each The corresponding disk address of described sub-file data.
9. the file management method of the Virtual File System based on DAS framework storage server according to claim 6, its It is characterised by, also include:
Described task management subsystem, will be described when judging the task type of described file operation task for management file task File operation task is sent to described directory subsystem;Described management file task includes establishment file task, deletion file is appointed Business, modification file task or locating file task;
Described directory subsystem receives and executes described file operation task.
10. the file management method of the Virtual File System based on DAS framework storage server according to claim 9, It is characterized in that, also include:
Described directory subsystem, when the task type of described file operation task is establishment file task, judges described file behaviour Whether the file size making task file to be created has determined that;
Described directory subsystem when described file size has determined that, by described task management subsystem to described storage subsystem System sends application spatial cache request, receives the distribution spatial cache of described storage subsystem;Based on described file operation task Generate an establishment file directory information, described establishment file directory information is associated with described distribution spatial cache, and by institute State establishment file directory information to be added in described preset data structure;
Described directory subsystem, when described file size does not determine, generates an establishment file mesh based on described file operation task Record information, and described establishment file directory information is added in described preset data structure.
CN201610843245.XA 2016-09-22 2016-09-22 A virtual file system based on DAS architecture storage servers and a file management method thereof Pending CN106407409A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610843245.XA CN106407409A (en) 2016-09-22 2016-09-22 A virtual file system based on DAS architecture storage servers and a file management method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610843245.XA CN106407409A (en) 2016-09-22 2016-09-22 A virtual file system based on DAS architecture storage servers and a file management method thereof

Publications (1)

Publication Number Publication Date
CN106407409A true CN106407409A (en) 2017-02-15

Family

ID=57997704

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610843245.XA Pending CN106407409A (en) 2016-09-22 2016-09-22 A virtual file system based on DAS architecture storage servers and a file management method thereof

Country Status (1)

Country Link
CN (1) CN106407409A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973109A (en) * 2017-03-31 2017-07-21 努比亚技术有限公司 Broadcast message pushes control method and device
CN113569277A (en) * 2021-06-25 2021-10-29 北京鼎普科技股份有限公司 Method and device for checking security file data and electronic equipment
CN116049131A (en) * 2022-06-10 2023-05-02 荣耀终端有限公司 File management method, system, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101398744A (en) * 2007-09-26 2009-04-01 新奥特硅谷视频技术有限责任公司 Method and system for implementing network data process by utilizing virtual file system technology
CN101639848A (en) * 2009-06-01 2010-02-03 北京四维图新科技股份有限公司 Spatial data engine and method applying management spatial data thereof
CN102004760A (en) * 2010-11-03 2011-04-06 深圳市同洲电子股份有限公司 Multimedia file storing and applying method, related device and system
US20130339786A1 (en) * 2012-06-19 2013-12-19 Lsi Corporation Smart active-active high availability das systems

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101398744A (en) * 2007-09-26 2009-04-01 新奥特硅谷视频技术有限责任公司 Method and system for implementing network data process by utilizing virtual file system technology
CN101639848A (en) * 2009-06-01 2010-02-03 北京四维图新科技股份有限公司 Spatial data engine and method applying management spatial data thereof
CN102004760A (en) * 2010-11-03 2011-04-06 深圳市同洲电子股份有限公司 Multimedia file storing and applying method, related device and system
US20130339786A1 (en) * 2012-06-19 2013-12-19 Lsi Corporation Smart active-active high availability das systems

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973109A (en) * 2017-03-31 2017-07-21 努比亚技术有限公司 Broadcast message pushes control method and device
CN106973109B (en) * 2017-03-31 2020-06-09 南京秦淮紫云创益企业服务有限公司 Broadcast message push control method and device
CN113569277A (en) * 2021-06-25 2021-10-29 北京鼎普科技股份有限公司 Method and device for checking security file data and electronic equipment
CN113569277B (en) * 2021-06-25 2024-01-26 北京鼎普科技股份有限公司 Method and device for checking security document data and electronic equipment
CN116049131A (en) * 2022-06-10 2023-05-02 荣耀终端有限公司 File management method, system, electronic equipment and storage medium
CN116049131B (en) * 2022-06-10 2023-10-13 荣耀终端有限公司 File management method, system, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US11593319B2 (en) Virtualized data storage system architecture
CN104899156B (en) A kind of diagram data storage and querying method towards extensive social networks
US9836419B2 (en) Efficient data movement within file system volumes
US9152332B2 (en) Storage system and method for reducing energy consumption
CN105095094B (en) EMS memory management process and equipment
CN101840308B (en) Hierarchical memory system and logical volume management method thereof
US9037828B2 (en) Transferring storage resources between snapshot storage pools and volume storage pools in a data storage system
CN104765575B (en) information storage processing method
CN107193758A (en) The mapping table management method and solid state hard disc of a kind of solid state hard disc
JP5944587B2 (en) Computer system and control method
CN105549905A (en) Method for multiple virtual machines to access distributed object storage system
JP2003162377A (en) Disk array system and method for taking over logical unit among controllers
CN106201335B (en) Storage system
KR20070024573A (en) Method for managing files for optimal performance
CN103067433A (en) Method, device and system of data migration of distributed type storage system
JP2010541060A (en) Method and apparatus for migration and cancellation of thin provisioning
JP2017134817A (en) Systems and methods for adaptive partitioning in distributed cache memories
CN110196818A (en) Data cached method, buffer memory device and storage system
CN100530069C (en) Virtualizing system and method for non-homogeny storage device
US20140089562A1 (en) Efficient i/o processing in storage system
US20180307426A1 (en) Storage apparatus and storage control method
CN106407409A (en) A virtual file system based on DAS architecture storage servers and a file management method thereof
CN109582658A (en) A kind of distributed file system realizes the method and device of data consistency
CN111488125A (en) Cache Tier Cache optimization method based on Ceph cluster
WO2024152614A1 (en) Data request method and apparatus, and device and non-volatile readable storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20170215

RJ01 Rejection of invention patent application after publication