CN105138545A - Method and system for asynchronously pre-reading directory entries in distributed file system - Google Patents
Method and system for asynchronously pre-reading directory entries in distributed file system Download PDFInfo
- Publication number
- CN105138545A CN105138545A CN201510401114.1A CN201510401114A CN105138545A CN 105138545 A CN105138545 A CN 105138545A CN 201510401114 A CN201510401114 A CN 201510401114A CN 105138545 A CN105138545 A CN 105138545A
- Authority
- CN
- China
- Prior art keywords
- page
- directory entry
- client
- read
- server end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
Abstract
The invention discloses a method and a system for asynchronously pre-reading directory entries. The method comprises: step 1, obtaining a page index number of a directory entry by a client, looking for a first page of the directory entry in a local page cache according to the page index number, if the first page exists, performing the step S2, or otherwise, synchronously sending a directory reading request to a server, sending the first page to the client by the server, and storing the first page in the local page cache by the client; and step 2, analyzing a closing tag eof and a cookie value of the last directory entry in the head of the first page by the client, judging whether the closing tag eof is 1, if so, reading the directory entry in the first page by the client, or otherwise, asynchronously sending the directory reading request to the server by the client according to the page index number and the cookie value, and pre-reading a next page storing the directory entries.
Description
Technical field
The present invention relates to the interaction technique of distributed type file system client side and server end, particularly the asynchronous method and system pre-read of a kind of directory entry.
Background technology
Along with the high speed development of cloud computing, the large infotech such as data, Internet of Things, global metadata informational capacity presents explosive growth, according to the prediction of IDC, from now to the year two thousand twenty, data volume can every two years be doubled, and during to the year two thousand twenty, the data volume of generation will reach 40ZB, therefore, day by day urgent to the demand of mass data storage.
For the distributed file system of satisfied storage mass data, the structure of metadata and data separating has become a kind of main flow framework, metadata and data are kept in proprietary metadata storage devices and data storage device respectively, server end is in charge of all metadata informations, adopt out-of-band access mode, client, according to the metadata information first obtained, directly can access proprietary data storage device to read information.
Directory entry read operation, object is the metadata information such as title, inode number, type of directory entries all under obtaining this catalogue, to be supplied to the directory entry list under this catalogue of user.
Current parallel network file system (pNFS) is when directory entry reads, in local page buffer memory, first search the one page of an information that saves contents, if do not had in local page buffer memory, then synchronous transmission to server end is read catalog request and is obtained this page, waiting for server end returns to one page that client is asked, client could resolve the directory entry in this page one by one, but for there being the large catalogue of a large amount of directory entry, when client reads catalogue, catalog request is read in each transmission can only obtain one page, one page can only comprise several directory entries, need repeatedly to send when causing reading large catalogue to read catalog request, each also will wait returns asked page according to server end, client could start the directory entry of resolving in page, add the execution time that directory entry reads, reduce the performance that directory entry reads.
The patent of invention access method of catalogue " in a kind of distributed file system ", in the invention, the content of catalogue uses file to store, and for the directory entry in catalogue, by carrying out Hash to directory entry title, directory entry is divided into some subsets.For different subsets, its mode according to striping in catalogue file stores, and the size of band is relatively large, pre-reads function with what enable to make full use of during reading disk underlying file systems.For all directory entries in the band block of each subset, adopt the mode of binary tree to store, need to set up binary tree during to avoid reading for the first time.For all band blocks, adopt the mode of memory-mapped (mmap) to conduct interviews, when avoiding accessing data in magnetic disk, carry out Memory Allocation at every turn, expense that the system call of calling system file read-write brings.But the present invention relates to the interaction technique of a kind of distributed type file system client side and server end, the asynchronous directory entry that pre-reads of client is to local page buffer memory, difference is that this patent is to provide the method for a kind of directory entry access, does not relate to the interaction technique of client and server.
Patent of invention " a kind of method for organizing of client directory cache in distributed file system ", in the invention, distributed file system adopts multivariate data server framework, and namely the distribution of content of single catalogue is on multiple meta data server.Why selecting the framework of multivariate data, mainly in order to disperse the pressure of metadata access, improving concurrency.Write for network application and read many characteristics less, this invention retains the content of directory entry and the index node of correspondence in the buffer memory of client, needs repeatedly to communicate with server when repeatedly reading to avoid client; Simultaneously, when a maiden visit catalogue, parallel pre-reading is carried out to the directory entry of this catalogue be distributed on different meta data server, simultaneously, according to acquiescence pre-read that strategy or application program issue pre-read strategy, file inode and file content are pre-read.Like this, when application program needs to access certain file of directory entry, the metadata of this file and data may pre-read in client local cache, thus can the execution speed of accelerating application greatly.But the present invention is applicable to single MDS environment, client is being read in directory process first, asynchronous lower one page pre-reading the item that saves contents, and when reading the directory entry in lower one page, client directly can read at local cache, does not need synchronously to obtain directory entry again.Difference is that this patent is applicable to many MDS environment, directory entry under this catalogue is kept on multiple MDS, client obtains directory entry to multiple MDS is parallel, be kept at local cache, and if this patent be applied in single MDS environment, do not pre-read the lower one page of the item that saves contents, when client reads catalogue first, when reading the directory entry not in local cache, still need to obtain, so two patents are different to MDS end is synchronous.
Summary of the invention
For the deficiencies in the prior art, the present invention proposes the asynchronous method and system pre-read of directory entry in a kind of distributed file system.
The present invention proposes the asynchronous method pre-read of directory entry in a kind of distributed file system, comprising:
Step 1, client obtains the page call number of described directory entry, the first page of directory entry described in local page buffer memory is searched according to described page call number, if exist, then perform step 2, otherwise read catalog request to server end synchronized transmission, described first page is sent to described client by described server end, described first page is kept in local page buffer memory by described client;
Described step 2, described client resolve preserve in described first page stem last described in ending mark eof and the cookie value of directory entry, judge whether described ending mark eof is 1, if, then described client reads the directory entry in described first page, otherwise described client, according to described page call number and described cookie value, reads catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
The asynchronous method pre-read of directory entry in described distributed file system, also comprises the directory entry of described client parsing at current page, and is shown to user.
The asynchronous method pre-read of directory entry in described distributed file system, also comprise when described client is resolved to ending mark eof and the cookie value of last directory entry of the stem preservation of described current page, if the ending mark eof of last directory entry described is 1, then do not need to pre-read the lower one page preserving described directory entry, otherwise, described page call number adds 1, described client is according to described page call number one page under described local page cache lookup, if there is lower one page, then described client reads catalog request to server end asynchronous transmission, otherwise described client reads catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
The asynchronous method pre-read of directory entry in described distributed file system, also comprise described client and find lower one page at described local page buffer memory, check the renewal zone bit of described lower one page, if the set of described renewal zone bit, then described client receives the lower one page preserving described directory entry, if the non-set of described renewal zone bit, then described client continues to wait for, until described server end returns described lower one page, and upgrade one page of described local page buffer memory.
The asynchronous method pre-read of directory entry in described distributed file system, also comprise the described directory entry reading described one page, until the ending mark eof of last directory entry of current page stem preservation is 1, otherwise continue to read catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
The present invention also proposes the asynchronous system pre-read of directory entry in a kind of distributed file system, comprising:
Client modules, the page call number of described directory entry is obtained for client, the first page of directory entry described in local page buffer memory is searched according to described page call number, if exist, then read the directory entry in described page, otherwise read catalog request to server end synchronized transmission, described first page is sent to described client by described server end, described first page is kept in local page buffer memory by described client;
Described client resolve preserve in described first page stem last described in ending mark eof and the cookie value of directory entry, judge whether described ending mark eof is 1, if, then described client reads the directory entry in described page, otherwise described client is according to described page call number and described cookie value, read catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
The asynchronous system pre-read of directory entry in described distributed file system, also comprises the directory entry of described client parsing at current page, and is shown to user.
The asynchronous system pre-read of directory entry in described distributed file system, also comprise when described client is resolved to ending mark eof and the cookie value of last directory entry of the stem preservation of described current page, if the ending mark eof of last directory entry described is 1, then do not need to pre-read the lower one page preserving described directory entry, otherwise, described page call number adds 1, described client is according to described page call number one page under described local page cache lookup, if there is lower one page, then described client reads catalog request to server end asynchronous transmission, otherwise described client reads catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
The asynchronous system pre-read of directory entry in described distributed file system, also comprise described client and find lower one page at described local page buffer memory, check the renewal zone bit of described lower one page, if the set of described renewal zone bit, then described client receives the lower one page preserving described directory entry, if the non-set of described renewal zone bit, then described client continues to wait for, until described server end returns described lower one page, and upgrade one page of described local page buffer memory.
The asynchronous system pre-read of directory entry in described distributed file system, also comprise the described directory entry reading described one page, until the ending mark eof of last directory entry of current page stem preservation is 1, otherwise continue to read catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
From inventing above, the invention has the advantages that:
The asynchronous method and system pre-read of a kind of directory entry that the present invention proposes, space is adopted to exchange the mode of time for, take a small amount of local page buffer memory to preserve the asynchronous page pre-read, client is when reading catalogue first, first page except the item that saves contents is synchronous acquisition, each page after second page is all asynchronously one by one pre-read and be kept at local page buffer memory, directory entry can directly find at local page buffer memory when reading, decrease the expense that directory entry synchronously obtains, decrease the execution time that directory entry reads, improve the performance that directory entry reads.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of the asynchronous pre-read apparatus of directory entry of the present invention;
Fig. 2 is that the directory entry of the page in local page buffer memory of the present invention organizes schematic diagram;
Fig. 3 is the client schematic flow sheet of the asynchronous pre-head method of directory entry of the present invention;
Fig. 4 is directory entry of the present invention asynchronous pre-head method server end schematic flow sheet.
Wherein Reference numeral is:
Step 1/2/3/4/5/6/7/8/9/10;
Step 31/32/33/34/35/36;
Step 41/42/43;
Step 51/52/53;
Client modules 111 server module 222
Client synchronization sends and reads catalog request submodule 11
Client reception is read catalog request and is returned submodule 12
Customer terminal webpage cache sub-module 13
The client-cache page searches submodule 14
Client-cache page stem analyzing sub-module 15
Client directory item is asynchronous pre-reads triggers module 16
Catalog request submodule 17 is read in client asynchronous transmission
Client directory item analyzing sub-module 18
Client directory item display sub-module 19
Received server-side reads catalog request submodule 21
Server end directory entry searches submodule 22
Server end returns reads catalog request submodule 23
Embodiment
In order to solve the problem, the object of the invention is to provide the asynchronous method and system pre-read of directory entry in a kind of distributed file system, can reduce the execution time that directory entry reads, and improves the performance that directory entry reads.
In order to achieve the above object, the present invention proposes the asynchronous system pre-read of directory entry in a kind of distributed file system, comprising:
Client modules: the page searching the item that saves contents in local page buffer memory; Catalog request is read to server module synchronized transmission; Catalog request is read to server module asynchronous transmission; The one page returned by server module is stored in local page buffer memory; Resolve the information that in page, stem is preserved; Resolve the directory entry information in page; By the directory entry information displaying under catalogue to user.
Server module: what receive client modules synchronized transmission reads catalog request; What receive client modules asynchronous transmission reads catalog request; Search the directory entry that client modules reads catalog request, the directory entry of client-requested is kept in the page, fill necessary information in page stem; Return to one page that client reads catalog request.
This client modules specifically comprises: client synchronization sends and reads catalog request submodule, for reading catalog request to server module synchronized transmission, and the directory entry under catalogue described in acquisition request; Client reception is read catalog request and is returned submodule, returns the one page of the item that saves contents reading catalog request for reception server module; Customer terminal webpage cache sub-module, receives for cache client and reads one page that catalog request returns submodule acquisition, be kept at client local page buffer memory; The client-cache page searches submodule, for the one page of the item that saves contents in the cache lookup of client local page; Client-cache page stem analyzing sub-module, for resolve client local page buffer memory one page in the stem information of preserving; Client directory item is asynchronous pre-reads triggers module, according to the header message that client-cache page stem analyzing sub-module parses, judge that client reads catalog request the need of to server module asynchronous transmission, if needed, then catalog request submodule is read in trigger clients asynchronous transmission; Catalog request submodule is read in client asynchronous transmission, and for reading catalog request to server module asynchronous transmission, request pre-reads the lower one page of the item that saves contents; Client directory item analyzing sub-module, for resolving directory entry information in the page in client local page buffer memory; Client directory item display sub-module, for all directory entries under user one by one Display directory.
This server module specifically comprises: received server-side reads catalog request submodule, for receive client send read catalog request; Server end directory entry searches submodule, for searching the directory entry that client reads catalog request at server end, is kept in one page by directory entry information, preserves necessary information in the stem of this page simultaneously; Server end returns reads catalog request submodule, reads one page of catalog request for returning to client.
Client synchronization sends and reads catalog request submodule: read catalog request for client synchronized transmission, waiting for server end returns the one page reading catalog request, after client receives one page that server end returns, could continue to perform process below.
Customer terminal webpage cache sub-module: read catalog request for client reception and return submodule, reception server end returns the one page reading catalog request, customer terminal webpage cache sub-module preserves the page returned at client local page buffer memory, every page with page call number index, storing directory item in order in every page, and ending mark eof and the cookie value of preserving last directory entry in this page the stem of every page.Each directory entry has the mark eof that ends up, and for judging that whether this directory entry is last directory entry under catalogue, if ending mark eof is 1, then this directory entry is last directory entry under catalogue; If ending mark eof is 0, then this directory entry is not last directory entry under catalogue.Each directory entry has a cookie value, is used for representing the position of current directory item.
The client-cache page searches submodule: for client according to page call number in local page buffer memory, search the one page of the item that saves contents.
Client-cache page stem analyzing sub-module: find the one page that will resolve for client in local page buffer memory, client-cache page stem analyzing sub-module resolves ending mark eof and the cookie value of last directory entry preserved in the stem of this page.
Client directory item is asynchronous pre-reads triggers module; Client-cache page stem analyzing sub-module parses the header message of one page, client is according to the ending mark eof of last directory entry in this page, judge whether to need trigger clients asynchronous transmission to read catalog request submodule, pre-read the lower one page of the item that saves contents, if its ending mark eof is 1, then trigger clients asynchronous transmission is not needed to read catalog request submodule; If its ending mark eof is 0, then trigger clients asynchronous transmission is needed to read catalog request submodule.
Catalog request submodule (key modules of the asynchronous pre-read apparatus of directory entry) is read in client asynchronous transmission: first current page call number is added 1 for client, then read catalog request according to the cookie value of last directory entry in current page and current page call number to server end asynchronous transmission, pre-read the lower one page of the item that saves contents; Be different from client synchronization transmission and read catalog request submodule, catalog request submodule is read in client asynchronous transmission does not need synchronous waiting for server end to return the one page reading catalog request, just can continue the directory entry resolving performed below.
Client directory item analyzing sub-module: for client directory item analyzing sub-module, resolve the information such as cookie value, inode number, title, file handle of directory entry, parse first directory entry information in one page, first directory entry is shown to user by client directory item display sub-module, resolve remaining directory entry in this page more one by one, client directory item display sub-module more one by one directory entry be shown to user.
Server end directory entry searches submodule: search for server end the directory entry that client reads catalog request, the directory entry information of client-requested is kept in one page, preserves ending mark eof and the cookie value of last directory entry in this page simultaneously in the stem of this page.
Present invention also offers a kind of asynchronous method pre-read of directory entry of distributed file system, the method comprises:
Step 1, client modules performs first and reads directory operation;
Step 2, client searches the first page of the item that saves contents in local page buffer memory according to page call number, if there is no then enter step 3, if existed, enters step 4;
Step 3, client modules reads catalog request to server end synchronized transmission, and waiting for server end returns to one page that client reads catalog request, and server end is returned the one page reading catalog request by client modules, is kept in client local page buffer memory;
Step 4, client finds the one page that will resolve at local page buffer memory, first resolve ending mark eof and the cookie value of last directory entry in this page preserved in this beginning of the page portion, judge whether the ending mark eof of last directory entry is 1, if ending mark eof is not 1, then client is according to the cookie value of page call number with last directory entry, reads catalog request, pre-read the lower one page of the item that saves contents to server end asynchronous transmission; If ending mark eof is 1, then client does not need to read catalog request to server end asynchronous transmission;
Step 5, client modules starts to resolve first directory entry information in current page, by first directory entry information displaying in this page to user, then resolves remaining directory entry in this page one by one, and is shown to user one by one;
Step 6, client is resolved to last directory entry of current page always, if its ending mark eof is 1, then under this catalogue, all directory entries run through, and read directory operation and terminate; If be not 1, then under this catalogue, directory entry does not run through, then enter step 7;
Step 7, page call number adds 1, and client is according to page call number one page under the cache lookup of client local page, if there is lower one page, then client reads catalog request to server end asynchronous transmission; If no, namely client does not read catalog request to server end asynchronous transmission, then client synchronized transmission reads catalog request;
Step 8, client finds lower one page at local page buffer memory, check the renewal zone bit of this page, if the renewal zone bit set of this page, namely in step 4, catalog request is read in client asynchronous transmission, the lower one page pre-reading the item that saves contents returns, and client receives the lower one page of the item that saves contents; If the non-set of renewal zone bit of this page, namely catalog request is read in the asynchronous transmission of step 4 client, and server end does not also return one page that client reads catalog request, then client continues to wait for, until server end returns the one page reading catalog request, upgrade the page in buffer memory;
Step 9, client finds the page upgraded at local page buffer memory, repeat step 4 to step 7, until the ending mark eof reading a directory entry is 1, then client all directory entries under running through this catalogue.
This step 3 comprises the steps: further
Step 31, client synchronization sends and reads catalog request submodule, reads catalog request to server module synchronized transmission, obtains the one page of the item that saves contents;
Step 32, received server-side client reads catalog request submodule, and what receive client transmission reads catalog request;
Step 33, server end directory entry searches submodule, search at server end the directory entry that client reads catalog request, directory entry information is kept in one page, preserve ending mark eof and the cookie value of last directory entry in this page simultaneously in the stem of this page;
Step 34, server end returns reads catalog request submodule, and server end returns to one page that client reads catalog request;
Step 35, client reception is read catalog request and is returned submodule, and client reception server end returns the one page reading catalog request;
Step 36, customer terminal webpage cache sub-module, returns by server end to the one page reading catalog request and is kept at local page buffer memory.
This step 4 comprises the steps: further
Step 41, client-cache page stem analyzing sub-module, resolves ending mark eof and the cookie value of last directory entry in this page preserved in the stem of current page;
Step 42, client directory item is asynchronous pre-reads triggers module, according to the ending mark eof of last directory entry in current page, judges whether ending mark eof is 1, if not 1, then preserves the cookie value of last directory entry, enters step 43; If 1, then client does not need to read catalog request to server end asynchronous transmission;
Step 43, catalog request submodule is read in client asynchronous transmission, and page call number adds 1, and according to the cookie value of page call number with last directory entry, catalog request is read in client asynchronous transmission.
This step 5 comprises the steps: further
Step 51, client directory item analyzing sub-module, parses first directory entry information in this page;
Step 52, client directory item display sub-module, by first directory entry information displaying in this page to user;
Step 53, client directory item analyzing sub-module, then resolve remaining directory entry in this page one by one, client directory item display sub-module, is shown to user by directory entry one by one.
Be different from parallel network file system pNFS former directory entry reading process, client sends and reads catalog request, server end returns to the page of client storing directory item in order, in the asynchronous pre-head method of directory entry of the present invention, server end not only returns to the page of client storing directory item in order, and can preserve ending mark eof and the cookie value of last directory entry in this page in the stem of the page.The ending mark eof of last directory entry in this page that page stem is preserved, is that client is used for judging whether needing triggering that directory entry is asynchronous pre-reads submodule, reads catalog request, pre-read the lower one page of the item that saves contents to server end asynchronous transmission.In this page that page stem is preserved, the cookie value of last directory entry, is that client is used for reading catalog request to server end asynchronous transmission.
In step 3, client synchronization sends to server end and reads catalog request, obtain the first page of the item that saves contents, the information that client is preserved according to stem in first page, can judge whether second page, if there is second page, catalog request is read in the asynchronous transmission of triggered step 4 client, the asynchronous second page pre-reading the item that saves contents, do not need waiting for server end to return the second page of request, client starts to resolve directory entry information in first page, and is shown to user one by one.Client is resolved and is shown process and the asynchronous process concurrence performance pre-reading second page of client of directory entry in first page, client runs through the directory entry in first page, the second page pre-read in step 4 has returned and has been kept at client local page buffer memory, then directly in local page buffer memory, second page is found, do not need synchronously to send request to server end to obtain second page and waiting for server end returns the second page reading catalog request, decrease the synchronous time waited for, namely decrease directory entry and read the time, improve directory entry reading performance.If this catalogue has the 3rd page, then after finding second page in local page buffer memory, the asynchronous transmission of triggered step 4 client the process that catalog request pre-reads the 3rd page can be read, the like.
Be below one embodiment of the invention, as follows:
To be that directory entry of the present invention is asynchronous pre-read system schematic to Fig. 1, and as shown in Figure 1, this system comprises:
Client modules 1, searches the page of the item that saves contents in local page buffer memory; Catalog request is read to server module 2 synchronized transmission; One page that server module 2 returns is stored in local page buffer memory; Resolve the information that in page, stem is preserved; Catalog request is read to server module 2 asynchronous transmission; Resolve the directory entry information in page; By the directory entry information displaying under catalogue to user;
Wherein, client modules also comprises as lower module:
Client synchronization sends and reads catalog request submodule 11, and for reading catalog request to server module synchronized transmission, acquisition request saves contents one page of item;
Client reception is read catalog request and is returned submodule 12, returns the one page reading catalog request for reception server module;
Customer terminal webpage cache sub-module 13, receives for cache client and reads one page that catalog request returns submodule acquisition, be kept at local page buffer memory;
The client-cache page searches submodule 14, for searching the one page of the item that saves contents in local page buffer memory;
Client-cache page stem analyzing sub-module 15, for resolve client local page buffer memory one page in the stem information of preserving;
Client directory item is asynchronous pre-reads triggers module 16, according to the header message that client-cache page stem analyzing sub-module parses, judge that client reads catalog request the need of to server module asynchronous transmission, if needed, then catalog request submodule is read in trigger clients asynchronous transmission;
Catalog request submodule 17 is read in client asynchronous transmission, and for reading catalog request to server module asynchronous transmission, request pre-reads the lower one page of the item that saves contents;
Client directory item analyzing sub-module 18, for resolving the directory entry information in the page of client local page buffer memory preservation;
Client directory item display sub-module 19, for all directory entries under user one by one Display directory;
Server module 2, what receive client modules 1 synchronized transmission reads catalog request, and what receive client modules 1 asynchronous transmission reads catalog request; Search the directory entry that client modules 1 reads catalog request, the directory entry of client-requested is kept in the page, fill necessary information in page stem; Return to one page that client 1 reads catalog request.
Wherein, server module 2 also comprises as lower module:
Received server-side reads catalog request submodule 21, for receive that client synchronization sends read catalog request and asynchronous transmission read catalog request;
Server end directory entry searches submodule 22, for searching the directory entry that client reads catalog request at server end, directory entry information is kept in one page, and preserves ending mark eof and the cookie value of last directory entry in this page in one page stem;
Server end returns reads catalog request submodule 23, for returning the one page reading catalog request to client;
The present invention also provides a kind of directory entry of distributed file system asynchronous pre-head method, and Fig. 2 is that the present invention reads catalogue asynchronous method client schematic flow sheet, and composition graphs 2, the method comprises the steps:
Step 1, client modules performs first and reads directory operation;
Step 2, client modules is searched in local page buffer memory according to page call number, whether there is the page of the item that saves contents, if there is no then enters step 3, if existed, enters step 4;
Step 3, client modules synchronously sends to server end and reads catalog request, and waiting for server end returns to one page that client reads catalog request, and server end is returned the one page reading catalog request by client modules, is kept in local page buffer memory;
Step 4, client finds the one page that will resolve at local page buffer memory, first resolve ending mark eof and the cookie value of last directory entry in this page preserved in this beginning of the page portion, judge whether the ending mark eof of last directory entry is 1, if ending mark eof is not 1, then client is according to the cookie value of page call number with last directory entry, reads catalog request, pre-read the lower one page of the item that saves contents to server end asynchronous transmission; If ending mark eof is 1, then client does not need to read catalog request to server end asynchronous transmission;
Step 5, client modules parses first directory entry information in this page, by first directory entry information displaying in this page to user, then resolves remaining directory entry in this page one by one, and is shown to user one by one;
Step 6, client reads last directory entry under this page always, if its ending mark eof is 1, then under this catalogue, all directory entries run through, and enter step 10; If be not 1, then under this catalogue, directory entry does not run through, and enters step 7;
Step 7, page call number adds 1, client is according to page call number one page under local page cache lookup, find lower one page, check the renewal zone bit of this page, if the renewal zone bit set of this page, namely in step 4, catalog request is read in client asynchronous transmission, the lower one page pre-reading the item that saves contents returns, and client receives the lower one page of the item that saves contents, and enters step 9; If the non-set of renewal zone bit of this page, namely catalog request is read in the asynchronous transmission of step 4 client, and server end does not also return one page of request, enters step 8;
Step 8, client continues to wait for, until client receives server end return the one page reading catalog request, the renewal zone bit of this page of set, enters step 9;
Step 9, client finds the page upgraded at local page buffer memory, repeat step 4 to step 7, until the ending mark eof of a directory entry is 1, then client all directory entries under running through this catalogue;
Step 10, client is read catalogue and is terminated;
This step 3 comprises the steps: further
Step 31, client synchronization sends and reads catalog request submodule 12, synchronously sends to server module and reads catalog request, obtain the one page of the item that saves contents;
Step 32, received server-side client reads catalog request submodule 21, and what receive client transmission reads catalog request;
Step 33, server end directory entry searches submodule 22, searches the directory entry of client-requested at server end, and is kept in one page;
Step 34, server end returns reads catalog request submodule 23, and server end returns to one page that client reads catalog request;
Step 35, client reception is read catalog request and is returned submodule 12, and client reception server end returns the one page reading catalog request;
Step 36, customer terminal webpage cache sub-module 13, returns by server end to the one page reading catalog request and is kept at local cache;
This step 4 comprises the steps: further
Step 41, client directory item analyzing sub-module 14, resolves directory entry one by one in current page, only resolves cookie and the ending mark eof of directory entry, is resolved to last directory entry of this page always;
Step 42, client judges whether the ending mark eof of last directory entry is 1, if not, then preserve the cookie value of last directory entry, enter step 43; If so, then client does not need to read catalog request to server end asynchronous transmission;
Step 43, catalog request submodule 15 is read in client asynchronous transmission, and page call number adds 1, and according to the cookie value of page call number with last directory entry, catalog request is read in client asynchronous transmission;
This step 5 comprises the steps: further
Step 51, client directory item analyzing sub-module 15, parses first directory entry information in this page;
Step 52, client directory item display sub-module 17, by first directory entry information displaying in this page to user;
Step 53, client directory item analyzing sub-module 15, then resolve remaining directory entry in this page one by one, client directory item display sub-module 17, is shown to user by directory entry one by one;
The present invention also provides a kind of directory entry of distributed file system asynchronous pre-head method, and Fig. 3 is that the present invention reads catalogue asynchronous method server schematic flow sheet, composition graphs 3, and the method comprises:
Step 1, received server-side client synchronization or asynchronous transmission read catalog request,
Step 2, server end searches the directory entry that client reads catalog request, is kept in one page by directory entry information, preserves ending mark eof and the cookie value of last directory entry in this page in the stem of this page simultaneously;
Step 3, server end returns to one page that client reads catalog request.
Claims (10)
1. the asynchronous method pre-read of directory entry in distributed file system, is characterized in that, comprising:
Step 1, client obtains the page call number of described directory entry, the first page of directory entry described in local page buffer memory is searched according to described page call number, if exist, then perform step 2, otherwise read catalog request to server end synchronized transmission, described first page is sent to described client by described server end, described first page is kept in local page buffer memory by described client;
Described step 2, described client resolve preserve in described first page stem last described in ending mark eof and the cookie value of directory entry, judge whether described ending mark eof is 1, if, then described client reads the directory entry in described first page, otherwise described client, according to described page call number and described cookie value, reads catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
2. the asynchronous method pre-read of directory entry in distributed file system as claimed in claim 1, is characterized in that, also comprises the directory entry of described client parsing at current page, and is shown to user.
3. the asynchronous method pre-read of directory entry in distributed file system as claimed in claim 2, it is characterized in that, also comprise when described client is resolved to ending mark eof and the cookie value of last directory entry of the stem preservation of described current page, if the ending mark eof of last directory entry described is 1, then do not need to pre-read the lower one page preserving described directory entry, otherwise, described page call number adds 1, described client is according to described page call number one page under described local page cache lookup, if there is lower one page, then described client reads catalog request to server end asynchronous transmission, otherwise described client reads catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
4. the asynchronous method pre-read of directory entry in distributed file system as claimed in claim 3, it is characterized in that, also comprise described client and find lower one page at described local page buffer memory, check the renewal zone bit of described lower one page, if the set of described renewal zone bit, then described client receives the lower one page preserving described directory entry, if the non-set of described renewal zone bit, then described client continues to wait for, until described server end returns described lower one page, and upgrade one page of described local page buffer memory.
5. the asynchronous method pre-read of directory entry in distributed file system as claimed in claim 4, it is characterized in that, also comprise the described directory entry reading described one page, until the ending mark eof of last directory entry of current page stem preservation is 1, otherwise continue to read catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
6. the asynchronous system pre-read of directory entry in distributed file system, is characterized in that, comprising:
Client modules, the page call number of described directory entry is obtained for client, the first page of directory entry described in local page buffer memory is searched according to described page call number, if exist, then read the directory entry in described page, otherwise read catalog request to server end synchronized transmission, described first page is sent to described client by described server end, described first page is kept in local page buffer memory by described client;
Described client resolve preserve in described first page stem last described in ending mark eof and the cookie value of directory entry, judge whether described ending mark eof is 1, if, then described client reads the directory entry in described page, otherwise described client is according to described page call number and described cookie value, read catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
7. the asynchronous system pre-read of directory entry in distributed file system as claimed in claim 6, is characterized in that, also comprises the directory entry of described client parsing at current page, and is shown to user.
8. the asynchronous system pre-read of directory entry in distributed file system as claimed in claim 7, it is characterized in that, also comprise when described client is resolved to ending mark eof and the cookie value of last directory entry of the stem preservation of described current page, if the ending mark eof of last directory entry described is 1, then do not need to pre-read the lower one page preserving described directory entry, otherwise, described page call number adds 1, described client is according to described page call number one page under described local page cache lookup, if there is lower one page, then described client reads catalog request to server end asynchronous transmission, otherwise described client reads catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
9. the asynchronous system pre-read of directory entry in distributed file system as claimed in claim 8, it is characterized in that, also comprise described client and find lower one page at described local page buffer memory, check the renewal zone bit of described lower one page, if the set of described renewal zone bit, then described client receives the lower one page preserving described directory entry, if the non-set of described renewal zone bit, then described client continues to wait for, until described server end returns described lower one page, and upgrade one page of described local page buffer memory.
10. the asynchronous system pre-read of directory entry in distributed file system as claimed in claim 9, it is characterized in that, also comprise the described directory entry reading described one page, until the ending mark eof of last directory entry of current page stem preservation is 1, otherwise continue to read catalog request to described server end asynchronous transmission, pre-read the lower one page preserving described directory entry.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510401114.1A CN105138545B (en) | 2015-07-09 | 2015-07-09 | The asynchronous method and system pre-read of directory entry in a kind of distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510401114.1A CN105138545B (en) | 2015-07-09 | 2015-07-09 | The asynchronous method and system pre-read of directory entry in a kind of distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105138545A true CN105138545A (en) | 2015-12-09 |
| CN105138545B CN105138545B (en) | 2018-10-09 |
Family
ID=54723895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510401114.1A Active CN105138545B (en) | 2015-07-09 | 2015-07-09 | The asynchronous method and system pre-read of directory entry in a kind of distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105138545B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107168649A (en) * | 2017-05-05 | 2017-09-15 | 南京城市职业学院 | The method and device of data distribution in a kind of distributed memory system |
| CN107491545A (en) * | 2017-08-25 | 2017-12-19 | 郑州云海信息技术有限公司 | The catalogue read method and client of a kind of distributed memory system |
| CN110765086A (en) * | 2019-10-25 | 2020-02-07 | 浪潮电子信息产业股份有限公司 | Directory reading method and system for small files, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030101200A1 (en) * | 2001-11-28 | 2003-05-29 | Noritaka Koyama | Distributed file sharing system and a file access control method of efficiently searching for access rights |
| CN101388824A (en) * | 2008-10-15 | 2009-03-18 | 中国科学院计算技术研究所 | File reading method and system under sliced memory mode in cluster system |
| CN102385623A (en) * | 2011-10-25 | 2012-03-21 | 曙光信息产业(北京)有限公司 | Catalogue access method in DFS (distributed file system) |
| CN102541985A (en) * | 2011-10-25 | 2012-07-04 | 曙光信息产业(北京)有限公司 | Organization method of client directory cache in distributed file system |
| CN103916465A (en) * | 2014-03-21 | 2014-07-09 | 中国科学院计算技术研究所 | Data pre-reading device based on distributed file system and method thereof |
-
2015
- 2015-07-09 CN CN201510401114.1A patent/CN105138545B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030101200A1 (en) * | 2001-11-28 | 2003-05-29 | Noritaka Koyama | Distributed file sharing system and a file access control method of efficiently searching for access rights |
| CN101388824A (en) * | 2008-10-15 | 2009-03-18 | 中国科学院计算技术研究所 | File reading method and system under sliced memory mode in cluster system |
| CN102385623A (en) * | 2011-10-25 | 2012-03-21 | 曙光信息产业(北京)有限公司 | Catalogue access method in DFS (distributed file system) |
| CN102541985A (en) * | 2011-10-25 | 2012-07-04 | 曙光信息产业(北京)有限公司 | Organization method of client directory cache in distributed file system |
| CN103916465A (en) * | 2014-03-21 | 2014-07-09 | 中国科学院计算技术研究所 | Data pre-reading device based on distributed file system and method thereof |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107168649A (en) * | 2017-05-05 | 2017-09-15 | 南京城市职业学院 | The method and device of data distribution in a kind of distributed memory system |
| CN107168649B (en) * | 2017-05-05 | 2019-12-17 | 南京城市职业学院 | method and device for data distribution in distributed storage system |
| CN107491545A (en) * | 2017-08-25 | 2017-12-19 | 郑州云海信息技术有限公司 | The catalogue read method and client of a kind of distributed memory system |
| CN110765086A (en) * | 2019-10-25 | 2020-02-07 | 浪潮电子信息产业股份有限公司 | Directory reading method and system for small files, electronic equipment and storage medium |
| CN110765086B (en) * | 2019-10-25 | 2022-08-02 | 浪潮电子信息产业股份有限公司 | Directory reading method and system for small files, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105138545B (en) | 2018-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101201827B (en) | Method and system for displaying web page | |
| CN101668004B (en) | A kind of webpage acquisition methods, Apparatus and system | |
| US6754799B2 (en) | System and method for indexing and retrieving cached objects | |
| CN100452041C (en) | Method and system for reading information at network resource site, and searching engine | |
| US9846711B2 (en) | LSM cache | |
| CN108647304B (en) | Report display system, method, computer device and storage medium | |
| CN104252536B (en) | A kind of internet log data query method and device based on hbase | |
| US10747951B2 (en) | Webpage template generating method and server | |
| CN103020315A (en) | Method for storing mass of small files on basis of master-slave distributed file system | |
| CN104679898A (en) | Big data access method | |
| CN102541985A (en) | Organization method of client directory cache in distributed file system | |
| CN103488732A (en) | Generation method and device of static pages | |
| CN101741986A (en) | Page cache method for mobile communication equipment terminal | |
| CN103595797B (en) | Caching method for distributed storage system | |
| CN104765840A (en) | Big data distributed storage method and device | |
| CN102307234A (en) | Resource retrieval method based on mobile terminal | |
| CN109766318B (en) | File reading method and device | |
| CN102722405A (en) | Counting method in high concurrent and multithreaded application and system | |
| CN104778229A (en) | Telecommunication service small file storage system and method based on Hadoop | |
| CN110515920A (en) | A kind of mass small documents access method and system based on Hadoop | |
| CN105138545A (en) | Method and system for asynchronously pre-reading directory entries in distributed file system | |
| CN102438004A (en) | Method and system for acquiring metadata information of media file and multimedia player | |
| CN101770511B (en) | Browse system for accelerating backspace operation of mobile communication equipment terminal and application method thereof | |
| CN105119955B (en) | The method and system that catalogue multipage is supported are read in a kind of distributed file system | |
| CN106909623B (en) | A kind of data set and date storage method for supporting efficient mass data to analyze and retrieve |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |