CN108319634A - The directory access method and apparatus of distributed file system - Google Patents
The directory access method and apparatus of distributed file system Download PDFInfo
- Publication number
- CN108319634A CN108319634A CN201711347711.6A CN201711347711A CN108319634A CN 108319634 A CN108319634 A CN 108319634A CN 201711347711 A CN201711347711 A CN 201711347711A CN 108319634 A CN108319634 A CN 108319634A
- Authority
- CN
- China
- Prior art keywords
- directory
- client
- file
- memory cache
- specified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The present invention provides a kind of directory access method and apparatus of distributed file system, and this method includes：Step 10：Client generates the specified directory access request to distributed file system according to user's operation；Step 11：Judge that if client thens follow the steps 12 with the presence or absence of the first memory cache of specified directory there are the first memory cache of specified directory；Step 12：Directory entry in the first memory cache that client is loaded into specified directory；Step 13：Continue to judge that client whether there is local directory file, if local directory file, there are local directory file, thens follow the steps 14 to preserve the directory entry under specified directory；Step 14：It is loaded into the directory entry in specified directory file in client.The directory access method and apparatus of the distributed file system of the present invention, can improve the access speed of a small amount of catalogue or file and the above catalogue of million ranks or file, improve the user experience of client.
Description
Technical field
The present invention relates to computer realm, more particularly to the directory access method and apparatus of a kind of distributed file system.
Background technology
Currently, large-scale distributed file system can provide PB grades even EB grades of data storage, memory space is not
It is the bottleneck of memory technology again.Theoretically, distributed file system can provide uncountable number of files.
On the other hand, if a catalogue in distributed file system contains a subfiles up to a million or subdirectory,
When client accesses the catalogue by network, since client's end memory can not cache above subfiles up to a million or subdirectory,
Therefore it often causes client low memory or memory to exhaust, gently then influences user's access speed, client is caused when serious
It crashes, affects user experience.
In view of the above-mentioned problems, currently no effective solution has been proposed.
Invention content
In view of this, the present invention provides a kind of directory access method and apparatus of distributed file system, client is solved
The problem of speed is slow when accessing heap file or catalogue or crashes.
The present invention provides a kind of directory access method of distributed file system, and this method includes：
Step 10：Client generates the specified directory access request to distributed file system according to user's operation；
Step 11：Judge that client whether there is the first memory cache of specified directory, if there are the first of specified directory
Memory cache thens follow the steps 12；
Step 12：Directory entry in the first memory cache that client is loaded into specified directory；
Step 13：Continue to judge that client whether there is local directory file, local directory file is preserving specified mesh
If directory entry under record thens follow the steps 14 there are local directory file；
Step 14：It is loaded into the directory entry in specified directory file in client.
The present invention also provides a kind of directory access device of distributed file system, which includes：
User requests to generate module：Client is generated and is visited the specified directory of distributed file system according to user's operation
Ask request；
Comparison module：Judge that client whether there is the first memory cache of specified directory, if there are the of specified directory
One memory cache then executes caching and is loaded into mould；
Caching insmods：Directory entry in the first memory cache that client is loaded into specified directory；
Comparison module 1：Continue to judge that client whether there is local directory file, local directory file is specified to preserve
If directory entry under catalogue executes file and insmods there are local directory file；
File insmods：It is loaded into the directory entry in specified directory file in client.
In the present invention, the directory entry storage of each specified directory is divided into two parts, and directory entry is less than the first predetermined directory
In first memory cache of the specified directory that the part of item (such as 100,000) is cached in client；Directory entry is more than the first predetermined directory
The part of item (such as 100,000) is stored in the file of client local disk.When user accesses distributed field system by client
When the catalogue for server-side of uniting, by being loaded into corresponding directory entry and/or the directory entry being locally stored in caching, can not only it improve
The speed of catalogue neutron file item or subdirectory item is accessed, while the client of the application limits part memory in client's end memory
That is the first memory cache is used to preserve the directory entry for the specified directory that client is downloaded from distributed file system server-side, not shadow
The overall operation for ringing FTP client FTP will not bring and access heap file or the low memory brought or what is exhausted ask when catalogue
Topic.
The directory access method and apparatus of distributed file system of the present invention not only increase a small amount of number of files destination directory
Access speed also improves access and the inquiry velocity of the above file directory of million ranks, improves the user experience of client.
Description of the drawings
Fig. 1 is the first embodiment of the directory access method of distributed file system of the present invention；
Fig. 2 is the second embodiment of the directory access method of distributed file system of the present invention；
Fig. 3 is the 3rd embodiment of the directory access method of distributed file system of the present invention；
Fig. 4 is the fourth embodiment of the directory access method of distributed file system of the present invention；
Fig. 5 is the structure chart of the directory access device of distributed file system of the present invention.
Specific implementation mode
To make the objectives, technical solutions, and advantages of the present invention clearer, right in the following with reference to the drawings and specific embodiments
The present invention is described in detail.
Client accesses the directory entry obtained when distributed file system server-side in the present invention, in the storage of client
It is divided into two parts, directory entry is cached in the first of the specified directory of client less than the part of the first predetermined directory item (such as 100,000)
In memory cache；Directory entry is stored in the file of client local disk more than the part of the first predetermined directory item (such as 100,000)
In.
Based on this, the directory access method of the distributed file system of the present invention is proposed, as shown in Figure 1, this method includes
Following steps：
Step 10 (S101)：Client is generated and is asked to the specified directory access of distributed file system according to user's operation
It asks.
In step 10, user's operation can be the mouse clicking operation to specified directory.
Step 11 (S102)：Judge that client whether there is the first memory cache of specified directory, if there are specified directories
The first memory cache, then follow the steps 12；
Step 12 (S103)：Directory entry in the first memory cache that client is loaded into specified directory；
Step 13 (S104)：Continue to judge that client whether there is local directory file, local directory file is preserving
If directory entry under specified directory thens follow the steps 14 there are local directory file；
Step 14 (S105)：It is loaded into the directory entry in specified directory file in client.
In the method for Fig. 1, local directory file, can for the ease of search to preserve the directory entry under specified directory
Local directory file is stored in preset storage location, such as some file.Each local directory file is only deposited
The directory entry under a certain specified directory is put, different specified directories has different local directory files.Further, in order to just
Local directory file is specified in positioning, the filename of local directory file can be the link information of specified directory, such as specified mesh
The full directory information of record is filename, or with specified directory inode (index node) for filename.
Optionally, step 12 further includes：Directory entry under specified directory in first memory cache is subdivided into X catalogue
Group, client are loaded into the directory entry under X directory group successively.
Optionally, step 14 further includes：The directory entry for including in local directory file is subdivided into Y directory group, client
End is loaded into the directory entry under Y directory group successively.
When user accesses the catalogue of distributed file system by client, by being loaded into corresponding directory entry in caching
And/or the directory entry being locally stored, it can not only improve the speed for accessing catalogue neutron file item or subdirectory item.Meanwhile by
Part memory i.e. the first memory cache in client's end memory is limited in the client of the application to be used to preserve client from distribution
The directory entry that file system is downloaded, does not influence the overall operation of FTP client FTP, will not bring and access heap file or catalogue
When the low memory that brings or the problem of exhaust.
Further, if the first memory cache of specified directory does not update within the preset time, specified mesh is emptied
First memory cache of record.
When accessing different specified directories because of user, can correspond in memory generate different specified directories the first memory it is slow
It deposits, using " memory is removed automatically " strategy, can avoid more and more first memory caches of generation and constantly consume in client
It deposits, it is ensured that the speed of service of FTP client FTP.In addition, " memory is removed automatically " also ensures that the first memory cache or sheet in Fig. 1
The directory entry that ground catalogue file preserves is the directory entry at newest moment, need to only be loaded into the directory entry of client preservation, without visiting
Ask that the server of distributed file system obtains newest directory entry information, saving network flow provides user's access speed.
Preset time, can rule of thumb or the Installed System Memory of user demand or client setting, such as 30 seconds or 1 minute.
Further, as shown in Fig. 2, the step 11 of Fig. 1 further includes：If there is no the first memory cache of specified directory,
Then follow the steps 21；
Step 21 (S201)：Specified directory access request is sent to the server-side of distributed file system by client；
Step 22 (S202)：The server-side of distributed file system is read according to specified directory access request under specified directory
Directory entry, directory entry is subdivided into N number of directory group and is sent to client；
Step 23 (S203)：Judge whether the directory entry sum in the second memory cache of currently assigned catalogue is more than or equal to
First predetermined directory item, if so, 24 are thened follow the steps, if not, thening follow the steps 25；
Step 24 (S204)：The directory entry of n-th of the directory group received is updated the local mesh to client by client
It records in file, n=1,2 ... N, client is loaded into the directory entry (directory entry of n-th of directory group of display) of n-th of directory group, returns
Step 23 is returned, until N number of directory group receives；
Step 25 (S205)：The directory entry of n-th of the directory group received is saved in the second of specified directory by client
The end of memory cache, client are loaded into the directory entry (directory entry of n-th of directory group of display) of n-th of directory group, return to step
Rapid 23, until N number of directory group receives；
Step 26 (S206)：Directory entry in second memory cache of specified directory is saved in the first of specified directory
After depositing caching, the second memory cache is emptied.
Further, it is contemplated that after all directory entries all receive, the directory entry in the second memory cache may
More than the first predetermined directory item, as shown in figure 3, step 26 can also include the following steps：
Step 27 (S207)：Judge whether the directory entry sum in the second memory cache of currently assigned catalogue is more than first
Predetermined directory item, if so, 28 are thened follow the steps, if not, thening follow the steps 29；
Step 28 (S208)：First predetermined directory item before in second memory cache of specified directory is saved in specified mesh
In first memory cache of record, by other catalogues except the first predetermined directory item before in the second memory cache of specified directory
In item update to the local directory file of specified directory, the second memory cache is emptied；
Step 29 (S209)：Directory entry in second memory cache of specified directory is saved in the first of specified directory
After depositing caching, the second memory cache is emptied.
Further, in the step 24 of Fig. 2, client updates the directory entry of n-th of the directory group received to visitor
The local directory file at family end includes：Local directory file is judged whether, if it is not, newly-built local directory text
Part, and the directory entry of n-th of directory group is saved in the newly-built local directory file of client；If there is local directory text
Part then continues to judge that the renewal time of local directory file for last time update or this update, updates if it is the last time,
The directory entry that the raw content of the local directory file of client is then replaced with to n-th of directory group, if it is this update, by the
The directory entry of n directory group is appended in the local directory file of client.
Judge local directory file is updated to last update or this update, can be according to local directory file
Renewal time stabs the comparison with current time to judge, if two times are very close, such as less than 1 minute, then be this
Otherwise update updates for the last time.
In the method for Fig. 2, server can determine the value of N according to the directory entry sum under specified directory, such as：
If directory entry sum is less than 20, N=1 can be enabled, if directory entry sum is more than 1000, N=100 can be enabled.
Fig. 2 is when user accesses distributed file system by client, and it is slow that there is no the first memories of specified directory
Deposit, then illustrate user may be the first memory cache of the specified directory for accessing the specified directory for the first time, or access before
By " automatic to remove ", at this point, the directory entry under the specified directory can only be obtained by connecting distributed file system server-side.
In the method for Fig. 2, the directory entry of acquisition is divided into N number of directory group by distributed file system, is with directory group
Directory entry under specified directory is sent to client by unit, and client often receives a directory group, first by the catalogue of the directory group
Item is saved in the second memory cache, after receiving, then the directory entry in the second memory cache is transferred to the first memory and is delayed
In depositing.Second memory cache can be that backstage caches, and correspondingly, the first memory cache can be that foreground caches.The of the application
It is the first predetermined directory item that the directory entry of one (two) memory cache, which is provided with the directory entry upper limit, and the first predetermined directory item can be the
The maximum directory entry that one memory cache can be stored, such as the first predetermined directory item are 100,000.
Directory entry is divided into N number of directory group by the method for Fig. 2, and receiving while may be implemented and refresh the mesh that client is loaded into
Item is recorded, need not so wait receptions completely after a directory entry, reload to user, user experience can be improved.
In the above-mentioned methods, no matter client locally reads directory entry (the first memory cache or local directory file), also
It is that distributed file system server-side reads directory entry, can realizes reading by several times by building iterator.
For example, setting directory entry number that iterator is read every time as a fixed numbers, can change according to demand, it is assumed that be
128.The current position for reading the directory entry in caching container can be recorded by running through the position of iterator after 128 files every time
(offset, corresponding directory entry serial number) and name (name, corresponding directory entry title).Next time reads direct when 128 directory entries again
Using offset or name positioning iterators position, avoids traversing whole container again, accelerate the reading of directory entry.
Offset and name can be all kept after running through 128 directory entries every time, even if network causes to read when occurring abnormal
Interruption is taken, the position of iterator can also be repositioned by the offset and name of preservation, guarantee will not read or mispronounce less
Data.Meanwhile even if the last one directory entry is deleted, also iterator position can be navigated to by offset and name,
It is accurate to ensure that iteration reads data.
As shown in figure 4, the application distributive catalogue of document system access method further includes：
Step 30 (S301)：When client creates new file in distributed file system, the directory entry of new file is increased
It is added in the first memory cache of new file parent directory；
Step 31 (S302)：Directory entry in first memory cache of new file parent directory is ranked up, the side of sequence
Method is consistent with the sort method of the directory entry at distribution system services end；
Step 32 (S303)：Judge whether the directory entry number in the first memory cache of new file parent directory is more than first
The sum of predetermined directory item and the second predetermined directory item, if so, thening follow the steps 33；
Step 33 (S304)：Catalogue entry deletion will be exceeded in first memory cache of new file parent directory；Beyond catalogue
Exceed the directory entry of the first predetermined directory item in first memory cache of Xiang Weixin file parent directories.
In the step 31 of Fig. 3, client and distributed file system use identical sort algorithm, are to ensure visitor
The reading order for the directory entry that family end includes and distributed file system read it is smoothly consistent when directory entry, when display, in display
Hold consistent.
In the step 32 of Fig. 3, the second predetermined directory item is in order to avoid one file of every increase will execute step 33
Once, it is assumed that the first predetermined directory item is 100,000, and the second predetermined directory item is 280, when in the first memory cache of specified directory
Directory entry it is primary beyond step 33 after 100280, is just executed, focus on and can avoid occupying resource, raising efficiency.
In addition, the application distributive catalogue of document system access method further includes：
Step 40：When client deletes the file in distributed file system, judge to delete the corresponding directory entry of file
Whether it is stored in the first memory cache of deletion file parent directory of client, if so, the of file parent directory will be deleted
The corresponding catalogue entry deletion of deletion file in one memory cache.
Distributive catalogue of document system access method of the present invention and device not only increase a small amount of number of files destination directory and visit
It asks speed, also improves access and the inquiry velocity of the above file directory of million ranks, improve the user experience of client.
The present invention also provides a kind of directory access device of distributed file system, the device as shown in Figure 5 includes following
Module：
User requests to generate module：Client is generated and is visited the specified directory of distributed file system according to user's operation
Ask request；
Comparison module：Judge that client whether there is the first memory cache of specified directory, if there are the of specified directory
One memory cache then executes caching and is loaded into mould；
Caching insmods：Directory entry in the first memory cache that client is loaded into specified directory；
Comparison module 1：Continue to judge that client whether there is local directory file, local directory file is specified to preserve
If directory entry under catalogue executes file and insmods there are local directory file；
File insmods：It is loaded into the directory entry in specified directory file in client.
Optionally, if the first memory cache of specified directory does not update within the preset time, specified directory is emptied
The first memory cache.
Optionally, comparison module further includes in Figure 5：If there is no the first memory cache of specified directory, use is executed
Family request sending module；
User's request sending module：Specified directory access request is sent to the service of distributed file system by client
End；
Server-side catalogue module：The server-side of distributed file system reads specified directory according to specified directory access request
Under directory entry, directory entry is subdivided into N number of directory group and is sent to client；
Comparison module 2：Judge whether directory entry number in the second memory cache of currently assigned catalogue is more than or equal to the
One predetermined directory item, if it is, local directory file update module is executed, if it is not, then executing the second buffer update module；
Local directory file update module：Client updates the directory entry of n-th of the directory group received to client
Local directory file in, n=1,2 ... N, client are loaded into the directory entry of n-th of directory group, comparison module 2 are returned to, until N
A directory group receives；
Second buffer update module：The directory entry of n-th of the directory group received is saved in specified directory by client
The end of second memory cache, client are loaded into the directory entry of n-th of directory group, return to comparison module 2, until N number of directory group
It receives；
First buffer update module：Directory entry in second memory cache of specified directory is saved in the of specified directory
After one memory cache, second memory cache is emptied.
Optionally, the first buffer update module includes：Comparison module 3, the first buffer update module 1 and the first buffer update
Module 2.
Comparison module 3：It is pre- to judge whether the directory entry sum in the second memory cache of currently assigned catalogue is more than first
If directory entry, the first buffer update module 1 is if it is executed, if it is not, then executing the first buffer update module 2.
First buffer update module 1：First predetermined directory item before in second memory cache of specified directory is saved in
In first memory cache of specified directory, by its except the first predetermined directory item before in the second memory cache of specified directory
In his directory entry update to the local directory file of specified directory, the second memory cache is emptied.
First buffer update module 2：Directory entry in second memory cache of specified directory is saved in specified directory
After first memory cache, the second memory cache is emptied.
Optionally, the device of Fig. 5 further includes：
New files module：When client creates new file in distributed file system, the directory entry of new file is increased
It is added in the first memory cache of new file parent directory；
Directory entry sorting module：Directory entry in first memory cache of new file parent directory is ranked up, sequence
Method is consistent with the sort method of the directory entry at distribution system services end；
Comparison module 4：It is pre- to judge whether the directory entry number in the first memory cache of new file parent directory is more than first
If the sum of directory entry and the second predetermined directory item, if so, thening follow the steps 43；
Removing module 1：Catalogue entry deletion will be exceeded in first memory cache of new file parent directory；It is beyond directory entry
Exceed the directory entry of the first predetermined directory item in first memory cache of new file parent directory.
Optionally, the device of Fig. 5 further includes：
Removing module 2：When client deletes the file in distributed file system, judge to delete the corresponding catalogue of file
Whether item is stored in the first memory cache of deletion file parent directory of client, if so, file parent directory will be deleted
The corresponding catalogue entry deletion of deletion file in first memory cache.
Optionally, in the directory access device of the distributed file system of the present invention, local directory file is to specify mesh
Record inode is filename.
It should be noted that the embodiment of the directory access device of the distributed file system of the present invention, with distribution text
The embodiment principle of the directory access method of part system is identical, and related place can mutual reference.
The foregoing is merely illustrative of the preferred embodiments of the present invention, not to limit the present invention scope, it is all
Within the spirit and principle of technical solution of the present invention, any modification, equivalent substitution, improvement and etc. done should be included in this hair
Within bright protection domain.
Claims (10)
1. a kind of directory access method of distributed file system, which is characterized in that the described method comprises the following steps：
Step 10：Client generates the specified directory access request to distributed file system according to user's operation；
Step 11：Judge that client whether there is the first memory cache of the specified directory, if there are the specified directories
First memory cache, thens follow the steps 12；
Step 12：Directory entry in the first memory cache that client is loaded into the specified directory；
Step 13：Continue to judge that client whether there is local directory file, the local directory file is preserving the finger
Determine the directory entry under catalogue, if there are the local directory file, thens follow the steps 14；
Step 14：It is loaded into the directory entry in the specified directory file in client.
2. according to the method described in claim 1, it is characterized in that, if the first memory cache of the specified directory is default
Time in do not update, then empty the first memory cache of the specified directory.
3. according to the method described in claim 1, it is characterized in that, the step 11 further includes：If the specified mesh is not present
First memory cache of record, thens follow the steps 21；
Step 21：The specified directory access request is sent to the server-side of distributed file system by client；
Step 22：The server-side of the distributed file system reads the specified directory according to the specified directory access request
Under directory entry, the directory entry is subdivided into N number of directory group and is sent to client；
Step 23：It is pre- to judge whether the directory entry sum in the second memory cache of presently described specified directory is more than or equal to first
If directory entry, if so, 24 are thened follow the steps, if not, thening follow the steps 25；
Step 24：The directory entry of n-th of the directory group received is updated the local directory file to client by client
In, n=1,2 ... N, client are loaded into the directory entry of n-th of directory group, return to the step 23, until N number of catalogue
Group receives；
Step 25：The second memory that the directory entry of n-th of the directory group received is saved in the specified directory by client delays
The end deposited, client are loaded into the directory entry of n-th of directory group, return to the step 23, until N number of directory group connects
It harvests complete；
Step 26：Directory entry in second memory cache of the specified directory is saved in the first memory of the specified directory
After caching, second memory cache is emptied.
4. according to the method described in claim 1, it is characterized in that, the method further includes：
Step 30：When client creates new file in the distributed file system, the directory entry of the new file is increased
Into the first memory cache of the new file parent directory；
Step 31：Directory entry in first memory cache of the new file parent directory is ranked up, the method for the sequence
It is consistent with the sort method of directory entry at distribution system services end；
Step 32：Judge whether the directory entry number in the first memory cache of the new file parent directory is more than the first default mesh
The sum of item and the second predetermined directory item are recorded, if so, thening follow the steps 33；
Step 33：Catalogue entry deletion will be exceeded in first memory cache of the new file parent directory；It is described to exceed directory entry
To exceed the directory entry of the first predetermined directory item in the first memory cache of the new file parent directory.
5. according to the method described in claim 1, it is characterized in that, the method further includes：
Step 40：When client deletes the file in the distributed file system, the corresponding mesh of the deletion file is judged
Whether record item is stored in the first memory cache of the deletion file parent directory of client, if so, by deletion text
The corresponding catalogue entry deletion of the deletion file in first memory cache of part parent directory.
6. a kind of directory access device of distributed file system, which is characterized in that described device includes：
User requests to generate module：Client is generated and is asked to the specified directory access of distributed file system according to user's operation
It asks；
Comparison module：Judge that client whether there is the first memory cache of the specified directory, if there are the specified directories
The first memory cache, then execute caching be loaded into mould；
Caching insmods：Directory entry in the first memory cache that client is loaded into the specified directory；
Comparison module 1：Continue to judge that client whether there is local directory file, the local directory file is described to preserve
If directory entry under specified directory executes file and insmods there are the local directory file；
File insmods：It is loaded into the directory entry in the specified directory file in client.
7. device according to claim 6, which is characterized in that if the first memory cache of the specified directory is default
Time in do not update, then empty the first memory cache of the specified directory.
8. device according to claim 6, which is characterized in that the comparison module further includes：If there is no described specified
First memory cache of catalogue, then execute user's request sending module；
User's request sending module：The specified directory access request is sent to the service of distributed file system by client
End；
Server-side catalogue module：The server-side of the distributed file system is according to described in specified directory access request reading
The directory entry is subdivided into N number of directory group and is sent to client by the directory entry under specified directory；
Comparison module 2：Judge whether directory entry number in the second memory cache of presently described specified directory is more than or equal to the
One predetermined directory item, if it is, local directory file update module is executed, if it is not, then executing the second buffer update module；
Local directory file update module：The directory entry of n-th of the directory group received is updated the institute to client by client
It states in local directory file, n=1,2 ... N, client is loaded into the directory entry of n-th of directory group, returns to the comparison module
2, until N number of directory group receives；
Second buffer update module：The directory entry of n-th of the directory group received is saved in the specified directory by client
The end of second memory cache, client are loaded into the directory entry of n-th of directory group, return to the comparison module 2, until institute
N number of directory group is stated to receive；
First buffer update module：Directory entry in second memory cache of the specified directory is saved in the specified directory
The first memory cache after, empty second memory cache.
9. device according to claim 6, which is characterized in that described device further includes：
New files module：When client creates new file in the distributed file system, by the catalogue of the new file
Item increases in the first memory cache of the new file parent directory；
Directory entry sorting module：Directory entry in first memory cache of the new file parent directory is ranked up, the row
The method of sequence is consistent with the sort method of directory entry at distribution system services end；
Comparison module 4：It is pre- to judge whether the directory entry number in the first memory cache of the new file parent directory is more than first
If the sum of directory entry and the second predetermined directory item, if it is, executing removing module 1；
Removing module 1：Catalogue entry deletion will be exceeded in first memory cache of the new file parent directory；It is described to exceed catalogue
Item is the directory entry beyond the first predetermined directory item in the first memory cache of the new file parent directory.
10. device according to claim 6, which is characterized in that described device further includes：
Removing module 2：When client deletes the file in the distributed file system, judge that the deletion file is corresponding
Whether directory entry is stored in the first memory cache of the deletion file parent directory of client, if so, by the deletion
The corresponding catalogue entry deletion of the deletion file in first memory cache of file parent directory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711347711.6A CN108319634B (en) | 2017-12-15 | 2017-12-15 | Directory access method and device for distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711347711.6A CN108319634B (en) | 2017-12-15 | 2017-12-15 | Directory access method and device for distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108319634A true CN108319634A (en) | 2018-07-24 |
| CN108319634B CN108319634B (en) | 2021-08-06 |
Family
ID=62892003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711347711.6A Active CN108319634B (en) | 2017-12-15 | 2017-12-15 | Directory access method and device for distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108319634B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287201A (en) * | 2019-07-02 | 2019-09-27 | 重庆紫光华山智安科技有限公司 | Data access method, device, equipment and storage medium |
| CN110781137A (en) * | 2019-10-28 | 2020-02-11 | 柏科数据技术（深圳）股份有限公司 | Directory reading method and device for distributed system, server and storage medium |
| CN110781159A (en) * | 2019-10-28 | 2020-02-11 | 柏科数据技术（深圳）股份有限公司 | Ceph directory file information reading method and device, server and storage medium |
| CN114048185A (en) * | 2021-11-18 | 2022-02-15 | 北京聚存科技有限公司 | Method for transparently packaging, storing and accessing massive small files in distributed file system |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697168A (en) * | 2009-10-22 | 2010-04-21 | 中国科学技术大学 | Method and system for dynamically managing metadata of distributed file system |
| CN102024020A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Efficient metadata memory access method in distributed file system |
| CN102024017A (en) * | 2010-11-04 | 2011-04-20 | 天津曙光计算机产业有限公司 | Method for traversing directory entries of distribution type file system in repetition-free and omission-free way |
| CN102024019A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Suffix tree based catalog organizing method in distributed file system |
| CN102523301A (en) * | 2011-12-26 | 2012-06-27 | 深圳市创新科信息技术有限公司 | Method for caching data on client in cloud storage |
| CN102541985A (en) * | 2011-10-25 | 2012-07-04 | 曙光信息产业(北京)有限公司 | Organization method of client directory cache in distributed file system |
| CN102955808A (en) * | 2011-08-26 | 2013-03-06 | 腾讯科技（深圳）有限公司 | Data acquisition method and distributed file system |
| CN103150394A (en) * | 2013-03-25 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Distributed file system metadata management method facing to high-performance calculation |
| CN103338242A (en) * | 2013-06-20 | 2013-10-02 | 华中科技大学 | Hybrid cloud storage system and method based on multi-level cache |
| US20140214889A1 (en) * | 2013-01-30 | 2014-07-31 | Hewlett-Packard Development Company, L.P. | Anticipatorily Retrieving Information In Response To A Query Of A Directory |
| US8805901B1 (en) * | 2011-07-19 | 2014-08-12 | Google Inc. | Geographically distributed file system |
| CN104008152A (en) * | 2014-05-21 | 2014-08-27 | 华南理工大学 | Distributed file system architectural method supporting mass data access |
| CN105095785A (en) * | 2014-05-22 | 2015-11-25 | 中兴通讯股份有限公司 | File access processing method, and file access method and device of distributed file system |
| CN106686113A (en) * | 2017-01-19 | 2017-05-17 | 郑州云海信息技术有限公司 | Intelligent pre-reading implementation method for distributed file system |
-
2017
- 2017-12-15 CN CN201711347711.6A patent/CN108319634B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697168A (en) * | 2009-10-22 | 2010-04-21 | 中国科学技术大学 | Method and system for dynamically managing metadata of distributed file system |
| CN102024020A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Efficient metadata memory access method in distributed file system |
| CN102024017A (en) * | 2010-11-04 | 2011-04-20 | 天津曙光计算机产业有限公司 | Method for traversing directory entries of distribution type file system in repetition-free and omission-free way |
| CN102024019A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Suffix tree based catalog organizing method in distributed file system |
| US8805901B1 (en) * | 2011-07-19 | 2014-08-12 | Google Inc. | Geographically distributed file system |
| CN102955808A (en) * | 2011-08-26 | 2013-03-06 | 腾讯科技（深圳）有限公司 | Data acquisition method and distributed file system |
| CN102541985A (en) * | 2011-10-25 | 2012-07-04 | 曙光信息产业(北京)有限公司 | Organization method of client directory cache in distributed file system |
| CN102523301A (en) * | 2011-12-26 | 2012-06-27 | 深圳市创新科信息技术有限公司 | Method for caching data on client in cloud storage |
| US20140214889A1 (en) * | 2013-01-30 | 2014-07-31 | Hewlett-Packard Development Company, L.P. | Anticipatorily Retrieving Information In Response To A Query Of A Directory |
| CN103150394A (en) * | 2013-03-25 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Distributed file system metadata management method facing to high-performance calculation |
| CN103338242A (en) * | 2013-06-20 | 2013-10-02 | 华中科技大学 | Hybrid cloud storage system and method based on multi-level cache |
| CN104008152A (en) * | 2014-05-21 | 2014-08-27 | 华南理工大学 | Distributed file system architectural method supporting mass data access |
| CN105095785A (en) * | 2014-05-22 | 2015-11-25 | 中兴通讯股份有限公司 | File access processing method, and file access method and device of distributed file system |
| CN106686113A (en) * | 2017-01-19 | 2017-05-17 | 郑州云海信息技术有限公司 | Intelligent pre-reading implementation method for distributed file system |
Non-Patent Citations (2)
| Title |
|---|
| XIUQIAO LI 等: "CEFLS: A Cost-Effective File Lookup Service in a Distributed Metadata File System", 《2012 12TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID 2012)》 * |
| 冯幼乐: "分布式文件系统元数据管理技术研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287201A (en) * | 2019-07-02 | 2019-09-27 | 重庆紫光华山智安科技有限公司 | Data access method, device, equipment and storage medium |
| CN110781137A (en) * | 2019-10-28 | 2020-02-11 | 柏科数据技术（深圳）股份有限公司 | Directory reading method and device for distributed system, server and storage medium |
| CN110781159A (en) * | 2019-10-28 | 2020-02-11 | 柏科数据技术（深圳）股份有限公司 | Ceph directory file information reading method and device, server and storage medium |
| CN114048185A (en) * | 2021-11-18 | 2022-02-15 | 北京聚存科技有限公司 | Method for transparently packaging, storing and accessing massive small files in distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108319634B (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9513817B2 (en) | Free space collection in log structured storage systems | |
| Liu et al. | Implementing WebGIS on Hadoop: A case study of improving small file I/O performance on HDFS | |
| JP4824753B2 (en) | Efficient handling of time-limited messages | |
| US7103794B2 (en) | Network object cache engine | |
| US8463846B2 (en) | File bundling for cache servers of content delivery networks | |
| CN107943718B (en) | Method and device for cleaning cache file | |
| CN110162528A (en) | Magnanimity big data search method and system | |
| CN108319634A (en) | The directory access method and apparatus of distributed file system | |
| KR101672901B1 (en) | Cache Management System for Enhancing the Accessibility of Small Files in Distributed File System | |
| US11561930B2 (en) | Independent evictions from datastore accelerator fleet nodes | |
| US9152575B2 (en) | Data staging area | |
| CN106155934B (en) | Caching method based on repeated data under a kind of cloud environment | |
| US8868863B2 (en) | Method and apparatus for a frugal cloud file system | |
| US10078649B2 (en) | Pre-caching of relational database management system based on data retrieval patterns | |
| WO2014201942A1 (en) | Method, server and system for managing content in content delivery network | |
| CN103338242B (en) | A kind of mixed cloud storage system based on multi-level buffer and method | |
| CN109144413A (en) | A kind of metadata management method and device | |
| CN110321325A (en) | File inode lookup method, terminal, server, system and storage medium | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| CN100458792C (en) | Method and data processing system for managing a mass storage system | |
| CN105915619B (en) | Take the cyberspace information service high-performance memory cache method of access temperature into account | |
| CN107562806A (en) | Mix the adaptive perception accelerated method and system of memory file system | |
| US7836248B2 (en) | Methods and systems for managing persistent storage of small data objects | |
| CN102055795A (en) | Distributed file system metadata management method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 518057 Shenzhen Software Park, No. 9, 501, 502, Science and Technology Middle Road, Nanshan District, Shenzhen City, Guangdong Province Applicant after: Shenzhen Innovation Technology Co., Ltd. Address before: 518057 Shenzhen Software Park, No. 9, 501, 502, Science and Technology Middle Road, Nanshan District, Shenzhen City, Guangdong Province Applicant before: UIT Storage Technology (Shenzhen) Co., Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |