CN104202384A - File uploading and searching method of distributed file system - Google Patents
File uploading and searching method of distributed file system Download PDFInfo
- Publication number
- CN104202384A CN104202384A CN201410425307.6A CN201410425307A CN104202384A CN 104202384 A CN104202384 A CN 104202384A CN 201410425307 A CN201410425307 A CN 201410425307A CN 104202384 A CN104202384 A CN 104202384A
- Authority
- CN
- China
- Prior art keywords
- file
- server
- client
- load
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a file uploading and searching method of a distributed file system, relates to the distributed file system and aims at providing a distributed file system based file uploading and searching method capable of realizing rapid positioning of an object file. The technical key points are that: a client sends out a file uploading request to a load balancing server; the load balancing server receives the file uploading request and generates a file storing name to ensure that the file name can directly locate the storage position of the file; the load balancing server distributes a file server used for storing the uploaded file for the client, and returns the file storing name and the address of the file server to the client; the client visits the file server based on the address, sends the file and the file storing name to the file server, and the like.
Description
Technical field
The present invention relates to distributed file system, the method that especially a kind of file of distributed file system is uploaded and searched.
Background technology
Along with the development of cloud memory technology, a large amount of user's access are all huge tests to the data throughout of cloud server and system concurrency amount.
Distributed file system puts forward the demand of file server access in order to meet a large amount of clients.Its basic thought is that to utilize many file servers be a large amount of client service.The efficiency of the file service providing for client has effectively been provided this method.
It is found that afterwards, in this system, some file servers are very high by the frequency of client-access, operating load amount is large especially, and some file servers are in idle state.Can not make full use of the advantage of file server resources, performance system.For this reason, people have proposed the thought of load balancing.
Distributed file system comprises a load-balanced server and many file servers.Its operation principle is, client is sent access request to load-balanced server, require access file server, load-balanced server is assigned a less file server of load capacity according to the operating state of each file server for client and is provided file service for it.File service in the application mainly refers to that file server provides file storage for client, the operation such as client can complete the uploading of file, downloads on file server, renewal.
In file server, stored mass file, if all need to travel through whole file server storage area by causing a large amount of time loss when each client query or transaction file, to the traversal of file server, also meaned for disk access pressure very large.
Summary of the invention
Technical problem to be solved by this invention is: for the problem of above-mentioned existence, the invention provides a kind of new file based on distributed file system and upload and lookup method, to realize the direct location for file destination, reduce the consumption of query time.
The file uploading method of distributed file system provided by the invention, comprising:
Step 1: client is sent file upload request to load-balanced server;
Step 2: load-balanced server receives described file upload request, for preserving name by file of the file generated of uploading;
Step 3: load-balanced server is that client distributes one for preserving the file server of upload file; The address of described file being preserved to name and file server returns to client;
Step 4: client, according to described address access file server, sends to file server by file and described file preservation name;
Step 5: file server is store path of described file allocation according to self file storage directory, is kept at described file under described store path, and file described in file RNTO is preserved to name;
Step 6: file server returns to load-balanced server and the store path of file and file preservation name are returned to load-balanced server uploading successful message;
Step 7: load-balanced server is combined file server address, file store path and file preservation name to be encrypted coding, obtain a character string, and the file access title using described character string as upload file returns to client;
Step 8: client shows described file access title to user.
Further, described file preservation name comprises current time and file attribute; Described file attribute at least comprises file type and file size.
Further, described file server is divided into many groups, and each file server group has some file servers, is positioned at the file server data backup each other of same group.
Further, described step 7 is: load-balanced server is combined file server grouping information, file server address, file store path and file preservation name to be encrypted coding, obtain a character string, and the file access title using described character string as upload file returns to client.
Preferably, in file server, the file of client upload is according to three layers of tree-like file structure storage.
The present invention also provides a kind of ff method of distributed file system, comprising:
Step 1: client is initiated access request and file access title is sent to load-balanced server to load-balanced server;
Step 2: load-balanced server resolution file access title obtains file server address, file store path and the file at file place and preserves name; And described file server address, file store path and file preservation name are returned to client;
Step 3: client is preserved name according to file server address, file store path and file and find corresponding file on file server.
In sum, owing to having adopted technique scheme, the invention has the beneficial effects as follows:
The inventive method can navigate to fast file server and the store path thereof at file place in heap file server, has saved the time of file access and has reduced file server disk access amount.In addition, the present invention is shown to user after the true storing path of file is encrypted to coding, and user can not see the true path of file storage, for file designation provides the larger degree of freedom, realizes the concurrent and high security feature of the height of system.
Accompanying drawing explanation
Examples of the present invention will be described by way of reference to the accompanying drawings, wherein:
Fig. 1 is that file of the present invention is uploaded flow chart.
Fig. 2 is the flow chart of a specific embodiment of ff of the present invention.
Fig. 3 is that in the present invention, file server creates file storage directory Establishing process figure.
Embodiment
Disclosed all features in this specification, or the step in disclosed all methods or process, except mutually exclusive feature and/or step, all can combine by any way.
Disclosed arbitrary feature in this specification, unless narration especially all can be replaced by other equivalences or the alternative features with similar object.That is,, unless narration especially, each feature is an example in a series of equivalences or similar characteristics.
As Fig. 1, the file uploading method of distributed file system provided by the invention, comprising:
Step 1: client is sent file upload request to load-balanced server.In other embodiments, load-balanced server can be resolution server.
Step 2: load-balanced server receives described file upload request, for preserving name by file of the file generated of uploading.In other embodiments, this step is load-balanced server according to the file attribute information in request, for one of described file generated comprises the file of file attribute, preserves a name NFS(Name in File Service).This document is preserved name and is named described file for file server end, invisible to user.
Step 3: load-balanced server is that client distributes one for preserving the file server of upload file; The address of described file being preserved to name NFS and file server returns to client.
Step 4: client, according to described address access file server, sends to file server by file and a described file preservation name NFS.
Step 5: file server is store path of described file allocation according to self file storage directory, is kept at described file under described store path, and file described in file RNTO is preserved to a name NFS.
Step 6: file has been uploaded rear file server and returned to load-balanced server and the store path of file and a file preservation name NFS are returned to load-balanced server uploading successful message.
Step 7: load-balanced server is combined file server address, file store path and file preservation name to be encrypted coding, obtain a character string, and the file access title NFC(Name of File in Client using described character string as upload file) return to client.In other embodiments, the file server address of load-balanced server by original attribute, distribution and store path, these three groups of information of the storage name NFS in file server are respectively as three information source S1, S2 and the S3 of scrambled, S1, S2 and S3 are encrypted respectively to coding, obtain three sections of intermediate character string M1, the M2 and the M3 that by numeral and letter, are formed, again M1, M2 and M3 being combined to unified scrambled becomes the file access title NFC that finally returns to client, encrypts and uses existing des encryption algorithm to realize.
Step 8: client shows described file access title NFC to user.
In other embodiments, described file preservation name also comprises current time.Described file attribute at least comprises file type and file size.
Described file server is divided into many groups, and each file server group has some file servers, is positioned at the file server data backup each other of same group.So-called data backup each other, that is to say and be positioned at the file server of same group regularly to the file of himself storage of alternative document server sync on the same group, and the file on the file server in so is on the same group same, backups each other.
So in the present embodiment, in step 7, load-balanced server is combined file server grouping information, file server address, file store path and file preservation name to be encrypted coding, obtain a character string, and the file access title using described character string as upload file returns to client.
As Fig. 3, in another embodiment, file server is stored the file of client upload according to three layers of tree-like file structure.First under the file storage location of file server, set up 100 files (ground floor) of numbering from 00 to 99, then under each file, set up again respectively 100 files (second layer) of from 00 to 99, under interior file in the second layer, set up again 100 files (the 3rd layer) of 00 to 99, form 1,000,000 root folders, each root folder (being the file of the bottom) is lower can store at least one file, according to the overall load situation of file server, determines.
Whether the directory tree under file server detects it and exists when file server brings into operation, and illustrates that this document server is to move for the first time to take file storage service, sets up three layers of file directory in a manner mentioned above if do not exist.If exist, in journal file the storage condition of reading folder in internal memory, as the foundation of its file storage directory service condition of monitoring, to be from the suitable store path of the file allocation of client upload according to it.
As Fig. 2, a specific embodiment of the ff method of distributed file system provided by the invention comprises:
Step 1: client is initiated access request and file access title NFC is sent to load-balanced server to load-balanced server; More specifically, access request is the requests such as file download, renewal.
Step 2: load-balanced server resolution file access title NFC obtains file server address, file store path and the file at file place and preserves name; And described file server address, file store path and file preservation name are returned to client.More specifically, first load-balanced server resolves to file access title NFC three independently intermediate character string M1, M2 and M3, resolve respectively again three intermediate character strings, acquisition comprises three groups of information S1, S2 and the S3 of file name NFS in source file attribute, file server address and store path, file server, and these information are returned to client.
Step 3: client is preserved name according to file server address, file store path and file and find corresponding file on file server.Further complete file and download, check and upgrade operation.
The embodiment that has grouping for file server, also comprises the grouping information of file server in the information that in above-mentioned steps 2, load-balanced server resolution file access title NFC obtains.
The present invention is not limited to aforesaid embodiment.The present invention expands to any new feature or any new combination disclosing in this manual, and the arbitrary new method disclosing or step or any new combination of process.
Claims (10)
1. a file uploading method for distributed file system, described distributed file system comprises some clients, at least one load-balanced server and some file servers; Load-balanced server has the state information of All Files server; Described load-balanced server is for receiving the access request of client, and is assigned as the file server that it provides file service according to the state information of file server for client;
It is characterized in that, comprising:
Step 1: client is sent file upload request to load-balanced server;
Step 2: load-balanced server receives described file upload request, for preserving name by file of the file generated of uploading;
Step 3: load-balanced server is that client distributes one for preserving the file server of upload file; The address of described file being preserved to name and file server returns to client;
Step 4: client, according to described address access file server, sends to file server by file and described file preservation name;
Step 5: file server is store path of described file allocation according to self file storage directory, is kept at described file under described store path, and file described in file RNTO is preserved to name;
Step 6: file server returns to load-balanced server and the store path of file and file preservation name are returned to load-balanced server uploading successful message;
Step 7: load-balanced server is combined file server address, file store path and file preservation name to be encrypted coding, obtain a character string, and the file access title using described character string as upload file returns to client;
Step 8: client shows described file access title to user.
2. the file uploading method of a kind of distributed file system according to claim 1, is characterized in that, described file is preserved name and comprised current time and file attribute; Described file attribute at least comprises file type and file size.
3. the file uploading method of a kind of distributed file system according to claim 1, is characterized in that, described file server is divided into many groups, and each file server group has some file servers, is positioned at the file server data backup each other of same group.
4. the file uploading method of a kind of distributed system according to claim 3, it is characterized in that, described step 7 is: load-balanced server is combined file server grouping information, file server address, file store path and file preservation name to be encrypted coding, obtain a character string, and the file access title using described character string as upload file returns to client.
5. the file uploading method of a kind of distributed system according to claim 1, is characterized in that, in file server, the file of client upload is according to three layers of tree-like file structure storage.
6. a ff method for distributed file system, described distributed file system comprises some clients, at least one load-balanced server and some file servers; Load-balanced server has the state information of All Files server; Described load-balanced server is for receiving the access request of client, and is assigned as the file server that it provides file service according to the state information of file server for client;
It is characterized in that, comprising:
Step 1: client is initiated access request and file access title is sent to load-balanced server to load-balanced server;
Step 2: load-balanced server resolution file access title obtains file server address, file store path and the file at file place and preserves name; And described file server address, file store path and file preservation name are returned to client;
Step 3: client is preserved name according to file server address, file store path and file and find corresponding file on file server.
7. the ff method of a kind of distributed file system according to claim 6, is characterized in that, described file is preserved name and comprised current time and file attribute; Described file attribute at least comprises file type and file size.
8. the ff method of a kind of distributed file system according to claim 6, is characterized in that, described file server is divided into many groups, and each file server group has some file servers, is positioned at the file server data backup each other of same group.
9. the ff method of a kind of distributed system according to claim 8, it is characterized in that, described file access name is called load-balanced server file server grouping information, file server path, file store path and file preservation name is combined and be encrypted the character string that coding obtains.
10. the file uploading method of a kind of distributed system according to claim 6, is characterized in that, in file server, the file of client upload is according to three layers of tree-like file structure storage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410425307.6A CN104202384A (en) | 2014-08-27 | 2014-08-27 | File uploading and searching method of distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410425307.6A CN104202384A (en) | 2014-08-27 | 2014-08-27 | File uploading and searching method of distributed file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104202384A true CN104202384A (en) | 2014-12-10 |
Family
ID=52087609
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410425307.6A Pending CN104202384A (en) | 2014-08-27 | 2014-08-27 | File uploading and searching method of distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104202384A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104754031A (en) * | 2015-02-15 | 2015-07-01 | 四川长虹电器股份有限公司 | Method for uploading file and air conditioner |
| CN104794188A (en) * | 2015-04-15 | 2015-07-22 | 努比亚技术有限公司 | Document positioning method and device |
| CN105227981A (en) * | 2015-07-07 | 2016-01-06 | 深圳市美贝壳科技有限公司 | A kind of by method that is aobvious for picture projection and that preserve |
| CN105357584A (en) * | 2015-12-11 | 2016-02-24 | Tcl集团股份有限公司 | User behavior log classification uploading method and system based on intelligent television |
| CN105763604A (en) * | 2016-02-04 | 2016-07-13 | 四川长虹电器股份有限公司 | Lightweight distributed file system and method for recovering original name of downloaded file |
| CN105812427A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Methods and devices for file uploading and downloading, and file server |
| CN106506613A (en) * | 2016-10-31 | 2017-03-15 | 大唐高鸿信安（浙江）信息科技有限公司 | The data storage location encryption method of distributed key value storage systems |
| CN107040503A (en) * | 2016-02-04 | 2017-08-11 | 福建天晴数码有限公司 | Automatic login of client side method and system |
| CN107040500A (en) * | 2016-02-04 | 2017-08-11 | 福建天晴数码有限公司 | Service end generates the method and system of automatic login of client side information |
| CN107040502A (en) * | 2016-02-04 | 2017-08-11 | 福建天晴数码有限公司 | The method and system of automated log on after client software is downloaded |
| CN107426271A (en) * | 2017-03-21 | 2017-12-01 | 咪咕文化科技有限公司 | Data processing method and system in a kind of server |
| CN108093207A (en) * | 2016-11-23 | 2018-05-29 | 杭州海康威视数字技术股份有限公司 | A kind of data storage, acquisition methods and device |
| CN109743405A (en) * | 2019-02-20 | 2019-05-10 | 高新兴科技集团股份有限公司 | Load balancing file uploading method, system, computer storage medium and equipment |
| CN110377579A (en) * | 2019-07-24 | 2019-10-25 | 南京中孚信息技术有限公司 | File memory method, device and server |
| CN111225032A (en) * | 2019-12-19 | 2020-06-02 | 福建升腾资讯有限公司 | Method, system, device and medium for separating application service and file service |
| CN112131202A (en) * | 2020-10-10 | 2020-12-25 | 厦门悦讯信息科技股份有限公司 | Distributed file storage and reading method, terminal device and storage medium |
| CN112579525A (en) * | 2019-09-30 | 2021-03-30 | 成都长虹网络科技有限责任公司 | WEB-based unified file processing method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088491A (en) * | 2011-02-01 | 2011-06-08 | 西安建筑科技大学 | Distributed storage oriented cloud storage security architecture and data access method thereof |
| CN102404399A (en) * | 2011-11-18 | 2012-04-04 | 浪潮电子信息产业股份有限公司 | Fuzzy dynamic allocation method for cloud storage resource |
| US20120226773A1 (en) * | 2011-03-03 | 2012-09-06 | Hiromi Tsuda | Image reading apparatus, image reading method, and computer program product |
| CN102664914A (en) * | 2012-03-22 | 2012-09-12 | 北京英孚斯迈特信息技术有限公司 | IS/DFS-Image distributed file storage query system |
| CN103020144A (en) * | 2012-11-21 | 2013-04-03 | 用友软件股份有限公司 | File management system and method |
-
2014
- 2014-08-27 CN CN201410425307.6A patent/CN104202384A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088491A (en) * | 2011-02-01 | 2011-06-08 | 西安建筑科技大学 | Distributed storage oriented cloud storage security architecture and data access method thereof |
| US20120226773A1 (en) * | 2011-03-03 | 2012-09-06 | Hiromi Tsuda | Image reading apparatus, image reading method, and computer program product |
| CN102404399A (en) * | 2011-11-18 | 2012-04-04 | 浪潮电子信息产业股份有限公司 | Fuzzy dynamic allocation method for cloud storage resource |
| CN102664914A (en) * | 2012-03-22 | 2012-09-12 | 北京英孚斯迈特信息技术有限公司 | IS/DFS-Image distributed file storage query system |
| CN103020144A (en) * | 2012-11-21 | 2013-04-03 | 用友软件股份有限公司 | File management system and method |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105812427B (en) * | 2014-12-31 | 2019-08-13 | 中国电信股份有限公司 | File uploads and method for down loading, device and file server |
| CN105812427A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Methods and devices for file uploading and downloading, and file server |
| CN104754031A (en) * | 2015-02-15 | 2015-07-01 | 四川长虹电器股份有限公司 | Method for uploading file and air conditioner |
| CN104794188B (en) * | 2015-04-15 | 2020-05-05 | 努比亚技术有限公司 | File positioning method and device |
| CN104794188A (en) * | 2015-04-15 | 2015-07-22 | 努比亚技术有限公司 | Document positioning method and device |
| CN105227981A (en) * | 2015-07-07 | 2016-01-06 | 深圳市美贝壳科技有限公司 | A kind of by method that is aobvious for picture projection and that preserve |
| CN105357584A (en) * | 2015-12-11 | 2016-02-24 | Tcl集团股份有限公司 | User behavior log classification uploading method and system based on intelligent television |
| CN105357584B (en) * | 2015-12-11 | 2020-09-22 | Tcl科技集团股份有限公司 | User behavior log classification uploading method and system based on smart television |
| CN105763604B (en) * | 2016-02-04 | 2019-03-19 | 四川长虹电器股份有限公司 | Lightweight distributed file system and the method for restoring downloading file original name |
| CN107040500A (en) * | 2016-02-04 | 2017-08-11 | 福建天晴数码有限公司 | Service end generates the method and system of automatic login of client side information |
| CN107040502A (en) * | 2016-02-04 | 2017-08-11 | 福建天晴数码有限公司 | The method and system of automated log on after client software is downloaded |
| CN107040503A (en) * | 2016-02-04 | 2017-08-11 | 福建天晴数码有限公司 | Automatic login of client side method and system |
| CN105763604A (en) * | 2016-02-04 | 2016-07-13 | 四川长虹电器股份有限公司 | Lightweight distributed file system and method for recovering original name of downloaded file |
| CN106506613B (en) * | 2016-10-31 | 2018-04-13 | 大唐高鸿信安（浙江）信息科技有限公司 | The data storage location encryption method of distributed key value storage systems |
| CN106506613A (en) * | 2016-10-31 | 2017-03-15 | 大唐高鸿信安（浙江）信息科技有限公司 | The data storage location encryption method of distributed key value storage systems |
| CN108093207A (en) * | 2016-11-23 | 2018-05-29 | 杭州海康威视数字技术股份有限公司 | A kind of data storage, acquisition methods and device |
| CN107426271A (en) * | 2017-03-21 | 2017-12-01 | 咪咕文化科技有限公司 | Data processing method and system in a kind of server |
| CN109743405A (en) * | 2019-02-20 | 2019-05-10 | 高新兴科技集团股份有限公司 | Load balancing file uploading method, system, computer storage medium and equipment |
| CN110377579A (en) * | 2019-07-24 | 2019-10-25 | 南京中孚信息技术有限公司 | File memory method, device and server |
| CN112579525A (en) * | 2019-09-30 | 2021-03-30 | 成都长虹网络科技有限责任公司 | WEB-based unified file processing method and system |
| CN111225032A (en) * | 2019-12-19 | 2020-06-02 | 福建升腾资讯有限公司 | Method, system, device and medium for separating application service and file service |
| CN112131202A (en) * | 2020-10-10 | 2020-12-25 | 厦门悦讯信息科技股份有限公司 | Distributed file storage and reading method, terminal device and storage medium |
| CN112131202B (en) * | 2020-10-10 | 2022-08-16 | 厦门悦讯信息科技股份有限公司 | Distributed file storage and reading method, terminal device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104202384A (en) | File uploading and searching method of distributed file system | |
| CN101692239B (en) | Method for distributing metadata of distributed type file system | |
| CN105550371A (en) | Big data environment oriented metadata organization method and system | |
| CN104123359B (en) | Resource management method of distributed object storage system | |
| US9967298B2 (en) | Appending to files via server-side chunking and manifest manipulation | |
| US9405776B2 (en) | Remote backup and restore | |
| JP6479020B2 (en) | Hierarchical chunking of objects in a distributed storage system | |
| US9183213B2 (en) | Indirection objects in a cloud storage system | |
| US8548957B2 (en) | Method and system for recovering missing information at a computing device using a distributed virtual file system | |
| CN106066896B (en) | Application-aware big data deduplication storage system and method | |
| CN103327052B (en) | Date storage method and system and data access method and system | |
| CN106156359A (en) | A kind of data synchronization updating method under cloud computing platform | |
| CN105025053A (en) | Distributed file upload method based on cloud storage technology and system | |
| CN104239511B (en) | A kind of user's space file system implementation method towards MongoDB | |
| CN103888499A (en) | Distributed object processing method and system | |
| CN109522283A (en) | A kind of data de-duplication method and system | |
| CN103198153A (en) | Metadata clustering management method and module applied to distributed file system | |
| CN103473337A (en) | Massive catalogs and files oriented processing method in distributed type storage system | |
| CN106960011A (en) | Metadata of distributed type file system management system and method | |
| CN102523301A (en) | Method for caching data on client in cloud storage | |
| CN103902577A (en) | Method and system for searching and locating resources | |
| CN111782150B (en) | Multi-bucket storage system and method based on object storage | |
| CN104537081A (en) | File management system and method | |
| KR101140484B1 (en) | File management system and file management method for applying hierarchy file system using flat file system | |
| Junping | Analysis of key technologies of distributed file system based on big data [J] |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20141210 |