CN105005611B - A kind of file management system and file management method - Google Patents
A kind of file management system and file management method Download PDFInfo
- Publication number
- CN105005611B CN105005611B CN201510401914.3A CN201510401914A CN105005611B CN 105005611 B CN105005611 B CN 105005611B CN 201510401914 A CN201510401914 A CN 201510401914A CN 105005611 B CN105005611 B CN 105005611B
- Authority
- CN
- China
- Prior art keywords
- file
- data
- name
- server
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to Internet technical field, a kind of file management system is made of client, main control server and data server, and three passes through network each other and communicates, wherein：The client is used to send to the main control server and request, and receives the data information that the main control server or the data server are sent；The request or data that the main control server is used to receive the client and the data server is sent, and content carries out control processing based on the received, while the data server is managed and being safeguarded；The data server is for storing the data sent by the client or the main control server.File management system proposed by the present invention, the small documents of magnanimity can be more effectively managed, file needed for capable of quickly and effectively finding user, and can be when small documents quantity increases, the memory space for simply and effectively increasing system, without modifying to system architecture.
Description
【Technical field】
The present invention relates to Internet technical fields, and in particular to one of internet file management system and file management
Method.
【Background technique】
In existing Internet technical field, user has higher requirement to file management system, in file management
In system, how to store and how rapidly to be called and be emphasis therein.Demand of the user to storage is mainly reflected in three
A aspect：
（1）All data must reliably, cannot absolutely be lost.This is tolerance baseline of the user for storage, because
This reliability is placed in center-stage.
（2）Under the premise of meeting reliability, operation or online storage system are needed with high-performance.This is because with
The business datum at family can be almost stored entirely in data server, and high-performance can be convenient the rapid response speed of user.
（3）Offline or Backup Data needs high capacity, low price.This partial data amount commonly is very big, but wants to performance
It asks not high, is infrequently used especially for those but the data that must save, user are not intended to bear great number cost thus.
Traditional network store system stores all data using the storage server concentrated, and storage server becomes system
The bottleneck and reliability of performance and the focus of safety are not able to satisfy the needs of Mass storage application.
Currently, file management system in the prior art mostly uses distributed file system（Distributed File
System）, distributed file system refers to that the physical memory resources of file system management are not necessarily connected directly between local node
On, but be connected by computer network with node.Distributed file system uses expansible system structure, is deposited using more
Storage server shares storage load, positions storage information using location server, it not only increases the reliability of system, can be used
Property and access efficiency, are also easy to extend.
With the development of internet technology, the concept of cloud storage is gradually received by user, and cloud storage gets rid of single storage
How the bottleneck of node still can provide reasonable frame and effective design method toward (scale-out) extending transversely,
High-performance, high scalable, High Availabitity file storage service are established, is still pendulum is rich in challenge in face of system designer
Task.
【Summary of the invention】
In view of the above problems, propose the present invention, overcome the above problem or at least be partially solved in order to provide one kind on
State a kind of file management system and file management method of problem.
As one aspect of the present invention, a kind of file management system, by client, main control server and data server
Composition, three pass through network each other and communicate, wherein：
The client is used to send to the main control server and request, and receives the main control server or the number
The data information sent according to server；
The request or data that the main control server is used to receive the client and the data server is sent, and root
Control processing is carried out according to received content, while the data server is managed and is safeguarded；
The data server is for storing the data sent by the client or the main control server.
Further, the main control server adds file group name to the filename that the client intends storage file, and
Data server corresponding with file group name is distributed to the file of quasi- storage to store.
Further, the file group name can be the name of assigned data server, be also possible to master control service
Device is added to before the filename according to certain rule or the name of sequence definition, the file group name.
When further, using the name of data server as file group name, a file group name can represent a number
According to the segment space in library, it is also possible to the combination of a data server or several data servers；With master control service
It, can be using same type of file as a group, different types of file conduct when the name that device defines is as file group name
Different groups.
Further, the client is intended the file group name and this document group name pair of storage file by the main control server
The data server name answered is sent to the client, and the file group name is added on filename by the client, and should
File is sent to the corresponding data server and is stored.
Further, the client includes interface module, memory module and security module, wherein：The interface module
It is responsible for user and the interface of access data is provided；The memory module is responsible for storing the file of client, or client is used
It is cached in the metamessage of file access；The security module is responsible for encrypting data, guarantees the safety of data；
The main control server includes active and standby disaster tolerance module, name index module, data server management module and clothes
Business scheduler module, wherein：The active and standby disaster tolerance module guarantees to take over its work in main control server node failure；The name
Index module is responsible for establishing name index catalogue, the mapping relations of storage file name and corresponding data server；The data
Server management module manages the state of all data servers, and executes load balancing plan；The service dispatch mould
Block is responsible for receiving the request from client and data server；
The data server includes data memory module, replica management module, name search module and state-maintenance mould
Block, wherein：The data memory module is responsible for file in the storage for locally carrying out persistence；The replica management module will be literary
Part stores on one or more copies to data server；The name search module is responsible for in the data memory module
File establishes name search catalogue；The state-maintenance module is responsible for periodically that the status information of oneself is all in a manner of heartbeat packet
The report of phase property is to main control server.
Further, the client further includes name module, the file group name for sending the main control server
Addition is in filename.
According to another aspect of the present invention, a kind of method of file management system storage file is provided, specially：
Client sends the file to main control server；
Main control server adds file group name on the filename of this document, and according to file group star's this document of addition
It is stored in corresponding file group；
Main control server is the data server of this document group distribution storage, and establishes name index catalogue；
Main control server will be in the assigned data server of the total data deposit of this document group；
Data server establishes name search catalogue.
According to another aspect of the present invention, a kind of method that file management system reads file is provided, specially：
Client will need the filename read to be sent to main control server；
Main control server searched in name index catalogue the corresponding file group name of file name and with this document group name
Corresponding data server name, and file read request transmission is given to corresponding data server；
Corresponding data server retrieves the filename containing this document group name in its name search catalogue and should
The storage address of file and/or its copy in data server, and transfer file from corresponding storage address and be sent to master
Control server；
The file received is sent to client by main control server.
According to another aspect of the present invention, a kind of method of file management system storage file is also provided, specially：
File storage request is sent to main control server by client；
File group name and the data server name that should be stored are sent to client by main control server, and establish name index
Catalogue；
File group name is added on filename to be stored by client, and file to be stored is sent to corresponding name
Data server；
The data server of corresponding name stores file, and establishes name search catalogue.
According to another aspect of the present invention, a kind of method that file management system reads file is also provided, specially：
Filename containing file group name is sent to main control server by client；
Main control server indexes the corresponding data server name of this document group name according to name index catalogue, and will contain
There is the filename of this document group name to be sent to corresponding data server；
Corresponding data server retrieves filename and this article containing this document group name according to name search catalogue
The storage address of part and/or its copy in data server, and transfer file from corresponding storage address and be sent to client
End.
Further, the name index catalogue include file group name, the data server name for storing this group of file,
And/or store the data server name of this group of duplicate of the document；The name search catalogue includes the file containing file group name
Name, the storage address of file and/or its copy in data server.
File management system proposed by the present invention can more effectively manage the small documents of magnanimity, can quickly and effectively look into
File needed for finding user, and can simply and effectively increase the memory space of system when small documents quantity increases, without
It needs to modify to system architecture.The present invention, which solves mass small documents storage in the prior art, can waste asking for disk space
Topic and lookup time are continuously increased with the increase of quantity of documents, and then the problem of influence user experience.
【Detailed description of the invention】
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Attached drawing be briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill of field, without any creative labor, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is the structural schematic diagram of the file management system of one embodiment of the invention.
Fig. 2 is the flow diagram of the file memory method of one embodiment of the invention.
Fig. 3 is the flow diagram of the file reading of one embodiment of the present of invention.
Fig. 4 is the flow diagram of the file memory method of another embodiment of the present invention.
Fig. 5 is the flow diagram of the file reading of another embodiment of the invention.
【Specific embodiment】
Exemplary embodiment of the present invention is described in more detail below with reference to accompanying drawings.Although showing the present invention in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the present invention, without should be by embodiments set forth here
It is limited.On the contrary, provide these embodiments be in order to the more thorough explanation present invention, and can be complete by the scope of the present invention
Whole is communicated to those skilled in the art.
Fig. 1 shows file management system structural schematic diagram according to an embodiment of the invention.As shown in Figure 1, a kind of
File management system is made of client 10, main control server 20 and data server 30, is led to each other by network
Letter.Wherein：
Client 10 is used to send to main control server 20 and request, and receives main control server 20 or data server 30
The data information sent.Client 10 includes various desktop computers, laptop, intelligent terminal（Such as smart phone, plate
Computer, Intelligent bracelet, intelligent glasses）Etc. equipment.Client 10 includes that data storage is asked to the request that main control server 20 is sent
It asks, data inquiry request, data read request, data download request or data modification request etc..
The request or data that main control server 20 is used to receive client 10 and data server 30 is sent, and according to reception
Content carry out control processing.Specifically, working as client（User）When needing to carry out data storage, responsible pair of main control server 20
Client 10 intends the filename addition file group name of storage file, and distributes the file of quasi- storage corresponding with file group name
Data server is stored, and is exactly the file that client 10 sends quasi- storage to main control server 20, master control clothes briefly
After device 20 be engaged in the file addition file group name of quasi- storage, to one or more 30 turns of data servers corresponding with file group name
Send out this document；When client 10（User）When needing inquiry, reading or downloading data, main control server 20 needs to look into according to user
It askes, read or the filename of downloading, retrieve its file group name and the corresponding data server of file group name, and from corresponding number
According to transferring corresponding data in server and be fed back to client 10；Main control server 20 be also used to data server 30 into
Row manages and maintains.
Specifically, main control server wants the filename of storage file to add file group name client wants, and according to file
Group name distributes data server corresponding with this document group name and carries out data storage, and file group name can be number, can be
Letter can also be the combination of number with letter, and file group name can be named at random, can also intend storage file according to client
Suffix distributes certain a kind of file group name, for example what main control server judged that client will store is suffix for " .jpg " format
Picture group " 1.jpg, 2.jpg, 3.jpg ... ", then be this group of image data unified distribution file group name " 001_ ", text in this way
Part name can be renamed into " 001_1.jpg, 001_2.jpg, 001_3.jpg ... ", it is assumed that the number of file group entitled " 001_ "
According to all corresponding with data server A, then this group of picture after adding file group name is stored in data server A.In order to mention
Height storage and subsequent reading speed, file group name are preferably added at before the filename of file, can be in a file group name
Multiple files of the storage with group.File group name can be the name of assigned data server, be also possible to main control server
According to certain rule or the name of sequence definition.When using the name of data server as file group name, a file group name can
To represent the segment space in a database, it is also possible to the group of a data server or several data servers
It closes, preferably a group name represents a data server.When using the name that main control server defines as file group name, it can incite somebody to action
Same type of file is as a group, and different types of file is as different groups, for example suffix is that " .jpg " format is
One group, suffix is that " .doc " is another set ....
In order to guarantee the reliability of data, main control server is made of primary server and backup server, primary server work
It when making normal, is mainly responsible for and the data received is stored and controlled, storage can be interim storage, be also possible to permanent
Storage；Backup server is responsible for Backup Data, and substitutes it in primary server failure and work.
Data server 30 is for storing the data sent by client 10 or main control server 20.Data server 30
It can be one, be also possible to multiple, multiple data servers can be stored in different physical locations respectively, but its dispersion
Physical location has no effect on its United Dispatching for receiving main control server as a whole, certainly, if local sufficiently large, multiple numbers
It can also be placed in a physical location according to server.Data server can be stored according to file to be needed to increase or decrease, no
Whole system framework can be had an impact.
Client 10, main control server 20 and 30 three of data server pass through network each other and communicate, such as
Data transmitting is carried out by internet, 2G/3G/4G network, WIFI network, each local area network etc..
File management system of the invention can be used in the mass small documents environment towards cloud storage, and ensure file
The safety and confidentiality of storage.
As another embodiment of the invention, for client as the effect of data server, main control server can be with
There is different control principles.Specifically, working as client（User）When needing to carry out file storage, main control server is responsible for client
It holds the filename of quasi- storage file to add file group name, and the name after file group name will be added and this document group name is corresponding
The address of data server sends jointly to client, and the file after renaming is sent to corresponding number by address by client
According to server, data server stores this file, and storage result is fed back to control server；Work as client（With
Family）When needing to inquire, reading or download file, the filename downloaded will be needed to be sent to main control server, main control server root
Corresponding file group name is found according to filename, and then finds the corresponding data server of file group name, and order the data service
Corresponding data are sent to client by device；Main control server is also used to be managed data server and safeguard.
As the improvement of above two embodiment, the storage method to data and the inquiry, reading, method for down loading to data
It can be bonded to each other, for example be that data are transmitted directly to corresponding data server by client when data storage, in data
When inquiry, reading or downloading, the data on data server are sent to client eventually by control server；Alternatively, in number
When according to storage, the data of client complete the storage on corresponding data server by control server, in data query, read
When taking or downloading, the corresponding data on data server is transmitted directly to client.
As the improvement of above two embodiment, the process of file renaming can also be completed in client.Master control clothes
Device be engaged in client transmission file group name, file group name is added on filename, is preferably added at before filename by client,
As filename prefix.
The present invention can be used for the mass small documents management system towards cloud storage, its distributed column storage based on cloud storage
Data dispersion is stored in more independent equipment by system.Technical solution of the present invention makes when handling mass small documents,
It occupies little space for the single file of small documents storage, but the feature that quantity of documents is huge, using distributed storage technology,
Again planning small documents storage and search when NameSpace, the time needed for reducing locating file, make client not because
The increase of quantity of documents and take more time waiting, the user experience is improved.
The client 10 of the file management system of one embodiment of the invention, 20 sum number of main control server is also shown in Fig. 1
According to the schematic diagram of internal structure of server 30.As shown in the figure：
Client 10 includes interface module 11, memory module 12 and security module 13, wherein：
Interface module 11 is responsible for user and provides the interface of access data, such as posix interface.
Memory module 12 is responsible for the file of storage client, or the metamessage that client is used for file access is delayed
It deposits, is such as buffered in client local or disk, distal end can also be buffered in.
Security module 13 is responsible for encrypting data, guarantees the safety of data.
Main control server 20 include active and standby disaster tolerance module 21, name index module 22, data server management module 23, with
And service dispatch module 24, wherein：
Active and standby disaster tolerance module 21 is the single-point problem in order to avoid main control server, it will usually configure active service for it
Device, to guarantee to take over its work in main control server node failure.Active and standby disaster tolerance module 21 is also backup server.
Name index module 22 is responsible for establishing name index catalogue, and storage file name is reflected with corresponding data server
Penetrate relationship.Name index catalogue includes file group name, the filename containing file group name, the data server for storing this group of file
Name, and/or the data server name for storing this group of duplicate of the document.
Data server management module 23 manages the state of all data servers, and executes load balancing plan.
Main control server sends file group name to client, or adds file group name to filename, is all by data server pipe
Manage what module 23 was realized.Main control server manages data server concentratedly in the form of heartbeat packet.Receiving client storage
When request, main control server is needed according to each data server group（If the corresponding storable data clothes of each file group name
Business device is more than one）The information such as load select one group of data server to service for it；When main control server hair available data clothes
It when device delay machine of being engaged in, needs to execute duplication plan to the insufficient file of some number of copies, when there is new data server that cluster is added
Or some data server load too high, main control server, which also can according to need, executes copy migration plan.In distribution number
When according to server, data server can be sorted according to the distance with client, according to the information of client so that client
Preferentially the data server close from oneself is chosen to be accessed.
Service dispatch module 24 is responsible for receiving the request from client and data server.Usually by individual thread
Receive request, and adds it in task queue, and the thread in thread pool then constantly takes out task from task queue
It is handled.The request of client includes needing to store data, reading, inquiring, downloading, modifying, data server
Request include to control server timing or not timing send data server status information and to the sound of main control server
Answer information.
Data server 30 includes data memory module 31, replica management module 32, name search module 33 and state dimension
Protect module 34.
Data memory module 31 is responsible for file in the storage for locally carrying out persistence.
Replica management module 32 is the safety in order to guarantee data, by the file in distributed file system, storage one
On a or multiple copies to data server.
Name search module 33 is responsible for establishing name search catalogue to the file in data memory module 31.Name search mesh
Record includes the storage address of the filename containing file group name, file and/or its copy in data server.It in this way can pole
The utilization rate of big raising memory space, and time needed for capable of reducing client locating file, promote user experience.
State-maintenance module 34 is responsible for periodically periodically reporting the status information of oneself to master in a manner of heartbeat packet
Server is controlled, main control server is made to know whether oneself works normally.Usual heartbeat packet can also be current comprising data server
Loading condition, such as CPU, memory, disk I/O, disk storage space, process resource, network I/O etc..
According to above-described embodiment as can be seen that the file consolidation pipe that the present invention will store on nodes all in server cluster
Reason and storage.These nodes include main control server name index module and data server name search words module, in facing cloud
Metadata Service is provided inside the mass small documents management system of storage；Many data servers provide storage service.It is stored in
File in mass small documents management system towards cloud storage is divided into group, these groups are then copied to one or more numbers
According in server, this differs widely with traditional RAID framework.Data server can be determined according to the size of file and be stored
Space needed for file.The name search module of data server is responsible for the real physical location of storage file group, master control service
The name index module monitors of device are there are the file operation on nodes all in data server cluster, such as document creation, delete
Remove, move, rename etc..
As another embodiment of the invention, the method for filename addition file group name can be completed in client,
Therefore, client 10 further includes name module 14, for adding the file group name distributed by main control server to filename.Master control
Server can distribute file group name to the file of the quasi- storage of client, and this document group name is sent to client, client
Name module 14 the file group name received is added before filename or any position of filename, realize to filename
Renaming, preferably say file group name addition before filename.
The name index module 22 of the main control server 20 of mass small documents management system towards cloud storage is deposited in determination
After storing up file to the mapping relations of data server 30, it is grouped according to file group star's file（Such as according to beginning letter point
Group, or be grouped according to the value of the byte of beginning）, and according to group result by file and its copy according to file group name
It is assigned in corresponding data server, each group of correspondence one or more data server, or corresponding individual data clothes
A part of memory space of business device.Name index module 22 only saves group name after grouping where file and corresponding with the group name
Data server name.This group of file and its copy will be respectively stored in data server corresponding with the group name, in each group
File storage physical location information will be stored in the name search module 33 of corresponding data server, client access text
When part, main control server need to only find group name belonging to this document, and the data service of this group of file of storage is found further according to group name
Device name searches the physical storage address of respective file then in the name search module 33 of the data server, and then from phase
Corresponding document is read in the physical storage locations answered, this will undoubtedly save a large amount of lookup time, as long as also, adding new group
Name and data server corresponding with the group name, so that it may which the memory space for increasing whole system with minimum cost makes system
Become more easily to extend.
One mass small documents management system towards cloud storage is taken by least two main control servers and a large amount of data
Business device composition, and allow multiple client while accessing.
As another embodiment of the invention, client 10 is added by file of the security module 13 to quasi- storage
It is close, and the filename of quasi- storage is passed through into the service dispatch module 24 that interface module 11 is sent to main control server 20, service is adjusted
It spends module 24 and the filename received is sent to data server management module 23, server management module 23 is according to filename
Suffix judges file type, and file group name corresponding with the type file addition will be added file before filename
Filename after group name is sent to name index module 22, and name index module 22 should be stored according to file group name locating file
Corresponding data server name, and the filename containing file group name and corresponding data server name are sent to service dispatch mould
The filename containing file group name received and corresponding data server name are sent to client by block 24, service dispatch module 24
The file of quasi- storage containing file group name is sent to corresponding data server by end, client, and data server will receive
File is stored in data memory module 31, and the copy for making file is stored in replica management module 32, while will be contained
The filename of file group name and the specific storage address information in memory module 31 are sent to name search module 33, name search
Module 33 establishes name search catalogue to facilitate client or main control server is subsequent is called to file.
The mass small documents management system towards cloud storage of above-described embodiment is deposited using the distributed column based on cloud storage
Data dispersion is exactly stored in more independent equipment by storage system.Traditional network store system is using the storage concentrated
All data of server repository, storage server become the bottleneck of system performance and the focus of reliability and safety, cannot
Meet the needs of Mass storage application.Distributed network storage system uses expansible system structure, is stored using more
Server shares storage load, positions storage information using location server, it not only increases the reliability of system, availability
And access efficiency, it is also easy to extend.Main control server and data server to be per family it is transparent, user need to only receive to come from
File is sent and is stored on filename by the file group name of main control server, it is not necessary to take notice of the specific position of file storage
It sets.Mass small documents management system towards cloud storage will ensure that the safety and secrecy of file storage.
Can be seen that from the technical solution of the above embodiment of the present invention can be used for documented by the present invention towards cloud storage
Name index module is arranged on main control server, while name being arranged on data server for mass small documents management system
Retrieval module, the name index module of main control server are mainly responsible for the file grouping to client submission and distribute to corresponding
The name search module of data server storage, data server establishes index to the file stored, facilitates user to search, makes
The memory space of main control server, the more flexible file size according in data server point can more be saved by obtaining storage system
With memory space, the more effectively small documents of management magnanimity, file needed for more rapidly and effectively finding user, and can
When small documents quantity increases, the memory space for simply and effectively increasing system can without modifying to system architecture
With with low cost storage server realize (scale-out) extending transversely reach high performance requirement, storage capacity cannot
When meeting the requirements, can be improved by being continuously increased memory node, system storage capacity and I/O parallel ability can at any time without
Seam upgrading, this is that conventional store is not accomplished.
The present invention also provides a kind of file management methods, since this document management method is based on above-mentioned file management system,
Its corresponding principle is the same, therefore is repeated no more.
Here is file management system storage of the present invention and a specific embodiment for reading document method.
（1）When client needs storage file, as shown in Figure 2：
Step S210, client send the file to main control server；
Step S220, main control server add file group name on the filename of this document, and according to the file group of addition
Star's this document is stored in corresponding file group.It is preferably added to improve storage and subsequent reading speed, file group name
Before the filename of this document, multiple files with group can be stored in a file group name.File group name can be assigned
Data server name, be also possible to main control server according to certain rule or sequence definition name.With data service
When the name of device is as file group name, a file group name can represent the segment space in a database, be also possible to one
The combination of a data server or several data servers, preferably a group name represent a data server.With master control
It, can be using same type of file as a group, different types of file when the name that server defines is as file group name
As different groups, for example it is one group that suffix, which is " .jpg " format, and suffix is that " .doc " is another set ....
Step S230, main control server are the data server of this document group distribution storage, and establish name index catalogue.
Name index catalogue include file group name, the filename containing file group name, the data server name for storing this group of file,
And/or store the data server name of this group of duplicate of the document.
Step S240, main control server will be in the assigned data servers of the total data deposit of this document group.Data
Server may be copied the storage file received, while the copy generated to duplication also stores.
Step S250, data server establish name search catalogue.Name search catalogue includes the text containing file group name
The storage address of part name, file and/or its copy in data server.
（2）When client needs to read file, as shown in Figure 3：
Step S310, client will need the filename read to be sent to main control server.
Step S320, main control server searched in name index catalogue the corresponding file group name of file name and with this
The corresponding data server name of file group name, and file read request transmission is given to corresponding data server.
Step S330, corresponding data server retrieve the file containing this document group name in its name search catalogue
The storage address of name and this document and/or its copy in data server, and file is transferred from corresponding storage address
It is sent to main control server.It is preferential to send file original, it, will in the data server cisco unity malfunction where file original
Duplicate of the document is sent to main control server.
The file received is sent to client by step S340, main control server.
Here is file management system storage of the present invention and another specific embodiment for reading document method.
（1）When client needs storage file, as shown in Figure 4：
File storage request is sent to main control server by step S410, client.
Step S420, file group name and the data server name that should be stored are sent to client by main control server, and are built
Vertical name index catalogue.Name index catalogue includes file group name, stores the data server name of this group of file, and/or deposit
Store up the data server name of this group of duplicate of the document.
File group name is added on filename to be stored by step S430, client, and file to be stored is sent
To the data server of corresponding name.File group name is preferably added at before the filename of file to be stored.File group name can be with
It is the name of assigned data server, is also possible to main control server according to certain rule or the name of sequence definition.With
When the name of data server is as file group name, a file group name can represent the segment space in a database,
It can be the combination of a data server or several data servers, a preferably group name represents a data service
Device.It, can be using same type of file as a group, inhomogeneity when using the name that main control server defines as file group name
The file of type is as different groups, for example it is one group that suffix, which is " .jpg " format, and suffix is that " .doc " is other one
Group ....
The data server of step S440, corresponding name store file, and establish name search catalogue.Data clothes
Business device may be copied the storage file received, while the copy generated to duplication also stores.Name search catalogue packet
Include the storage address of the filename containing file group name, file and/or its copy in data server.
（2）When client needs to read file, as shown in Figure 5：
Filename containing file group name is sent to main control server by step S510, client.
Step S520, main control server index the corresponding data server of this document group name according to name index catalogue
Name, and the filename containing this document group name is sent to corresponding data server.
Step S530, corresponding data server retrieve the file containing this document group name according to name search catalogue
The storage address of name and this document and/or its copy in data server, and file is transferred from corresponding storage address
It is sent to client.It is preferential to send file original, in the data server cisco unity malfunction where file original, by file
Copy is sent to client.
Mass small documents management system proposed by the present invention towards cloud storage, so that in the storage system towards cloud storage
In, the more effective small documents for managing magnanimity, file needed for more rapidly and effectively finding user, and can be in small documents
When quantity increases, simply and effectively increase the memory space of system, without modifying to system architecture.
As seen through the above description of the embodiments, those skilled in the art can be understood that the present invention can
It realizes by means of software and necessary general hardware platform.Based on this understanding, technical solution of the present invention essence
On in other words the part that contributes to existing technology can be embodied in the form of software products, the computer software product
It can store in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are used so that a computer equipment
（It can be personal computer, server or the network equipment etc.）Execute the certain of each embodiment or embodiment of the invention
Method described in part.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device or
For system embodiment, since it is substantially similar to the method embodiment, so describing fairly simple, related place is referring to method
The part of embodiment illustrates.Apparatus and system embodiment described above is only schematical, wherein the conduct
The unit of separate part description may or may not be physically separated, component shown as a unit can be or
Person may not be physical unit, it can and it is in one place, or may be distributed over multiple network units.It can root
According to actual need that some or all of the modules therein is selected to achieve the purpose of the solution of this embodiment.Ordinary skill
Personnel can understand and implement without creative efforts.
It should be noted that：
Algorithm and display are not inherently related to any particular computer, virtual system, or other device provided herein.
Various general-purpose systems can also be used together with teachings based herein.As described above, it constructs required by this kind of system
Structure be obvious.In addition, the present invention is not also directed to any specific programming language.It should be understood that can use each
Kind programming language realizes invention described herein content.
Those skilled in the art will understand that can to module each in embodiment carry out adaptivity change and
They are arranged in one or more devices different from this embodiment.Unless otherwise being expressly recited, disclosed in this specification
Each feature can be replaced with an alternative feature that provides the same, equivalent, or similar purpose.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors
Software module realize, or be implemented in a combination thereof.
The foregoing is merely the preferred embodiments of the invention, the claims that are not intended to limit the invention.
Simultaneously it is described above, for those skilled in the technology concerned it would be appreciated that and implement, therefore other be based on institute of the present invention
The equivalent change that disclosure is completed, should be included in the covering scope of the claims.
Claims (8)
1. a kind of file management system, which is characterized in that the file management system is taken by client, main control server and data
Business device composition, three pass through network each other and communicate, wherein：
The client is used to send to the main control server and request, and receives the main control server or data clothes
The data information that business device is sent；The client includes interface module, memory module and security module, wherein：The interface mould
Block is responsible for user and provides the interface of access data；The memory module is responsible for storing the file of client, or by client
Metamessage for file access is cached；The security module is responsible for encrypting data, guarantees the safety of data；
The main control server is used to receive the client and request or data that the data server is sent, and according to connecing
The content of receipts carries out control processing, while the data server is managed and being safeguarded；The main control server includes master
Standby disaster tolerance module, name index module, data server management module and service dispatch module, wherein：The active and standby disaster tolerance
Module guarantees to take over its work in main control server node failure；The name index module is responsible for establishing name index mesh
Record, the mapping relations of storage file name and corresponding data server；The data server management module, manages all numbers
According to the state of server, and execute load balancing plan；The service dispatch module is responsible for receiving from client and data
The request of server；
The data server is for storing the data sent by the client or the main control server, the data clothes
Business device includes data memory module, replica management module, name search module and state-maintenance module, wherein：The data are deposited
Storage module is responsible for file in the storage for locally carrying out persistence；The replica management module stores file one or more secondary
In sheet to data server；The name search module is responsible for establishing name search mesh to the file in the data memory module
Record；The state-maintenance module is responsible for periodically periodically reporting the status information of oneself in a manner of heartbeat packet to be taken to master control
Business device.
2. a kind of file management system according to claim 1, it is characterised in that：The main control server is to the client
It holds the filename of quasi- storage file to add file group name, and distributes data corresponding with file group name to the file of quasi- storage and take
Business device is stored.
3. a kind of file management system according to claim 2, it is characterised in that：The file group name can be assigned
Data server name, be also possible to main control server according to certain rule or sequence definition name, the file group
Name is added to before the filename.
4. a kind of file management system according to claim 3, it is characterised in that：Using the name of data server as text
When part group name, a file group name can represent the segment space in a database, be also possible to a data server, or
Person is the combination of several data servers；It, can be by same type when using the name that main control server defines as file group name
File as a group, different types of file is as different groups.
5. a kind of file management system according to claim 1, it is characterised in that：The main control server is by the client
The file group name and the corresponding data server name of this document group name for holding quasi- storage file are sent to the client, the client
The file group name is added on filename by end, and is sent the file to the corresponding data server and stored.
6. a kind of file management system according to claim 1, it is characterised in that：The client further includes name mould
Block, the file group name for sending the main control server add in filename.
7. a kind of file management method, which is characterized in that the file management method is based on being led to by network each other
Client, main control server and the data server of letter and execute, including：
Client sends to main control server and requests, and receives the data that the main control server or the data server are sent
Information；The client includes interface module, memory module and security module, wherein：The interface module is responsible for user and mentions
For accessing the interface of data；The memory module is responsible for storing the file of client, or client is used for file access
Metamessage is cached；The security module is responsible for encrypting data, guarantees the safety of data；
The request or data that main control server receives the client and the data server is sent, and content based on the received
Control processing is carried out, while the data server is managed and is safeguarded；The main control server includes active and standby disaster tolerance mould
Block, name index module, data server management module and service dispatch module, wherein：The active and standby disaster tolerance module guarantees
Its work is taken in main control server node failure；The name index module is responsible for establishing name index catalogue, storage text
The mapping relations of part name and corresponding data server；The data server management module, manages all data servers
State, and execute load balancing plan；The service dispatch module is responsible for receiving from client and data server
Request；
Data server stores the data sent by the client or the main control server, and the data server includes
Data memory module, replica management module, name search module and state-maintenance module, wherein：The data memory module is negative
Duty is by file in the storage for locally carrying out persistence；File is stored one or more copies to data by the replica management module
On server；The name search module is responsible for establishing name search catalogue to the file in the data memory module；It is described
State-maintenance module is responsible for periodically periodically reporting the status information of oneself to main control server in a manner of heartbeat packet.
8. according to the method described in claim 7, it is characterized in that：This method further includes：Main control server is to the client
The filename of quasi- storage file adds file group name, and distributes data service corresponding with file group name to the file of quasi- storage
Device is stored.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510401914.3A CN105005611B (en) | 2015-07-10 | 2015-07-10 | A kind of file management system and file management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510401914.3A CN105005611B (en) | 2015-07-10 | 2015-07-10 | A kind of file management system and file management method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105005611A CN105005611A (en) | 2015-10-28 |
| CN105005611B true CN105005611B (en) | 2018-11-30 |
Family
ID=54378287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510401914.3A Active CN105005611B (en) | 2015-07-10 | 2015-07-10 | A kind of file management system and file management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105005611B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106024039B (en) * | 2016-06-02 | 2018-11-02 | 深圳市爱思拓信息存储技术有限公司 | Magneto-optic cloud integral optical disk magnanimity secure storage array library |
| CN106024038B (en) * | 2016-06-02 | 2018-10-30 | 深圳市爱思拓信息存储技术有限公司 | Electromagnetism Shekinah integral optical disk magnanimity secure storage array library |
| CN106528830B (en) * | 2016-11-16 | 2019-05-10 | 华为技术有限公司 | A kind of method and apparatus for restoring file index catalogue |
| CN107194271A (en) * | 2017-04-18 | 2017-09-22 | 华南农业大学 | A kind of shared private cloud storage system of weak center |
| CN107092686B (en) * | 2017-04-24 | 2020-04-10 | 浙江宇视科技有限公司 | File management method and device based on cloud storage platform |
| CN107967366B (en) * | 2017-12-22 | 2022-03-01 | 深圳Tcl新技术有限公司 | File management method, U disk and computer readable storage medium |
| CN108156040A (en) * | 2018-01-30 | 2018-06-12 | 北京交通大学 | A kind of central control node in distribution cloud storage system |
| JP6708239B2 (en) * | 2018-09-21 | 2020-06-10 | 富士ゼロックス株式会社 | Document management system |
| CN109871365A (en) * | 2019-01-15 | 2019-06-11 | 苏州链读文化传媒有限公司 | A kind of distributed file system |
| CN109756573B (en) * | 2019-01-15 | 2022-02-08 | 苏州链读文化传媒有限公司 | File system based on block chain |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020315A (en) * | 2013-01-10 | 2013-04-03 | 中国人民解放军国防科学技术大学 | Method for storing mass of small files on basis of master-slave distributed file system |
| CN103532754A (en) * | 2013-10-12 | 2014-01-22 | 北京首信科技股份有限公司 | System and method for high-speed memory and distributed type processing of massive logs |
-
2015
- 2015-07-10 CN CN201510401914.3A patent/CN105005611B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020315A (en) * | 2013-01-10 | 2013-04-03 | 中国人民解放军国防科学技术大学 | Method for storing mass of small files on basis of master-slave distributed file system |
| CN103532754A (en) * | 2013-10-12 | 2014-01-22 | 北京首信科技股份有限公司 | System and method for high-speed memory and distributed type processing of massive logs |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105005611A (en) | 2015-10-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105005611B (en) | A kind of file management system and file management method | |
| US10581957B2 (en) | Multi-level data staging for low latency data access | |
| AU2013347807B2 (en) | Scaling computing clusters | |
| US20150215405A1 (en) | Methods of managing and storing distributed files based on information-centric network | |
| AU2014212780B2 (en) | Data stream splitting for low-latency data access | |
| US8312037B1 (en) | Dynamic tree determination for data processing | |
| CN103106249B (en) | A kind of parallel data processing system based on Cassandra | |
| US20110055494A1 (en) | Method for distributed direct object access storage | |
| US9774676B2 (en) | Storing and moving data in a distributed storage system | |
| KR20120072907A (en) | Distribution storage system of distributively storing objects based on position of plural data nodes, position-based object distributive storing method thereof, and computer-readable recording medium | |
| US20220269680A1 (en) | Context dependent execution time prediction for redirecting queries | |
| US10579597B1 (en) | Data-tiering service with multiple cold tier quality of service levels | |
| Zeng et al. | Optimal metadata replications and request balancing strategy on cloud data centers | |
| CN102664914A (en) | IS/DFS-Image distributed file storage query system | |
| US8930518B2 (en) | Processing of write requests in application server clusters | |
| KR20100048130A (en) | Distributed storage system based on metadata cluster and method thereof | |
| US20180060341A1 (en) | Querying Data Records Stored On A Distributed File System | |
| Sundarakumar et al. | A comprehensive study and review of tuning the performance on database scalability in big data analytics | |
| Pingle et al. | Big data processing using apache hadoop in cloud system | |
| EP2765517B1 (en) | Data stream splitting for low-latency data access | |
| WO2014180395A1 (en) | Mass data fusion storage method and system | |
| CN110633256A (en) | Session Session sharing method in distributed cluster system | |
| US10887429B1 (en) | Processing multi-protocol redirection links | |
| Deshpande et al. | A comparative analysis of data replication strategies and consistency maintenance in distributed file systems | |
| Duan et al. | A high‐performance distributed file system for large‐scale concurrent HD video streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |