CN105812427B - File uploads and method for down loading, device and file server - Google Patents
File uploads and method for down loading, device and file server Download PDFInfo
- Publication number
- CN105812427B CN105812427B CN201410851403.7A CN201410851403A CN105812427B CN 105812427 B CN105812427 B CN 105812427B CN 201410851403 A CN201410851403 A CN 201410851403A CN 105812427 B CN105812427 B CN 105812427B
- Authority
- CN
- China
- Prior art keywords
- file
- blocked
- summary info
- storage catalogue
- catalogue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The embodiment of the invention discloses a kind of upload of file and method for down loading, device and file servers, are related to cloud storage technical field, wherein the file uploading method includes: the summary info for generating the file that request uploads；The storage catalogue of the file is generated according to the summary info of the file；The file is saved under the storage catalogue of the file.The upload and speed of download of file can be improved in the embodiment of the present invention.
Description
Technical field
The present invention relates to cloud storage technical field, especially a kind of file uploading method and device, document down loading method and
Device and file server.
Background technique
Under internet and big data background prevailing, there be depositing for mass data in either personal or enterprise institution
Storage and process demand.Currently, in order to more efficiently store and processing data, large enterprise, operator, internet service provider etc.
Mostly using realizing the storage of big file and magnanimity day based on the HDFS (distributed file system) of Hadoop (distributed computing)
Will file process.
Figure 12 is the client/server schematic diagram of HDFS in the prior art.As shown in figure 12, it is needed in HDFS using a master
Server (NameNode) is held to store metadata information, wherein NameNode is located at the main side of Hadoop cluster file system,
It instructs the I/O task that bottom is executed from multiple server nodes (DataNode) at end, and tracks how file is divided
It is whether normal by the overall operation state of which node storage and HDFS at blocks of files, these blocks of files.
Compared with traditional memory module, the HDFS based on Hadoop is easier to extend clustered node, in memory technology
Advantageously and feasibility.But there are the following problems by HDFS:
1, it is delayed larger, the reading efficiency of file is low.HDFS is to use to be deployed in the metadata concentrated on the server of main side
File block map information is stored in library, and all files upload, downloading and piecemeal storage operation require frequent read/write data
Blocked file position is found in library, is then arrived from end server again and is found these blocked files, carries out I/O operation, delay compared with
Greatly.In addition, main side server is placed the meta data file of All Files system in memory, the text that file system can accommodate
Number of packages mesh is determined by the memory size of main side server.When quantity of documents is more, the memory of occupancy also can be bigger,
When reaching the memory limit of main side server, the efficiency of reading can be reduced.Therefore, HDFS is poorly suited for those and requires low prolong
When (several ms) access application (such as enterprise's Dropbox)；
2, safety is not high.Since HDFS all meta data files are all centrally saved in the database in the server of main side
In, safety depends on the complexity of the password setup of database administrator.Once database administrator password is attacked
Broken, all meta data files will face stolen risk, and can then reveal completely from the data in the server of end.
Summary of the invention
One purpose of the embodiment of the present invention is: improving the reading efficiency of file, reduces delay.
Another purpose of the embodiment of the present invention is: improving the safety of file storage.
According to an aspect of the present invention, a kind of file uploading method is provided, comprising: generate the abstract for the file that request uploads
Information；The storage catalogue of the file is generated according to the summary info of the file；The file is saved in the file
Under storage catalogue.
In one embodiment, the storage catalogue that the summary info according to the file generates the file includes:
At least one section of character in the summary info is obtained, and calculates the corresponding catalogue of every section of character；By the corresponding mesh of every section of character
Record splicing, to form the storage catalogue of the file.
It in one embodiment, include: described in judgement under the storage catalogue that the file is saved in the file
Whether with the summary info of the file identical file is had under storage catalogue；If not provided, the file is saved in described
Under the storage catalogue of file.
In one embodiment, the file includes meta data file and file content；The method specifically includes: according to
The file content of file generates the summary info of the file；The storage of the file is generated according to the summary info of the file
Catalogue；Meta data file and file content are saved under the storage catalogue of the file.
In one embodiment, the file includes meta data file and at least one blocked file, each blocked file
Including metadata header file and blocked file content；The method specifically includes: generating institute according to all blocked file contents
State the summary info of file；The storage catalogue of the meta data file is generated according to the summary info of the file；By metadata
File is saved under the storage catalogue of the meta data file；Each blocked file is generated respectively according to each blocked file content
Summary info；Generate the storage catalogue of each blocked file respectively according to the summary info of each blocked file；To each it divide
Block file is saved in respectively under respective storage catalogue.
In one embodiment, described each blocked file is saved in respectively under respective storage catalogue includes: to every
A blocked file is saved under respective storage catalogue respectively again after being encrypted.
In one embodiment, the method also includes: record in the meta data file point of each blocked file
Block message, the blocking information include at least the summary info of each blocked file and the status information of each blocked file.
In one embodiment, the summary info includes MD5 value.
According to another aspect of the present invention, a kind of document down loading method is provided, comprising: according to pre-stored filename and pluck
Want the corresponding relationship of message, the summary info of the file of search request downloading；The file is determined according to the summary info of file
Storage catalogue, so that client downloads the file from the storage catalogue of file.
In one embodiment, the file includes meta data file and blocked file；It is described to be believed according to the abstract of file
Breath determines the storage catalogue of the file, and downloading the file from the storage catalogue of file so as to client includes: according to file
Summary info determine the storage catalogue of the meta data file, and the meta data file is obtained according to the storage catalogue；Root
According to the summary info of each blocked file recorded in the meta data file, the storage catalogue of each blocked file is determined；From
Each blocked file is obtained under the storage catalogue of each blocked file, for client downloads.
In one embodiment, described that each blocked file is obtained under the storage catalogue of each blocked file, for visitor
The downloading of family end includes: each blocked file to be obtained under the storage catalogue of each blocked file, and carry out to each blocked file
Decryption, for client downloads.
In one embodiment, the summary info includes MD5 value.
According to another aspect of the invention, a kind of file upload device is provided, comprising: summary info generation unit is used for
Generate the summary info for the file that request uploads；Storage catalogue generation unit, for being generated according to the summary info of the file
The storage catalogue of the file；File storing unit, for the file to be saved under the storage catalogue of the file.
In one embodiment, the storage catalogue generation unit includes: that character obtains module, for obtaining the abstract
At least one section of character in information, and calculate the corresponding catalogue of every section of character；Catalogue splicing module, for every section of character is corresponding
Catalogue splicing, to form the storage catalogue of the file.
In one embodiment, whether the file storing unit is specifically used for: judging to have under the storage catalogue and institute
State the identical file of summary info of file；If not provided, the file is saved under the storage catalogue of the file.
In one embodiment, the file includes meta data file and file content；The summary info generation unit,
The summary info of the file is generated specifically for the file content according to file；The storage catalogue generation unit is specific to use
In the storage catalogue for generating the file according to the summary info of the file；The file storing unit, being specifically used for will be first
Data file and file content are saved under the storage catalogue of the file.
In one embodiment, the file includes meta data file and at least one blocked file, each blocked file
Including metadata header file and blocked file content；The summary info generation unit, specifically for according to all piecemeal texts
Part content generates the summary info of the file；Generate the abstract letter of each blocked file respectively according to each blocked file content
Breath；The storage catalogue generation unit, specifically for generating depositing for the meta data file according to the summary info of the file
Store up catalogue；Generate the storage catalogue of each blocked file respectively according to the summary info of each blocked file；The file saves
Unit, specifically for meta data file to be saved under the storage catalogue of the meta data file；Each blocked file is distinguished
It is saved under respective storage catalogue.
In one embodiment, each blocked file is being saved in respective storage mesh by the file storing unit respectively
When under record, it is specifically used for: is saved under respective storage catalogue respectively again after being encrypted to each blocked file.
In one embodiment, the file uploads device further include: blocking information recording unit, in first number
According to the blocking information for recording each blocked file in file, the blocking information includes at least the summary info of each blocked file
With the status information of each blocked file.
In one embodiment, the summary info includes MD5 value.
In accordance with a further aspect of the present invention, a kind of file downloading device is provided, comprising: summary info searching unit is used for
According to the corresponding relationship of pre-stored filename and summary information, the summary info of the file of search request downloading；Storage catalogue
Determination unit determines the storage catalogue of the file for the summary info according to file, so that client is from the storage of file
Catalogue downloads the file.
In one embodiment, the file includes meta data file and blocked file；The storage catalogue determination unit
Include: that meta data file obtains module, the storage catalogue of the meta data file is determined for the summary info according to file, and
The meta data file is obtained according to the storage catalogue；Piecemeal catalogue determining module, for remembering according in the meta data file
The summary info of each blocked file of record, determines the storage catalogue of each blocked file；Blocked file obtain module, for from
Each blocked file is obtained under the storage catalogue of each blocked file, for client downloads.
In one embodiment, the blocked file obtains module, is specifically used for: from the storage catalogue of each blocked file
It is lower to obtain each blocked file, and each blocked file is decrypted, for client downloads.
In one embodiment, the summary info includes MD5 value.
Also one side according to the present invention, provides a kind of file server, comprising: the file that above-mentioned any embodiment provides
Upload device and file downloading device.
The embodiment of the present invention at least has the advantages that
On the one hand, the summary info for the file that request uploads is associated with storage catalogue, file is saved directly to basis
Under the storage catalogue that the summary info of file generates.Compared with transmitting file in the prior art requires through main side server, this
Inventive embodiments directly by under file storage to its corresponding storage catalogue, when there is the upload of multiple files, are not necessarily to pass through one
Identical main side server is completed, and is reduced delay, is improved the uploading speed of file.Correspondingly, when downloading file, root
It can be directly targeted to the storage catalogue of file according to the summary info of file, eliminate from the same main side server and search first number
According to the waiting time of the database of file, delay is reduced, improves the speed of download of file.
On the other hand, the present invention can disperse the meta data file of different files to be stored in the storage of file server difference
In catalogue, extreme difficulties are caused to splice all blocked files, safety greatly improves.
Below by drawings and examples, technical scheme of the present invention will be described in further detail.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is the flow chart of file uploading method one embodiment of the present invention；
Fig. 2 is the flow chart of another embodiment of file uploading method of the present invention；
Fig. 3 is the flow chart of another embodiment of file uploading method of the present invention；
Fig. 4 is an example of file block of the present invention storage；
Fig. 5 is the flow chart of document down loading method one embodiment of the present invention；
Fig. 6 is the flow chart of another embodiment of document down loading method of the present invention；
Fig. 7 is the structural schematic diagram that file of the present invention uploads device one embodiment；
Fig. 8 is the structural schematic diagram that file of the present invention uploads another embodiment of device；
Fig. 9 is the structural schematic diagram that file of the present invention uploads another embodiment of device；
Figure 10 is the structural schematic diagram of file downloading device one embodiment of the present invention；
Figure 11 is the structural schematic diagram of another embodiment of file downloading device of the present invention；
Figure 12 is the client/server schematic diagram of HDFS in the prior art.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Unless specifically stated otherwise, positioned opposite, the digital table of the component and step that otherwise illustrate in these embodiments
It is not limited the scope of the invention up to formula and numerical value.
Simultaneously, it should be appreciated that for ease of description, the size of various pieces shown in attached drawing is not according to reality
Proportionate relationship draw.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as authorizing part of specification.
It is shown here and discuss all examples in, any occurrence should be construed as merely illustratively, without
It is as limitation.Therefore, the other examples of exemplary embodiment can have different values.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
Fig. 1 is the flow chart of file uploading method one embodiment of the present invention, this method can by file server or
Person is deployed in the file on file server and uploads device to realize.As shown in Figure 1, the file uploading method of the present embodiment includes
Following steps:
Step 102, the summary info for the file that request uploads is generated.
Wherein, summary info is the eap-message digest (Message Digest) of file, for a specific file, is led to
It is certain for crossing the summary info of certain algorithm generation.
For example, MD5 (Message-Digest algorithm 5) value, the MD4 value, MD3 value, MD2 of file can be generated by hash algorithm
The summary info of other hashed values as file such as value or SHA1 value.It should be understood that the summary info in the embodiment of the present invention is not
It is limited to above-named specific example.
The corresponding relationship that filename and summary info can also be established after generating summary info, can when to download file
To find the summary info of file according to filename.
Step 104, the storage catalogue of file is generated according to the summary info of file.
Wherein, storage catalogue can be one or more levels, will be explained in hereinafter.
Step 106, file is saved under the storage catalogue of file.
The summary info for the file that request uploads is associated with by the present embodiment with storage catalogue, and file is saved directly to basis
Under the storage catalogue that the summary info of file generates.Compared with transmitting file in the prior art requires through main side server, this
Embodiment directly by under file storage to its corresponding storage catalogue, when there is the upload of multiple files, is not necessarily to identical by one
Main side server complete, improve the uploading speed of file.
Fig. 2 is the flow chart of another embodiment of file uploading method of the present invention.Step 104 in the embodiment can lead to
It crosses such as under type and realizes:
Step 202, at least one section of character in summary info is obtained, and calculates the corresponding catalogue of every section of character.
Step 204, the corresponding catalogue of every section of character is spliced, to form the storage catalogue of file.
Below to the step in the present embodiment by taking summary info is MD5 value, the storage catalogue of generation is three-level catalogue as an example
104 realization process is illustrated, and the generation method of other grade of catalogue is referred to following description.
It is possible, firstly, to preset fixed position takes three sections of characters from the character string of MD5 value, by this three sections of characters by two into
System is converted to hexadecimal, so as to obtain the corresponding catalogue of every section of character.Then, the corresponding catalogue of every section of character is spelled
It connects, forms the storage catalogue of file.The form of the storage catalogue for example can be with are as follows:/0A/1B/2C, wherein every grade of catalogue is 2
A 16 system character.
It should be noted that when forming storage catalogue, every section of character position taken can for different files
With difference, it is thusly-formed the corresponding storage catalogue of different files, the difficulty that file cracks is improved, enhances the peace of file storage
Quan Xing.
Fig. 3 is the flow chart of another embodiment of file uploading method of the present invention.The file uploading method packet of the present embodiment
Include following steps:
Step 102, the summary info for the file that request uploads is generated.
Step 104, the storage catalogue of file is generated according to the summary info of file.
Step 302, judge whether there is file identical with the summary info of file under storage catalogue.If so, executing step
Rapid 304；If not, executing step 106.
Step 106, file is saved under the storage catalogue of file.
Step 304, no longer file is saved under the storage catalogue of file.
The present embodiment judges before saving file compared with embodiment illustrated in fig. 1, if the storage of this document
There is file identical with the summary info of file under catalogue, then without repeating upload this document, " has ignored formula to reach
The effect of upload (second biography) ".
Under normal conditions, the summary info of same file is also identical.But it in order to avoid the generation of extreme case, is holding
It, can also other attributes, such as the modification time of file type, file etc. to the file under storage catalogue when row step 302
The file uploaded with request compares, to ensure to request the file uploaded that can be saved under corresponding catalogue.
For different size of file, when being uploaded, method shown in the various embodiments described above can be applicable in.Below
The file uploading method of small documents and big file is illustrated respectively.It should be pointed out that due to the various embodiments described above
Generation, the generation of storage catalogue etc. of summary info are described in detail, below for the upload of small documents and big file
Correlation step in method is described relatively simple, and related place may refer to above description.
For small documents, such as file is less than 128K, and file specifically includes meta data file and file content.Its
In, meta data file only includes the description information of file, and file content includes binary data content.
File uploading method shown in FIG. 1 can specifically include:
S1 generates the summary info of file according to the file content of file, i.e., raw according to the binary data file of file
At the summary info of file.
S2 generates the storage catalogue of file according to the summary info of file；
Meta data file and file content are saved under the storage catalogue of file by S3.
Meta data file and file content are saved in the same storage catalogue of generation for for small documents by the present embodiment
Under.For different small documents, respective meta data file is not stored in identical main side server, on file
Without completing by same main side server when biography, the uploading speed of file is improved.
For big file, such as more than the file of minimum piecemeal particle, file can be carried out protecting again after piecemeal
It is stored under corresponding storage catalogue.For piecemeal operation for, can be executed by client, can also by file server Lai
It executes.If executed by client, client can carry out file requesting to upload again after piecemeal, that is, request the text uploaded
Part has been the file after piecemeal, and file server directly stores file；If executed by file server, file clothes
Business device can carry out piecemeal to the file that request uploads, and carry out subsequent storage operation again later.
File server can limit the size of blocked file, to avoid too small or excessive piecemeal, for example, point
The size of block file can be defaulted as 256K.As a non-restrictive example, the magnitude range of blocked file be can be set as
128K to 1024K.For example, since file is smaller, the size of blocked file can be set as 128K for text file
Etc. relatively small value；For video file, since file is larger, the size of blocked file can be set as 1024K
Etc. relatively large value.
File after piecemeal includes meta data file and at least one blocked file, and each blocked file includes metadata
Header file and blocked file content.Wherein, meta data file only includes the description information and blocked file information of entire file；Often
A metadata header file includes the description information of corresponding blocked file, and blocked file content includes the two of corresponding blocked file
Binary data content.
For big file, file uploading method shown in FIG. 1 can specifically include following steps:
S1 generates the summary info of file according to all blocked file contents.
S2 generates the storage catalogue of meta data file according to the summary info of file.
Meta data file is saved under the storage catalogue of meta data file by S3.
S4 generates the summary info of each blocked file according to each blocked file content respectively.
S5 generates the storage catalogue of each blocked file according to the summary info of each blocked file respectively.
Each blocked file is saved under respective storage catalogue by S6 respectively.
In addition to above-mentioned steps, the blocking information of each blocked file can also be recorded in meta data file, wherein point
Block message includes at least the summary info of each blocked file and the status information of each blocked file.According in meta data file
The status information of blocked file can check whether each blocked file uploads success, and the blocked file for uploading failure can be with
Again it uploads.Alternatively, continuing to upload on the basis of blocked file uploads, the function of breakpoint transmission is realized.
Fig. 4 is an example of file block of the present invention storage.As shown in figure 4, file is single whole text before piecemeal
Part, after piecemeal, file is divided into meta data file and blocked file 1, blocked file 2, blocked file 3, wherein blocked file 1
Comprising header file 1 and blocked file content 1, blocked file 2 includes header file 2 and blocked file content 2, piecemeal text
Part 3 includes header file 3 and blocked file content 3.
Wherein, meta data file mainly includes two large divisions's content, first is that the summary info of entire file, such as MD5 value；
Second is that the blocking information of each blocked file, specifically includes the total quantity of blocked file, the size of blocked file, summary info
(such as MD5 value), the status information of blocked file, the location information of blocked file etc..
As a non-limiting example, the data structure of meta data file for example can be following form:
As a non-limiting example, the specific data structure of the header file of blocked file for example can be as follows
Form:
According to the file uploading method of above-mentioned big file, meta data file and each blocked file can be determined respectively
Then storage catalogue is respectively stored into the corresponding storage location of storage catalogue.As shown in figure 4, meta data file is possibly stored to
In physical volume 1, blocked file 1 is possibly stored in physical volume 2, and blocked file 2 is possibly stored in physical volume 3, blocked file 1
It is possibly stored in physical volume 1.
The meta data file of file is stored in the summary info according to entire file for for big file by the present embodiment
Under the storage catalogue of generation, each blocked file is stored in the storage generated respectively according to the summary info of each blocked file
Under catalogue.For different big files, respective meta data file is respectively stored under different storage catalogues, i.e., big text
The storage location of part is not fixed, relatively random.On the one hand, the present embodiment improves the uploading speed of file；On the other hand, even if
The content of blocked file is cracked, and since different meta data file dispersions is stored under different storage catalogues, increases and looks for
To the difficulty of corresponding meta data file, to improve the safety of file storage.
In order to further enhance the safety of file, file can be carried out before storing file in the various embodiments described above
It is stored again after encryption, for example, for blocked file AES 256 can be carried out to each blocked file by aes algorithm
It is saved under respective storage catalogue respectively again after the encryption of position.
By carrying out AES 256 encryptions to each blocked file, even if file server is broken, each piecemeal text
The original contents of part improve the safety of file there is still a need for can just be cracked for quite a long time.
File uploading method provided in an embodiment of the present invention has three-layer protection from the angle of safety, that is, only breaks through
At least three-layer protection could splice each blocked file: first is that the administrator's password of computer room file server, second is that often
The password of a blocked file, such as 256 private keys of AES, third is that the dispersion of meta data file stores.Even if computer room file service
Device administrator's password is broken, and the password of each blocked file is also stolen, but since the meta data file of different files disperses
It is stored in file server difference storage catalogue, causes extreme difficulties to splice all blocked files.Pass through number with HDFS
It is compared according to one layer of protection of library server, safety greatly improves.
File is uploaded onto the server behind end, can be carried out by clients such as mobile phone or computers according to the agreement set
Downloading.With above-mentioned file uploading method correspondingly, the embodiment of the invention also provides document down loading methods.
Fig. 5 is the flow chart of document down loading method one embodiment of the present invention.This method can by file server or
Person is deployed in the file downloading device on file server to realize.As shown in figure 5, the document down loading method of the present embodiment includes
Following steps:
Step 502, according to the corresponding relationship of pre-stored filename and summary information, the file of search request downloading is plucked
Want information.
Wherein, summary info for example can be MD5 value etc..
Step 504, the storage catalogue of file is determined, according to the summary info of file so that client is from the storage mesh of file
Record downloading file.
The present embodiment can be directly targeted to the storage catalogue of file according to the summary info of file, eliminate from same
Main side server searches the waiting time of the database of meta data file, realizes the quick-downloading of file.
Fig. 6 is the flow chart of another embodiment of document down loading method of the present invention.Wherein, file include meta data file and
Blocked file.As shown in fig. 6, the step 504 in the present embodiment can be achieved in that
Step 602, the storage catalogue of meta data file is determined according to the summary info of file, and is obtained according to the storage catalogue
Take meta data file.
Step 604, according to the summary info of each blocked file recorded in meta data file, each blocked file is determined
Storage catalogue；
Step 606, each blocked file is obtained, under the storage catalogue of each blocked file so that client downloads are each
Blocked file, and each blocked file is spliced into complete file.
The present embodiment eliminates HDFS file system in reading/locating file, reads into main side server metadata library
The operation for taking file metadata is greatly saved read access time and the download time of file, improves the operating efficiency of file.
For the piecemeal of encryption, the step 606 in above-described embodiment is under the storage catalogue from each blocked file
After obtaining each blocked file, in response to the password of user's input, each blocked file is decrypted, under client
It carries.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with its
The difference of its embodiment, the same or similar part cross-reference between each embodiment.For Installation practice
For, since it is substantially corresponding with embodiment of the method, so being described relatively simple, referring to the portion of embodiment of the method in place of correlation
It defends oneself bright.
Fig. 7 is the structural schematic diagram that file of the present invention uploads device one embodiment.The file of the present embodiment uploads device
It can be used for realizing the file uploading method of above-described embodiment.As shown in fig. 7, comprising:
Summary info generation unit 701, for generating the summary info for the file that request uploads；Storage catalogue generation unit
702, for generating the storage catalogue of file according to the summary info of file；File storing unit 703, for file to be saved in
Under the storage catalogue of file.
Fig. 8 is the structural schematic diagram that file of the present invention uploads another embodiment of device.Storage catalogue in the present embodiment
Generation unit 702 may include:
Character obtains module 712, and for obtaining at least one section of character in summary info, and it is corresponding to calculate every section of character
Catalogue；Catalogue splicing module 722, for splicing the corresponding catalogue of every section of character, to form the storage catalogue of file.
In one embodiment, referring to Fig. 7 or Fig. 8, file storing unit 703 is specifically used for: judge be under storage catalogue
It is no to have file identical with the summary info of file；If not provided, file is saved under the storage catalogue of file.
In one embodiment, referring to Fig. 7 or Fig. 8, file includes meta data file and file content.In the present embodiment,
Summary info generation unit 701, specifically for generating the summary info of file according to the file content of file；Storage catalogue generates
Unit 702, specifically for generating the storage catalogue of file according to the summary info of file；File storing unit 703, is specifically used for
Meta data file and file content are saved under the storage catalogue of file.
In one embodiment, referring to Fig. 7 or Fig. 8, file includes meta data file and at least one blocked file, each
Blocked file includes metadata header file and blocked file content.In the present embodiment, summary info generation unit 701 is specific to use
In the summary info for generating file according to all blocked file contents；Each point is generated respectively according to each blocked file content
The summary info of block file；Storage catalogue generation unit 702, specifically for generating meta data file according to the summary info of file
Storage catalogue；Generate the storage catalogue of each blocked file respectively according to the summary info of each blocked file；File saves
Unit 703, specifically for meta data file to be saved under the storage catalogue of meta data file；Each blocked file is protected respectively
It is stored under respective storage catalogue.
In one embodiment, referring to Fig. 7 or Fig. 8, file storing unit 703 is saved in respectively by each blocked file
When under respective storage catalogue, it is specifically used for: is saved in respective storage after being encrypted to each blocked file respectively again
Under catalogue.
Fig. 9 is the structural schematic diagram that file of the present invention uploads another embodiment of device.The file of the present embodiment uploads dress
Setting can also include: blocking information recording unit 901, and the piecemeal for recording each blocked file in meta data file is believed
Breath, blocking information include at least the summary info of each blocked file and the status information of each blocked file.
Figure 10 is the structural schematic diagram of file downloading device one embodiment of the present invention.The file downloading device of the present embodiment
It can be used for realizing the document down loading method of above-described embodiment.As shown in Figure 10, the file downloading device of the present embodiment includes:
Summary info searching unit 1001 is searched for the corresponding relationship according to pre-stored filename and summary information
Request the summary info of the file of downloading；Storage catalogue determination unit 1002, for determining file according to the summary info of file
Storage catalogue, so as to client from the storage catalogue of file download file.
Figure 11 is the structural schematic diagram of another embodiment of file downloading device of the present invention.Wherein, file includes metadata
File and blocked file.As shown in figure 11, the storage catalogue determination unit 1002 in the present embodiment includes:
Meta data file obtains module 1012, and the storage mesh of meta data file is determined for the summary info according to file
Record, and meta data file is obtained according to the storage catalogue；Piecemeal catalogue determining module 1022, for remembering according in meta data file
The summary info of each blocked file of record, determines the storage catalogue of each blocked file；Blocked file obtains module 1032, uses
Each blocked file is obtained, under the storage catalogue from each blocked file for client downloads.
In one embodiment, referring to Figure 11, blocked file obtains module 1032, is specifically used for: from each blocked file
Storage catalogue under obtain each blocked file, and each blocked file is decrypted, for client downloads.
As a non-limiting example, above-mentioned summary info includes MD5 value.
The embodiment of the invention also provides a kind of file servers, comprising: the file that above-mentioned any embodiment provides uploads
Device and file downloading device.
Each functional unit that file provided in an embodiment of the present invention uploads device can dispose on one server or
On different multiple servers.Similarly, each functional unit of file downloading device can dispose on one server or
On different multiple servers.
For example, file, which uploads device and file downloading device, can be deployed on a file server.In another example file
The unit of the unit and file downloading device that upload device can be used as an independent device and be deployed in another
On server, to alleviate the pressure of file server.Specifically, the summary info that file can be uploaded in device generates list
Member and file downloading device in summary info searching unit deployment on the application server, with realize summary info generation and
Locating function.
File provided in an embodiment of the present invention uploads and method for down loading is suitable for All Files type, such as all kinds of offices text
Shelves, audio, video file etc..In particular for Dropbox class application, need frequent progress file to upload, downloading, piecemeal save and
The operation such as splicing, the I/O efficiency that file system is carried to it have higher requirements；Simultaneously as the content of Dropbox class application memory
It is client's private data, there are requirements at the higher level to the safety and privacy of storing data.It therefore, can be by file of the invention
The backend file system that server is applied as Dropbox class.
The embodiment of the present invention with Dropbox class interface applications when merging, it is only necessary to slightly adjust at following two aspect:
First is that the size for being suitble to the blocked file of the main storage content of Dropbox is arranged.For example, video class file is larger, each blocked file
It can be set to slightly larger value, such as 1024K；Second is that establishing database.Main storage service data include filename, Yi Jiwen
The corresponding summary info of part name (such as MD5 value) information can obtain file when downloading file so as to client request quickly
Summary info, to quickly position the storage location of meta data file.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above method embodiment can pass through
The relevant hardware of program instruction is completed, and program above-mentioned can be stored in a computer readable storage medium, the program
When being executed, step including the steps of the foregoing method embodiments is executed；And storage medium above-mentioned includes: ROM, RAM, magnetic disk or light
The various media that can store program code such as disk.
Description of the invention is given for the purpose of illustration and description, and is not exhaustively or will be of the invention
It is limited to disclosed form.Many modifications and variations are obvious for the ordinary skill in the art.It selects and retouches
It states embodiment and is to more preferably illustrate the principle of the present invention and practical application, and those skilled in the art is enable to manage
The solution present invention is to design various embodiments suitable for specific applications with various modifications.
Claims (19)
1. a kind of file uploading method characterized by comprising
Generate the summary info for the file that request uploads；
The storage catalogue of the file is generated according to the summary info of the file；
The file is saved under the storage catalogue of the file；
Wherein, the storage catalogue that the summary info according to the file generates the file includes:
At least one section of character in the summary info is obtained, and calculates the corresponding catalogue of every section of character；
By the corresponding catalogue splicing of every section of character, to form the storage catalogue of the file；
Wherein, when generating the storage catalogue of different files, the position of the every section of character obtained from the summary info is different；
It wherein, include meta data file and at least one blocked file in the file, each blocked file includes metadata header
In the case where file and blocked file content:
The summary info of the file is generated according to all blocked file contents；
The storage catalogue of the meta data file is generated according to the summary info of the file；
Meta data file is saved under the storage catalogue of the meta data file；
Generate the summary info of each blocked file respectively according to each blocked file content；
Generate the storage catalogue of each blocked file respectively according to the summary info of each blocked file；
Each blocked file is saved in respectively under respective storage catalogue.
2. the method according to claim 1, wherein the storage mesh that the file is saved in the file
Include: under record
Judge whether there is file identical with the summary info of the file under the storage catalogue；
If not provided, the file is saved under the storage catalogue of the file.
3. the method according to claim 1, wherein including meta data file and file content in the file
In the case of:
The summary info of the file is generated according to the file content of file；
Meta data file and file content are saved under the storage catalogue of the file.
4. the method according to claim 1, wherein described be saved in respective deposit for each blocked file respectively
Include: under storage catalogue
It is saved under respective storage catalogue respectively again after being encrypted to each blocked file.
5. the method according to claim 1, wherein further include:
The blocking information of each blocked file is recorded in the meta data file, the blocking information includes at least each piecemeal
The status information of the summary info of file and each blocked file.
6. method described in -5 any one according to claim 1, which is characterized in that the summary info includes MD5 value.
7. a kind of document down loading method characterized by comprising
According to the corresponding relationship of pre-stored filename and summary information, the summary info of the file of search request downloading；
Determine the storage catalogue of the file, according to the summary info of file so that client downloads institute from the storage catalogue of file
State file；
Wherein, the storage catalogue of the file is generated according to following manner:
At least one section of character in the summary info is obtained, and calculates the corresponding catalogue of every section of character；
By the corresponding catalogue splicing of every section of character, to form the storage catalogue of the file；
Wherein, when generating the storage catalogue of different files, the position of the every section of character obtained from the summary info is different；
Wherein, the file includes meta data file and blocked file；
The summary info according to file determines the storage catalogue of the file, so that client is under the storage catalogue of file
Carrying the file includes:
The storage catalogue of the meta data file is determined according to the summary info of file, and the member is obtained according to the storage catalogue
Data file；
According to the summary info of each blocked file recorded in the meta data file, the storage mesh of each blocked file is determined
Record；
Each blocked file is obtained, under the storage catalogue of each blocked file for client downloads.
8. the method according to the description of claim 7 is characterized in that described obtain often under the storage catalogue of each blocked file
A blocked file, so that client downloads include:
Each blocked file is obtained under the storage catalogue of each blocked file, and each blocked file is decrypted, for
Client downloads.
9. according to method described in claim 7-8 any one, which is characterized in that the summary info includes MD5 value.
10. a kind of file uploads device characterized by comprising
Summary info generation unit, for generating the summary info for the file that request uploads；
Storage catalogue generation unit, for generating the storage catalogue of the file according to the summary info of the file；
File storing unit, for the file to be saved under the storage catalogue of the file；
Wherein, the storage catalogue generation unit includes:
Character obtains module, for obtaining at least one section of character in the summary info, and calculates the corresponding mesh of every section of character
Record；
Catalogue splicing module, for splicing the corresponding catalogue of every section of character, to form the storage catalogue of the file；
Wherein, when generating the storage catalogue of different files, the position of the every section of character obtained from the summary info is different；
Include meta data file and at least one blocked file in the file, each blocked file include metadata header file and
In the case where blocked file content:
The summary info generation unit, the abstract specifically for generating the file according to all blocked file contents are believed
Breath；Generate the summary info of each blocked file respectively according to each blocked file content；
The storage catalogue generation unit, specifically for generating depositing for the meta data file according to the summary info of the file
Store up catalogue；Generate the storage catalogue of each blocked file respectively according to the summary info of each blocked file；
The file storing unit, specifically for meta data file to be saved under the storage catalogue of the meta data file；It will
Each blocked file is saved in respectively under respective storage catalogue.
11. device according to claim 10, which is characterized in that the file storing unit is specifically used for:
Judge whether there is file identical with the summary info of the file under the storage catalogue；
If not provided, the file is saved under the storage catalogue of the file.
12. device according to claim 10, which is characterized in that in the file include meta data file and file content
In the case where:
The summary info generation unit, the summary info of the file is generated specifically for the file content according to file；
The storage catalogue generation unit, specifically for generating the storage mesh of the file according to the summary info of the file
Record；
The file storing unit, specifically for meta data file and file content to be saved in the storage catalogue of the file
Under.
13. device according to claim 10, which is characterized in that the file storing unit divides by each blocked file
When not being saved under respective storage catalogue, it is specifically used for:
It is saved under respective storage catalogue respectively again after being encrypted to each blocked file.
14. device according to claim 12, which is characterized in that further include:
Blocking information recording unit, for recording the blocking information of each blocked file in the meta data file, described point
Block message includes at least the summary info of each blocked file and the status information of each blocked file.
15. device described in 0-14 any one according to claim 1, which is characterized in that the summary info includes MD5 value.
16. a kind of file downloading device characterized by comprising
Summary info searching unit, for the corresponding relationship according to pre-stored filename and summary information, search request downloading
File summary info；
Storage catalogue determination unit determines the storage catalogue of the file, for the summary info according to file so as to client
The file is downloaded from the storage catalogue of file；
Wherein, the storage catalogue of the file is generated according to following manner:
At least one section of character in the summary info is obtained, and calculates the corresponding catalogue of every section of character；
By the corresponding catalogue splicing of every section of character, to form the storage catalogue of the file；
Wherein, when generating the storage catalogue of different files, the position of the every section of character obtained from the summary info is different；
Wherein, the file includes meta data file and blocked file；
The storage catalogue determination unit includes:
Meta data file obtains module, and the storage catalogue of the meta data file is determined for the summary info according to file, and
The meta data file is obtained according to the storage catalogue；
Piecemeal catalogue determining module, for the summary info according to each blocked file recorded in the meta data file, really
The storage catalogue of fixed each blocked file；
Blocked file obtains module, for obtaining each blocked file under the storage catalogue of each blocked file, for client
End downloading.
17. device according to claim 16, which is characterized in that the blocked file obtains module, is specifically used for:
Each blocked file is obtained under the storage catalogue of each blocked file, and each blocked file is decrypted, for
Client downloads.
18. device described in 6-17 any one according to claim 1, which is characterized in that the summary info includes MD5 value.
19. a kind of file server characterized by comprising file described in claim 10-15 any one upload device and
File downloading device described in claim 16-18 any one.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410851403.7A CN105812427B (en) | 2014-12-31 | 2014-12-31 | File uploads and method for down loading, device and file server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410851403.7A CN105812427B (en) | 2014-12-31 | 2014-12-31 | File uploads and method for down loading, device and file server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105812427A CN105812427A (en) | 2016-07-27 |
| CN105812427B true CN105812427B (en) | 2019-08-13 |
Family
ID=56420588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410851403.7A Active CN105812427B (en) | 2014-12-31 | 2014-12-31 | File uploads and method for down loading, device and file server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105812427B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106060177A (en) * | 2016-08-11 | 2016-10-26 | 浪潮电子信息产业股份有限公司 | Local area network distributed file sharing system |
| CN107220388A (en) * | 2017-06-30 | 2017-09-29 | 郑州云海信息技术有限公司 | A kind of method and apparatus of upper transmitting file |
| CN108366010A (en) * | 2018-01-15 | 2018-08-03 | 华南理工大学 | A kind of Email filing system and its data processing method based on cloud storage |
| CN110737635B (en) * | 2018-07-02 | 2023-02-10 | 深圳联友科技有限公司 | Data blocking method |
| CN109118461A (en) * | 2018-07-06 | 2019-01-01 | 航天星图科技（北京）有限公司 | A kind of HIS fusion method based on Distributed Architecture |
| CN109890026B (en) * | 2019-01-31 | 2021-12-07 | 维沃移动通信有限公司 | File loading method and terminal equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101699822A (en) * | 2009-08-06 | 2010-04-28 | 腾讯科技（深圳）有限公司 | File uploading method and device, and mass storage system |
| CN102480494A (en) * | 2010-11-23 | 2012-05-30 | 金蝶软件（中国）有限公司 | File updating method, device and system |
| CN103384256A (en) * | 2012-05-02 | 2013-11-06 | 天津书生投资有限公司 | Cloud storage method and device |
| CN104202384A (en) * | 2014-08-27 | 2014-12-10 | 四川长虹电器股份有限公司 | File uploading and searching method of distributed file system |
-
2014
- 2014-12-31 CN CN201410851403.7A patent/CN105812427B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101699822A (en) * | 2009-08-06 | 2010-04-28 | 腾讯科技（深圳）有限公司 | File uploading method and device, and mass storage system |
| CN102480494A (en) * | 2010-11-23 | 2012-05-30 | 金蝶软件（中国）有限公司 | File updating method, device and system |
| CN103384256A (en) * | 2012-05-02 | 2013-11-06 | 天津书生投资有限公司 | Cloud storage method and device |
| CN104202384A (en) * | 2014-08-27 | 2014-12-10 | 四川长虹电器股份有限公司 | File uploading and searching method of distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105812427A (en) | 2016-07-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220078177A1 (en) | Identifying Related User Accounts Based on Authentication Data | |
| KR102243754B1 (en) | Data isolation in blockchain networks | |
| CN105812427B (en) | File uploads and method for down loading, device and file server | |
| US9961053B2 (en) | Detecting compromised credentials | |
| JP6779389B2 (en) | Processing and storage of blockchain data in a reliable execution environment | |
| US10404798B2 (en) | Systems and methods for third-party policy-based file distribution in an aggregation of cloud storage services | |
| US9961030B2 (en) | Method and system for sender-controlled messaging and content sharing | |
| KR101312125B1 (en) | Contents filtering apparatus and method thereof | |
| US11100107B2 (en) | Systems and methods for secure file management via an aggregation of cloud storage services | |
| Mulazzani et al. | Dark clouds on the horizon: Using cloud storage as attack vector and online slack space | |
| JP6622196B2 (en) | Virtual service provider zone | |
| US8452822B2 (en) | Universal file naming for personal media over content delivery networks | |
| US20120109830A1 (en) | Apparatus, system and method for a decentralized social network system and decentralized payment network system | |
| WO2020055926A2 (en) | Establishing provenance of digital assets using blockchain system | |
| US10043017B2 (en) | Systems and methods for jurisdiction independent data storage in a multi-vendor cloud environment | |
| US9542864B2 (en) | Methods and apparatus for digital steganography | |
| US10025811B2 (en) | Method and apparatus for deduplicating encrypted data | |
| EP3491808B1 (en) | Interchangeable retrieval of content | |
| US9930063B2 (en) | Random identifier generation for offline database | |
| US10984116B2 (en) | Systems and methods for digital currency or crypto currency storage in a multi-vendor cloud environment | |
| US20240106902A1 (en) | Communication protocols for an online content management system | |
| CN113468599A (en) | File storage certificate method, device, system, equipment and storage medium | |
| US20170235924A1 (en) | System and Network for Controlling Content and Accessibility | |
| Ghoshal et al. | Exploiting block-chain data structure for auditorless auditing on cloud data | |
| Bieniasz et al. | SocialStegDisc: Application of steganography in social networks to create a file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |