CN105808589A - File processing method and apparatus - Google Patents
File processing method and apparatus Download PDFInfo
- Publication number
- CN105808589A CN105808589A CN201410849959.2A CN201410849959A CN105808589A CN 105808589 A CN105808589 A CN 105808589A CN 201410849959 A CN201410849959 A CN 201410849959A CN 105808589 A CN105808589 A CN 105808589A
- Authority
- CN
- China
- Prior art keywords
- file
- log
- index
- file index
- index log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a file processing method and apparatus, and relates to the field of data processing. The method comprises the steps of receiving a file uploading request; and establishing a file index recording table in an index layer of a database for a file requested to be uploaded, wherein the file index recording table comprises recording table identifier information and a column family used for recording file index information. By establishing the file index recording table in the index layer of the database, the file can be positioned by utilizing the file index information recorded in the file index recording table; and part of tasks of NameNode are replaced, so that the burden of the NameNode is greatly reduced.
Description
Technical field
The present invention relates to data processing field, particularly to the method and apparatus of a kind of file process.
Background technology
Along with the development of science and technology, digital information increases rapidly, and how efficiently storage mass data becomes a urgent problem.Increase income by it, easy-to-use feature, Hadoop distributed platform become management and process mass data preferred option.Wherein, the distributed file system HDFS (Hadoop distributed file system) that Hadoop platform provides, owing to having the feature of high fault tolerance, can be deployed on cheap hardware, for needing the application program processing super large data set to provide the distributed storage solution of a kind of low cost, have also been obtained in recent years and be widely applied.
The design object of HDFS is primarily directed to the access of big file, is not appropriate for storage mass small documents.When the small documents of magnanimity stores in HDFS file system, the metadata information of these small documents will take a large amount of memory headrooms in NameNode (name) node, and NameNode node is caused great load.Meanwhile, every time during read-write small documents, will to information such as NameNode node request paths, the existence of mass small documents also result in NameNode node reading and writing data frequently, the performance of whole HDFS file system can be caused serious impact.
Summary of the invention
One to be solved of the embodiment of the present invention technical problem is that: reads and writes the burden causing NameNode node frequently heavier.
To be solved another of the embodiment of the present invention technical problem is that: the access efficiency of file particularly small documents is relatively low.
An aspect according to embodiments of the present invention, it is provided that a kind of method of file process, including: receive files passe request；For asking the file uploaded to set up file index log at database index layer；Wherein, file index log includes log identification information and the row race for log file index information.
In one embodiment, include for asking the file uploaded to set up file index log at database index layer: if the document size uploaded of request is less than default small documents threshold value, the file content at file index log arranges the content of storage this document；If the document size uploaded of request is more than or equal to default small documents threshold value, the value that the file content of file index log arranges is set to sky, the file that request is uploaded is stored in the data block of database purchase layer, and the store path of the store path row log file at file index log.
In one embodiment, the data block that the file that request is uploaded stores accumulation layer is included:when transmitting file in the mode adopting non-burst, according to non-slicing files size, judge that file takies the number block_count of memory block:if during block_count > 1, then obtain untapped data block by the namenode NameNode of database purchase layer, and to data block in write file, If block_count=1 and document size file_size=block size block_size, then obtain untapped data block by the namenode NameNode of database purchase layer, and to data block in written document, If block_count=1 and file_size <, block_size,, according to the byte length unused_size information that not yet stores data of the data block less than data block concordance list in database index layer, searches less than data block, if less than the unused_size > in data block, file_size, then be written to this less than in data block by the file content of non-slicing files.
In one embodiment, the file content of non-slicing files is written to this less than after in data block, method also includes: update the unused_size information less than data block concordance list less than data block, if update after the unused_size value less than data block be 0, then delete less than data block corresponding less than data block concordance list.
In one embodiment, the row race of file index log includes the document size row of each burst, the file fragmentation offset row in original, file fragmentation terminates identity column；When file adopts sliced fashion to upload, by in the data block of each burst write into Databasce accumulation layer of file, and set up file index log respectively at each burst that database index layer is file, offset in original of the document size of each burst of file, file fragmentation, file fragmentation are terminated in the corresponding row of row race of the file index log of the information write file fragmentation of mark.
After all bursts of file complete to upload, the method of file process also includes: all bursts of file are carried out merger operation, and delete the file index log that all bursts are corresponding, and the file index log that file after database index layer generates merger is corresponding.
In one embodiment, the method for file process also includes: file index log corresponding to file that read requests is downloaded, if the value of the file content row of file index log is non-NULL, then directly downloads file from file content row；If the value of the file content row of file index log is empty, then read the value of the store path row of file index log, the data block according to the value location storage file of store path row, download file from this data base.
In one embodiment, the method of file process includes: carry out the operation of traversal file directory according to file index log, specifically include: receive traversing directories request, user identifier and file directory in being asked by traversing directories are mated with the log identification information of file index log, if coupling, file in the index record identify this log identification information belongs to this document catalogue, returns the fileinfo in the index record that this log identification information identifies.
In one embodiment, the operation creatied directory is carried out according to file index log, specifically include: receive the request of createing directory, the file directory that user identifier in the request of createing directory and needs create is retrieved in file index log as search condition, if retrieving record, then there is the file directory needing to create, do not perform the operation creatied directory；If not arriving retrieval record, then set up the file index log of the correspondence that needs to create directory, and be whether that the value of row of catalogue is set to very by file index log.
In one embodiment, the method of file process also includes: carry out the operation deltreed according to file index log, specifically include: receive the request of deltreeing, the file directory that user identifier in the request of deltreeing and needs are deleted is retrieved in file index log as search condition, if retrieving, then delete the file index log retrieved.
In one embodiment, the method of file process also includes the operation carrying out renaming catalogue according to file index log, specifically include: receive renaming catalog request, obtain the user identifier in renaming catalog request, file directory newname and file directory Old Name, user identifier and file directory Old Name are retrieved in file index log as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；Set up the file index log that file directory newname is corresponding, copy the log identification information of the file index log stored to file index log that the file directory newname set up is corresponding with the row race being used for log file index information, and delete the file index log that the file directory Old Name retrieved is corresponding.
In one embodiment, the method of file process also includes: carry out the operation of copied files according to file index log, specifically include: receive copied files request, obtain the user identifier in copied files request, the new route of file and the original route of file, original route according to user identifier and file is retrieved in file index log as search condition, if retrieving, then store the log identification information of file index log and be used for the row race of log file index information, and set up corresponding file index log according to user's indications with the new route of file, the log identification information of the file index log stored is copied to file index log that the file new route set up is corresponding with the row race being used for log file index information.
In one embodiment, the method of file process also includes: carry out deleting the operation of file according to file index log, specifically include: receive and delete file request, the file path that user identifier in the request of deltreeing and needs are deleted is retrieved in file index log as search condition, if retrieving, then delete the file index log retrieved.
In one embodiment, the method of file process also includes the operation carrying out Rename file according to file index log, specifically include: receive Rename file request, obtain user's indications, file newname and file Old Name, user identifier and file Old Name are retrieved in file index log as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；Set up the file index log that file newname is corresponding, the log identification information of the file index log stored is copied to file index log that the file newname set up is corresponding with the row race being used for log file index information, and the value of file index log file name column is updated to the file newname got, and delete the file index log that file Old Name is corresponding.
In one embodiment, the method of file process also includes the operation moving file according to file index log, specifically include: receive the request of mobile file, obtain the information of the original route of user identifier, the new route of file and file, the original route of user identifier and file is retrieved in file index log as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；Set up the file index log that the new route of file is corresponding, the log identification information of the file index log stored is copied to the row race being used for log file index information the file index log that the new route of the file set up is corresponding, and deletes the file index log retrieved.
An aspect according to embodiments of the present invention, it is provided that the device of a kind of file process, including: files passe module, it is used for receiving files passe request；Index process module, sets up file index log for the file uploaded for request at database index layer；Wherein, file index log includes log identification information and the row race for log file index information.
In one embodiment, index process module, specifically for: if the document size uploaded of request is less than default small documents threshold value, the file content at file index log arranges the content of storage this document；If the document size uploaded of request is more than or equal to default small documents threshold value, the value that the file content of file index log arranges is set to sky, the file that request is uploaded is stored in the data block of database purchase layer, and the store path of the store path row log file at file index log.
In one embodiment, files passe module, specifically for when transmitting file in the mode adopting non-burst, according to non-slicing files size, judge that file takies the number block_count of memory block:if during block_count > 1, then obtain untapped data block by the namenode NameNode of database purchase layer, and in data block, write file, If block_count=1 and document size file_size=block size block_size, then obtain untapped data block by the namenode NameNode of database purchase layer, and to written document in data block, If block_count=1 and file_size <, block_size,, according to the byte length unused_size information that not yet stores data of the data block less than data block concordance list in database index layer, searches less than data block, if less than the unused_size > in data block, file_size, then be written to this less than in data block by the file content of non-slicing files.
In one embodiment, the file content of non-slicing files is written to this less than after in data block, index process module, it is additionally operable to update the unused_size information less than data block concordance list less than data block, if update after the unused_size value less than data block be 0, then delete less than data block corresponding less than data block concordance list.
In one embodiment, the row race of file index log includes the document size row of each burst, the file fragmentation offset row in original, file fragmentation terminates identity column；Files passe module, is additionally operable to when file adopts sliced fashion to upload, by the data block of each burst write into Databasce accumulation layer of file；Index process module, it is additionally operable to each burst database index layer is file and sets up file index log respectively, offset in original of the document size of each burst of file, file fragmentation, file fragmentation are terminated in the corresponding row of row race of the file index log of the information write file fragmentation of mark.
In one embodiment, after all bursts of file complete to upload, index process module is additionally operable to all bursts of file are carried out merger operation, and deletes the file index log that all bursts are corresponding, and generates, at database index layer, the file index log that the file after merger is corresponding.
In one embodiment, the device of file process also includes: file download module, for the file index log that the file of read requests download is corresponding, if the value of the file content row of file index log is non-NULL, then directly downloads file from file content row；If the value of the file content row of file index log is empty, then read the value of the store path row of file index log, the data block according to the value location storage file of store path row, download file from this data base.
In one embodiment, the device of file process also includes: directory traversal module, for receive traversing directories request in case obtain traversing directories request in user identifier and file directory；Index process module, mate with the log identification information of file index log for the user identifier during traversing directories is asked and file directory, if coupling, file in the index record identify this log identification information belongs to this document catalogue, returns the fileinfo in the index record that this log identification information identifies.
In one embodiment, the device of file process also includes: directory creating module, for receiving the request of createing directory to obtain and create directory the user identifier in asking and need the file directory created；Index process module, it is additionally operable to the user identifier in the request of createing directory and needs the file directory created to retrieve in file index log as search condition, if retrieving record, then there is the file directory needing to create, do not perform the operation creatied directory；If not arriving retrieval record, whether index process module, for setting up the file index log of the correspondence that needs to create directory, and be that the value of row of catalogue is set to very by file index log.
In one embodiment, the device of file process also includes: directory delete module, for receiving the request of deltreeing to obtain the user identifier in the request of deltreeing and to need the file directory deleted；Index process module, being additionally operable to the user identifier in the request of deltreeing and needs the file directory deleted to retrieve in file index log as search condition, if retrieving, being additionally operable to delete the file index log retrieved.
In one embodiment, the device of file process also includes: catalogue renamer module, receives renaming catalog request so that the user identifier obtained in renaming catalog request, file directory newname and file directory Old Name；Index process module, it is additionally operable in file index log, retrieve user identifier and file directory Old Name as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that file directory newname is corresponding, copy the log identification information of the file index log stored to file index log that the file directory newname set up is corresponding with the row race being used for log file index information, and delete the file index log that the file directory Old Name retrieved is corresponding.
In one embodiment, the device of file process also includes: file copy module, for receiving copied files request so that the original route of the user identifier, the new route of file and the file that obtain in copied files request；Index process module, it is additionally operable to the original route according to user identifier and file retrieve in file index log as search condition, if retrieving, then store the log identification information of file index log and be used for the row race of log file index information, and set up corresponding file index log according to user's indications with the new route of file, copy the log identification information of the file index log stored to file index log that the file new route set up is corresponding with the row race being used for log file index information.
In one embodiment, the device of file process also includes: file removing module, deletes file request to obtain and deltree the user identifier in asking and need the file path deleted for receiving；Index process module, being additionally operable to according to the user identifier in the request of deltreeing and needs the file path deleted to retrieve in file index log as search condition, if retrieving, then deleting the file index log retrieved.
In one embodiment, the device of file process also includes: file renamer module, is used for receiving Rename file and asks to obtain user's indications, file newname and file Old Name；Index process module, it is additionally operable in file index log, retrieve user identifier and file Old Name as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that file newname is corresponding, the log identification information of the file index log stored is copied to file index log that the file newname set up is corresponding with the row race being used for log file index information, and the value of file index log file name column is updated to the file newname got, and delete the file index log that file Old Name is corresponding.
In one embodiment, the device of file process also includes: file mobile module, for receiving the request of mobile file to obtain the information of the original route of user identifier, the new route of file and file；Index process module, it is additionally operable in file index log, retrieve the original route of user identifier and file as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that the new route of file is corresponding, the log identification information of the file index log stored is copied to the row race being used for log file index information the file index log that the new route of the file set up is corresponding, and deletes the file index log retrieved.
The embodiment of the present invention at least has the advantage that
On the one hand, set up file index log at database index layer, utilize the file index information wherein recorded can position file, instead of the partial task of NameNode, greatly alleviate the burden of NameNode.
Another further aspect, different size according to file adopts different access methods, small documents is stored in the file content row of file index log, can directly read the content of small documents from file index log, it is achieved that the efficient access of file particularly small documents.
Another aspect, set up less than data block concordance list at database index layer, without the data block not being filled with data of the direct location database accumulation layer of Namenode node, file can be stored in the data block not being filled with data, alleviates the burden of NameNode further when upper transmitting file.
By referring to the accompanying drawing detailed description to the exemplary embodiment of the present invention, the further feature of the present invention and advantage thereof will be made apparent from.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, the accompanying drawing used required in embodiment or description of the prior art will be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the premise not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 illustrates the principle schematic of the scheme of the file process of the present invention.
Fig. 2 illustrates the schematic flow sheet of an embodiment of the method for the file process of the present invention.
Fig. 3 illustrates the structural representation of an embodiment of the device of the file process of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is only a part of embodiment of the present invention, rather than whole embodiments.Description only actually at least one exemplary embodiment is illustrative below, never as any restriction to the present invention and application or use.Based on the embodiment in the present invention, the every other embodiment that those of ordinary skill in the art obtain under not making creative work premise, broadly fall into the scope of protection of the invention.
Unless specifically stated otherwise, the parts otherwise set forth in these embodiments and positioned opposite, the numerical expression of step and numerical value do not limit the scope of the invention.
The known technology of person of ordinary skill in the relevant, method and apparatus are likely to be not discussed in detail, but in the appropriate case, described technology, method and apparatus should be considered to authorize a part for description.
Shown here with in all examples discussed, any occurrence should be construed as merely exemplary, not as restriction.Therefore, other example of exemplary embodiment can have different values.
It should also be noted that similar label and letter below figure represent similar terms, therefore, once a certain Xiang Yi accompanying drawing is defined, then it need not be further discussed in accompanying drawing subsequently.
Fig. 1 is the principle schematic of the scheme of the file process of the present invention, with reference to Fig. 1, inventors herein proposes and sets up file index log at database index layer and carry out the scheme of file process:
At client layer, the access end as file is responsible for, according to user, the operation of file is generated corresponding request, and the request of generation is sent to Access Layer.Wherein, the action type of file is mainly included two classes by user: first kind operation is user side to the uploading of file, down operation；Equations of The Second Kind operation be user travel through file directory, create directory, deltree, renaming catalogue, copied files, deletion file, Rename file, mobile file.Wherein, the request generated can be such as HTML (Hypertext Markup Language) (HTTP, Hypertexttransferprotocol) request or Hyper text transfer security protocol (HTTPS, HypertextTransferProtocoloverSecureSocketLayer) request.
At Access Layer, it is responsible for receiving the file operation requests of mass users, from file operation requests, parses the corresponding action type of user, file operation requests is selected according to load balancing and is transmitted to respective server.Access Layer mainly includes load equalizer, uploads/download server and document management server (or claiming CGI proxy server).Upload/download server, for processing user side uploading and down operation file, and carry out with the file index log in database index layer that file index information is mutual and to carry out reading and writing data mutual with database purchase layer.Document processing server, for process user side traversal file directory, create directory, deltree, the operation such as copied files, deletion file, Rename file, and interact with the file index log in database index layer.Load equalizer, being used for the system that is responsible for provides load balancing, for the corresponding corresponding server of user operation request selecting.Wherein, receive the file operation requests of user at Access Layer after, it is also possible to the file operation requests of user carries out the step of authentication process, if authentication does not pass through, then returning information to user side, the authentication of the file operation requests of prompting user does not pass through；If authentication is passed through, then continue the operation to user file.
At database index layer, set up file index log and less than data block concordance list etc., it is provided that the index record information of mass file, the quick-searching function of index record is provided, can quickly position file, substitute the partial task of NameNode, greatly alleviate the burden of NameNode.
At database purchase layer, it is made up of the storage based on HDFS file system, it is provided that the read-write of data block and data block fault tolerant service.
Fig. 2 is the flow chart of an embodiment of the method for the file process of the present invention.With reference to Fig. 2, the method for the file process of the present invention includes:
S102, receives the files passe request sent.
S104, for asking the file uploaded to set up file index log at database index layer, wherein, file index log includes log identification information and the row race for log file index information.
The embodiment of the present invention, by setting up file index log at database index layer, utilizes the file index information wherein recorded can position file, instead of the partial task of NameNode, greatly alleviate the burden of NameNode.
Table 1 is an example of file index log.Reference table 1, file index log can comprise log identification information and the row race for log file index information.Wherein, log identification information can include user identifier and file directory etc..Row race such as may include that MD5 value, document size, file creation time, file update time, file name, file content, whether are that catalogue, store path arrange, are whether that file fragmentation, the file fragmentation deviant in original, file fragmentation terminate mark etc..
Wherein, file content row can directly store small documents.The data block list information of the store path row row record storage file in file index log, the train value of data block store path row dynamically generates after being made requests on to the namenode NameNode in database purchase layer by database index layer, wherein, the information of each data block can be carried out labelling by the format value of ID_offset_length, ID is data block identifier number, offset log file content-data deposits deviation post, length log file content-data storage length within the data block in the data block represented by ID.
Table 1
In an embodiment of upper transmitting file, if the document size uploaded of request is less than default small documents threshold value, what the request of illustrating was uploaded is small documents, the content directly storing this document is then arranged at the file content of file index log, if the document size uploaded of request is more than or equal to default small documents threshold value, the value that the file content of file index log arranges is set to sky, the file that request is uploaded is stored in the data block of database purchase layer, and the store path of the store path row log file at file index log.The threshold value of its small file can pre-set, for instance could be arranged to 20K byte.
In the embodiment downloading file, when small documents is downloaded, it is possible to file index log corresponding to file that read requests is downloaded, if the value of the file content row of file index log is non-NULL, then asking the file downloaded is small documents, then directly download file from file content row；If the value of the file content row of file index log is empty, illustrate that the file that request is downloaded is not small documents, then read the value of the store path row of file index log, the data block of the value location storage file according to store path row, downloading file from this data base, wherein the value of store path row can include the storage ID of data block of file, deviation post offset, file size length information.
Also set up less than data block concordance list at database index layer, table 2 is less than data block concordance list, reference table 2, mark ID less than data block concordance list record data block, unused_size list shows in data block that not yet storage has the information of the byte length of data, and the currently whether occupied operation of this data block is shown in isUsed list.
Table 2
In files passe, can generally adopt transmitting file in the mode of file fragmentation or the mode of non-burst according to file size.
In one embodiment, the method that the file of non-burst is uploaded by the basis of the present invention less than data block concordance list may include that
When transmitting file in the mode adopting non-burst, the size according to file, it is judged that file takies the number block_count of memory block:
If during block_count > 1, then obtain untapped data block by the namenode NameNode of database purchase layer, and in the data block obtained from database purchase layer, write file.
If block_count=1 and document size file_size=block size block_size, then obtain untapped data block by the namenode NameNode of database purchase layer, and to written document in the data block obtained from database purchase layer.
If block_count=1 and file_size <block_size,, according to the unused_size information of the data block less than data block concordance list in database index layer, searches less than data block, if should be less than the unused_size > in data block; File_size, then be written to this less than in data block by the file content of this document, and this is updated less than the ID of data block, deviation post offset, file size information length in the train value of data block path column of the file index log of file.
In one embodiment, by the files passe of non-burst to provide less than data block concordance list less than after data block, update the unused_size information less than data block concordance list less than data block, record the data block residue after the file storing non-burst and do not store the byte length of data, if update after the unused_size value less than data block be 0, then delete less than data block corresponding less than data block concordance list.
In order to improve file transmission efficiency, when transmitting big file, it is possible to adopt transmitting file in the mode of file fragmentation, it is explained in detail below according to the method uploaded by the mode of file fragmentation.
When file adopts sliced fashion to upload, by in the data block of each burst write into Databasce accumulation layer of file, and set up file index log respectively at each burst that database index layer is file, deviant in original of the document size of each burst of file, file fragmentation, file fragmentation are terminated in the corresponding row of file index log corresponding to the information write file fragmentation of mark.After completing the uploading of All Files burst, All Files burst is carried out merger operation, delete the file index log that all bursts are corresponding, and the file index log that file after database index layer generates merger is corresponding.
The scheme that according to file index log carry out file Equations of The Second Kind operation of the present invention is explained in detail below.
In one embodiment, carry out the operation of traversal file directory according to file index log, specifically include:
Receive the traversing directories request that user side sends, user identifier and file directory is obtained from the traversing directories request that user side sends, and according to user identifier and file directory, mate with the log identification information of file index log, if coupling, file in the index record that then this log identification information identifies belongs to this document catalogue, returns the fileinfo in the index record that this log identification information identifies to user side.
In one embodiment, carry out the operation of traversal file directory according to file index log, specifically include:
Receive the request of createing directory that user side sends, obtain the user identifier in the request of createing directory and need the file directory created, the file directory that user identifier in the request of createing directory and needs create is retrieved in file index log as search condition, if retrieving record, then there is the file directory needing to create, do not perform the operation creatied directory.Whether if not arriving retrieval record, then set up the file index log of the correspondence that needs to create directory, and be that the value of row of catalogue is set to very by file index log, the value of other row could be arranged to sky.
In one embodiment, carry out the operation deltreed according to file index log, specifically include:
Receive the request of deltreeing that user side sends, parse user identifier in the request of deltreeing and need the file directory deleted, retrieve in file index log as search condition according to the file directory that user identifier and needs are deleted, if retrieving the file index log of correspondence, then delete the file index log retrieved.
In one embodiment, carry out the operation of renaming catalogue according to file index log, specifically include:
Receive the renaming catalog request that user side sends, obtain the user identifier in renaming catalog request, file directory newname and file directory Old Name, user identifier and file directory Old Name are retrieved in file index log as search condition, if retrieving the file index log of correspondence, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；Set up the file index log that file directory newname is corresponding, copy the log identification information of the file index log stored to file index log that the file directory newname set up is corresponding with the row race being used for log file index information, and delete the file index log that the file directory Old Name retrieved is corresponding.
In one embodiment, carry out the operation of copied files according to file index log, specifically include:
Receive the copied files request that user side sends, obtain the user identifier in copied files request, the new route of file and the original route of file, original route according to user identifier and file is retrieved in file index log as search condition, if retrieving the file index log of correspondence, then store the log identification information of file index log and be used for the row race of log file index information, and set up corresponding file index log according to user's indications with the new route of file, the log identification information of the file index log stored is copied to file index log that the file new route set up is corresponding with the row race being used for log file index information.
In one embodiment, carry out deleting the operation of file according to file index log, specifically include:
Receive the deletion file request that user side sends, the file path that user identifier in the request of deltreeing and needs are deleted is retrieved in file index log as search condition, if retrieving the file index log needing to delete, then delete the file index log retrieved.
In one embodiment, carry out the operation of Rename file according to file index log, specifically include:
Receive the Rename file request that user side sends, obtain user's indications, file newname and file Old Name, user identifier and file Old Name are retrieved in file index log as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；Set up the file index log that file newname is corresponding, the log identification information of the file index log stored is copied to file index log that the file newname set up is corresponding with the row race being used for log file index information, and the value of file index log file name column is updated to the file newname got, and delete the file index log that file Old Name is corresponding.
In one embodiment, move the operation of file according to file index log, specifically include:
Receive the request of the mobile file that user side sends, obtain the information of the original route of user identifier, the new route of file and file, the original route of user identifier and file is retrieved in file index log as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；Set up the file index log that the new route of file is corresponding, the log identification information of the file index log stored is copied to the row race being used for log file index information the file index log that the new route of the file set up is corresponding, and deletes the file index log retrieved.
In one embodiment, before the scheme carrying out Equations of The Second Kind operation according to file index log of the present invention, it is also possible to include the process of the authentication process of the operation requests to user, if authentication does not pass through, then returning information to user side, prompting authentication does not pass through；If authentication is passed through, then continue that user in Equations of The Second Kind operation travels through file directory, creaties directory, deltrees, renaming catalogue, copied files, deletion file, Rename file, mobile file etc. is for the operation of file management.
Fig. 3 is the structural representation of an embodiment of the device of the file process of the present invention, and with reference to Fig. 3, the device 200 of file process includes:
Files passe module 204, is used for receiving files passe request.
Index process module 202, sets up file index log for the file uploaded for request at database index layer.
Wherein, file index log includes log identification information and the row race for log file index information.
In one embodiment, index process module 202, specifically for: if the document size uploaded of request is less than default small documents threshold value, the file content at file index log arranges the content of storage this document；If the document size uploaded of request is more than or equal to default small documents threshold value, the value that the file content of file index log arranges is set to sky, the file that request is uploaded is stored in the data block of database purchase layer, and the store path of the store path row log file at file index log.
In one embodiment, files passe module 204, specifically for when transmitting file in the mode adopting non-burst, according to non-slicing files size, judge that file takies the number block_count of memory block: if during block_count > 1, then obtain untapped data block by the namenode NameNode of database purchase layer, and in data block, write file；
If block_count=1 and document size file_size=block size block_size, then obtain untapped data block by the namenode NameNode of database purchase layer, and to written document in data block; If block_count=1 and file_size <block_size,, according to the byte length unused_size information that not yet stores data of the data block less than data block concordance list in database index layer, searches less than data block, if less than the unused_size > in data block; File_size, then be written to this less than in data block by the file content of non-slicing files.
In one embodiment, the file content of non-slicing files is written to this less than after in data block, index process module 202, it is additionally operable to update the unused_size information less than data block concordance list less than data block, if update after the unused_size value less than data block be 0, then delete less than data block corresponding less than data block concordance list.
In one embodiment, the row race of file index log includes the document size row of each burst, the file fragmentation offset row in original, file fragmentation terminates identity column.
Files passe module 204, is additionally operable to when file adopts sliced fashion to upload, by the data block of each burst write into Databasce accumulation layer of file.
Index process module 202, it is additionally operable to each burst database index layer is file and sets up file index log respectively, offset in original of the document size of each burst of file, file fragmentation, file fragmentation are terminated in the corresponding row of row race of the file index log of the information write file fragmentation of mark.
In one embodiment, after all bursts of file complete to upload, index process module 204 is additionally operable to all bursts of file are carried out merger operation, and deletes the file index log that all bursts are corresponding, and generates, at database index layer, the file index log that the file after merger is corresponding.
In one embodiment, file download module 206, for the file index log that the file of read requests download is corresponding, if the value of the file content row of file index log is non-NULL, then directly download file from file content row；If the value of the file content row of file index log is empty, then read the value of the store path row of file index log, the data block according to the value location storage file of store path row, download file from this data base.
In one embodiment, the device of file process also includes:
Directory traversal module 208, for receive traversing directories request in case obtain traversing directories request in user identifier and file directory.
Index process module, mate with the log identification information of file index log for the user identifier during traversing directories is asked and file directory, if coupling, file in the index record identify this log identification information belongs to this document catalogue, returns the fileinfo in the index record that this log identification information identifies.
In one embodiment, the device of file process also includes:
Directory creating module 210, for receiving the request of createing directory to obtain and create directory the user identifier in asking and need the file directory created.
Index process module 202, it is additionally operable to the user identifier in the request of createing directory and needs the file directory created to retrieve in file index log as search condition, if retrieving record, then there is the file directory needing to create, do not perform the operation creatied directory；If not arriving retrieval record, whether index process module, for setting up the file index log of the correspondence that needs to create directory, and be that the value of row of catalogue is set to very by file index log.
In one embodiment, the device of file process also includes:
Directory delete module 212, for the reception request of deltreeing, in order to obtains the user identifier in catalog request and needs the file directory deleted.
Index process module 202, being additionally operable to the user identifier in the request of deltreeing and needs the file directory deleted to retrieve in file index log as search condition, if retrieving, being additionally operable to delete the file index log retrieved.
In one embodiment, the device of file process also includes:
Catalogue renamer module 214, is used for receiving renaming catalog request, in order to obtain the user identifier in renaming catalog request, file directory newname and file directory Old Name.
Index process module 202, it is additionally operable in file index log, retrieve user identifier and file directory Old Name as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that file directory newname is corresponding, copy the log identification information of the file index log stored to file index log that the file directory newname set up is corresponding with the row race being used for log file index information, and delete the file index log that the file directory Old Name retrieved is corresponding.
In one embodiment, the device of file process also includes:
File copy module 216, is used for receiving copied files request, in order to obtain the original route of the user identifier in copied files request, the new route of file and file.
Index process module 202, it is additionally operable to the original route according to user identifier and file retrieve in file index log as search condition, if retrieving, then store the log identification information of file index log and be used for the row race of log file index information, and set up corresponding file index log according to user's indications with the new route of file, copy the log identification information of the file index log stored to file index log that the file new route set up is corresponding with the row race being used for log file index information.
In one embodiment, the device of file process also includes:
File removing module 218, is used for receiving deletion file request, in order to obtains the user identifier in the request of deltreeing and needs the file path deleted.
Index process module 202, being additionally operable to according to the user identifier in the request of deltreeing and needs the file path deleted to retrieve in file index log as search condition, if retrieving, then deleting the file index log retrieved.
In one embodiment, the device of file process also includes:
File renamer module 220, is used for receiving Rename file request, in order to obtain user's indications, file newname and file Old Name.
Index process module 202, it is additionally operable in file index log, retrieve user identifier and file Old Name as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that file newname is corresponding, the log identification information of the file index log stored is copied to file index log that the file newname set up is corresponding with the row race being used for log file index information, and the value of file index log file name column is updated to the file newname got, and delete the file index log that file Old Name is corresponding.
In one embodiment, the device of file process also includes:
File mobile module 222, for receiving the request of mobile file, in order to obtains the information of the original route of user identifier, the new route of file and file.
Index process module 202, it is additionally operable in file index log, retrieve the original route of user identifier and file as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that the new route of file is corresponding, the log identification information of the file index log stored is copied to the row race being used for log file index information the file index log that the new route of the file set up is corresponding, and deletes the file index log retrieved.
One of ordinary skill in the art will appreciate that all or part of step realizing above-described embodiment can be completed by hardware, can also be completed by the hardware that program carrys out instruction relevant, described program can be stored in a kind of computer-readable recording medium, storage medium mentioned above can be read only memory, disk or CD etc..
The foregoing is only presently preferred embodiments of the present invention, not in order to limit the present invention, all within the spirit and principles in the present invention, any amendment of making, equivalent replacement, improvement etc., should be included within protection scope of the present invention.
Claims (30)
1. the method for a file process, it is characterised in that described method includes:
Reception files passe is asked；
For asking the file uploaded to set up file index log at database index layer；
Wherein, described file index log includes log identification information and the row race for log file index information.
2. method according to claim 1, it is characterised in that described is ask the file uploaded to set up file index log at database index layer to include:
If the document size uploaded of request is less than default small documents threshold value, the file content at file index log arranges the content of storage this document；
If the document size uploaded of request is more than or equal to default small documents threshold value, the value that the file content of file index log arranges is set to sky, the file that request is uploaded is stored in the data block of database purchase layer, and the store path of the store path row log file at file index log.
3. method according to claim 2, it is characterised in that the described data block that the file that request is uploaded stores accumulation layer includes:
When transmitting file in the mode adopting non-burst, according to non-slicing files size, it is judged that file takies the number block_count of memory block:
If during block_count > 1, then obtain untapped data block by the namenode NameNode of database purchase layer, and in described data block, write file；
If block_count=1 and document size file_size=block size block_size, then obtain untapped data block by the namenode NameNode of database purchase layer, and to written document in described data block；
If block_count=1 and file_size <block_size,, according to the byte length unused_size information that not yet stores data of the data block less than data block concordance list in database index layer, searches less than data block, if less than the unused_size > in data block; File_size, then be written to this less than in data block by the file content of non-slicing files.
4. method according to claim 3, it is characterised in that the file content of non-slicing files being written to this less than after in data block, described method also includes:
Update the described unused_size information less than data block concordance list less than data block, if the unused_size value less than data block after updating is 0, then described in deleting less than data block corresponding less than data block concordance list.
5. method according to claim 1, it is characterised in that
The row race of described file index log includes the document size row of each burst, the file fragmentation offset row in original, file fragmentation terminates identity column；
When described file adopts sliced fashion to upload, by in the data block of each burst write into Databasce accumulation layer of file, and set up file index log respectively at each burst that database index layer is file, offset in original of the document size of each burst of file, file fragmentation, file fragmentation are terminated the information of mark write described file fragmentation file index log row race corresponding row in.
6. method according to claim 5, it is characterised in that after all bursts of file complete to upload, described method also includes:
All bursts of described file are carried out merger operation, and deletes the file index log that all bursts are corresponding, and generate, at database index layer, the file index log that the file after merger is corresponding.
7. method according to claim 1, it is characterised in that described method also includes:
File index log corresponding to file that read requests is downloaded, if the value of the file content row of described file index log is non-NULL, then directly downloads file from described file content row；
If the value of the file content row of described file index log is empty, then read the value of the store path row of described file index log, the data block according to the value location storage file of store path row, download file from this data base.
8. method according to claim 1, it is characterised in that described method includes: carry out the operation of traversal file directory according to described file index log, specifically include:
Reception traversing directories is asked, user identifier and file directory in being asked by described traversing directories are mated with the log identification information of file index log, if coupling, file in the index record identify this log identification information belongs to this document catalogue, returns the fileinfo in the index record that this log identification information identifies.
9. method according to claim 1, it is characterised in that described method also includes:
Carry out the operation creatied directory according to described file index log, specifically include:
The reception request of createing directory, retrieves the file directory that the user identifier in the request of createing directory and needs create as search condition in file index log,
If retrieving record, then there is the file directory needing to create, do not perform the operation creatied directory；
If not arriving retrieval record, then set up the file index log of the correspondence that needs to create directory, and be whether that the value of row of catalogue is set to very by described file index log.
10. method according to claim 1, it is characterised in that described method also includes carrying out, according to described file index log, the operation that deltrees, specifically includes:
The reception request of deltreeing, retrieves the file directory that the user identifier in the request of deltreeing and needs are deleted as search condition in file index log,
If retrieving, then delete the file index log retrieved.
11. method according to claim 1, it is characterised in that described method also includes the operation carrying out renaming catalogue according to described file index log, specifically includes:
Receive renaming catalog request, obtain the user identifier in renaming catalog request, file directory newname and file directory Old Name, user identifier and file directory Old Name are retrieved as search condition in file index log,
If retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；
Set up the file index log that file directory newname is corresponding, copy the log identification information of the file index log stored to file index log that the file directory newname set up is corresponding with the row race being used for log file index information, and delete the file index log that the file directory Old Name retrieved is corresponding.
12. method according to claim 1, it is characterised in that described method also includes the operation carrying out copied files according to described file index log, specifically includes:
Reception copied files is asked, and obtains the original route of the user identifier in copied files request, the new route of file and file, retrieves as search condition according to the original route of user identifier and file in file index log,
If retrieving, then store the log identification information of file index log and be used for the row race of log file index information, and set up corresponding file index log according to user's indications with the new route of file, copy the log identification information of the file index log stored to file index log that the file new route set up is corresponding with the row race being used for log file index information.
13. method according to claim 1, it is characterised in that described method also includes:
Carry out deleting the operation of file according to described file index log, specifically include:
Receive and delete file request, the file path that the user identifier in the described request of deltreeing and needs are deleted is retrieved as search condition in file index log,
If retrieving, then delete the file index log retrieved.
14. method according to claim 1, it is characterised in that described method also includes the operation carrying out Rename file according to described file index log, specifically includes:
Reception Rename file is asked, and obtains user's indications, file newname and file Old Name, user identifier and file Old Name is retrieved in file index log as search condition,
If retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；
Set up the file index log that file newname is corresponding, the log identification information of the file index log stored is copied to file index log that the file newname set up is corresponding with the row race being used for log file index information, and the value of file index log file name column is updated to the file newname got, and delete the file index log that file Old Name is corresponding.
15. method according to claim 1, it is characterised in that described method also includes the operation moving file according to described file index log, specifically includes:
Receive the request of mobile file, obtain the information of the original route of user identifier, the new route of file and file, the original route of user identifier and file is retrieved as search condition in file index log,
If retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；
Set up the file index log that the new route of file is corresponding, the log identification information of the file index log stored is copied to the row race being used for log file index information the file index log that the new route of the file set up is corresponding, and deletes the file index log retrieved.
16. the device of a file process, it is characterised in that described device includes:
Files passe module, is used for receiving files passe request；
Index process module, sets up file index log for the file uploaded for request at database index layer；
Wherein, described file index log includes log identification information and the row race for log file index information.
17. device according to claim 16, it is characterised in that described index process module, specifically for:
If the document size uploaded of request is less than default small documents threshold value, the file content at file index log arranges the content of storage this document；
If the document size uploaded of request is more than or equal to default small documents threshold value, the value that the file content of file index log arranges is set to sky, the file that request is uploaded is stored in the data block of database purchase layer, and the store path of the store path row log file at file index log.
18. device according to claim 17, it is characterised in that
Files passe module, specifically for when transmitting file in the mode adopting non-burst, according to non-slicing files size, it is judged that file takies the number block_count of memory block:
If during block_count > 1, then obtain untapped data block by the namenode NameNode of database purchase layer, and in described data block, write file；
If block_count=1 and document size file_size=block size block_size, then obtain untapped data block by the namenode NameNode of database purchase layer, and to written document in described data block；
If block_count=1 and file_size <block_size,, according to the byte length unused_size information that not yet stores data of the data block less than data block concordance list in database index layer, searches less than data block, if less than the unused_size > in data block; File_size, then be written to this less than in data block by the file content of non-slicing files.
19. device according to claim 18, it is characterized in that, the file content of non-slicing files is written to this less than after in data block, described index process module, it is additionally operable to update the described unused_size information less than data block concordance list less than data block, if update after the unused_size value less than data block be 0, then delete described in less than data block corresponding less than data block concordance list.
20. device according to claim 16, it is characterised in that
The row race of described file index log includes the document size row of each burst, the file fragmentation offset row in original, file fragmentation terminates identity column；
Described files passe module, is additionally operable to when described file adopts sliced fashion to upload, by the data block of each burst write into Databasce accumulation layer of file；
Described index process module, be additionally operable to each burst database index layer is file and set up file index log respectively, offset in original of the document size of each burst of file, file fragmentation, file fragmentation are terminated the information of mark write described file fragmentation file index log row race corresponding row in.
21. device according to claim 16, it is characterised in that after all bursts of file complete to upload,
Index process module is additionally operable to all bursts of described file are carried out merger operation, and deletes the file index log that all bursts are corresponding, and generates, at database index layer, the file index log that the file after merger is corresponding.
22. device according to claim 16, it is characterised in that described device also includes:
File download module, for the file index log that the file of read requests download is corresponding, if the value of the file content row of described file index log is non-NULL, then directly downloads file from described file content row；If the value of the file content row of described file index log is empty, then read the value of the store path row of described file index log, the data block according to the value location storage file of store path row, download file from this data base.
23. device according to claim 16, it is characterised in that described device also includes:
Directory traversal module, is used for receiving traversing directories request, in order to obtain the user identifier in traversing directories request and file directory；
Index process module, for the described log identification information with file index log is mated, if it does, the file in the index record identified by this log identification information belongs to this document catalogue, return the fileinfo in the index record that this log identification information identifies.
24. device according to claim 16, it is characterised in that described device also includes:
Directory creating module, for the reception request of createing directory, in order to obtains the user identifier in the request of createing directory and needs the file directory created；
Index process module, it is additionally operable to the user identifier in the request of createing directory and needs the file directory created to retrieve in file index log as search condition, if retrieving record, then there is the file directory needing to create, do not perform the operation creatied directory；If not arriving retrieval record, whether index process module, for setting up the file index log of the correspondence that needs to create directory, and be that the value of row of catalogue is set to very by described file index log.
25. device according to claim 16, it is characterised in that described device also includes:
Directory delete module, for the reception request of deltreeing, in order to obtains the user identifier in the request of deltreeing and needs the file directory deleted；
Index process module, being additionally operable to the user identifier in the request of deltreeing and needs the file directory deleted to retrieve in file index log as search condition, if retrieving, being additionally operable to delete the file index log retrieved.
26. device according to claim 16, it is characterised in that described device also includes:
Catalogue renamer module, receives renaming catalog request, in order to obtain the user identifier in renaming catalog request, file directory newname and file directory Old Name；
Index process module, it is additionally operable in file index log, retrieve user identifier and file directory Old Name as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that file directory newname is corresponding, copy the log identification information of the file index log stored to file index log that the file directory newname set up is corresponding with the row race being used for log file index information, and delete the file index log that the file directory Old Name retrieved is corresponding.
27. device according to claim 16, it is characterised in that described device also includes:
File copy module, is used for receiving copied files request, in order to obtain the original route of the user identifier in copied files request, the new route of file and file；
Index process module, it is additionally operable to the original route according to user identifier and file retrieve in file index log as search condition, if retrieving, then store the log identification information of file index log and be used for the row race of log file index information, and set up corresponding file index log according to user's indications with the new route of file, copy the log identification information of the file index log stored to file index log that the file new route set up is corresponding with the row race being used for log file index information.
28. device according to claim 16, it is characterised in that described device also includes:
File removing module, is used for receiving deletion file request, in order to obtains the user identifier deleted in file request and needs the file path deleted；
Index process module, being additionally operable to according to the user identifier deleted in file request and needs the file path deleted to retrieve in file index log as search condition, if retrieving, then deleting the file index log retrieved.
29. device according to claim 16, it is characterised in that described device also wraps
File renamer module, is used for receiving Rename file request, in order to obtain user's indications, file newname and file Old Name；
Index process module, it is additionally operable in file index log, retrieve user identifier and file Old Name as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that file newname is corresponding, the log identification information of the file index log stored is copied to file index log that the file newname set up is corresponding with the row race being used for log file index information, and the value of file index log file name column is updated to the file newname got, and delete the file index log that file Old Name is corresponding.
30. device according to claim 16, it is characterised in that described device also wraps
File mobile module, for receiving the request of mobile file, in order to obtains the information of the original route of user identifier, the new route of file and file；
Index process module, it is additionally operable in file index log, retrieve the original route of user identifier and file as search condition, if retrieving, then the log identification information of the file index log that memory scan arrives and the row race for log file index information；It is additionally operable to set up the file index log that the new route of file is corresponding, the log identification information of the file index log stored is copied to the row race being used for log file index information the file index log that the new route of the file set up is corresponding, and deletes the file index log retrieved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410849959.2A CN105808589A (en) | 2014-12-31 | 2014-12-31 | File processing method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410849959.2A CN105808589A (en) | 2014-12-31 | 2014-12-31 | File processing method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105808589A true CN105808589A (en) | 2016-07-27 |
Family
ID=56421208
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410849959.2A Pending CN105808589A (en) | 2014-12-31 | 2014-12-31 | File processing method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105808589A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106775459A (en) * | 2016-11-28 | 2017-05-31 | 深圳市中博睿存科技有限公司 | A kind of safe distribution file system framework based on FT 1500A processors |
| CN107577420A (en) * | 2017-07-26 | 2018-01-12 | 阿里巴巴集团控股有限公司 | Document handling method and device, server |
| CN107729570A (en) * | 2017-11-20 | 2018-02-23 | 北京百度网讯科技有限公司 | Data migration method and device for server |
| CN108650298A (en) * | 2018-04-10 | 2018-10-12 | 常州大学 | Cloud storage method towards gene sequencing big data |
| WO2018201583A1 (en) * | 2017-05-05 | 2018-11-08 | 平安科技（深圳）有限公司 | File management method and system, electronic device, and medium |
| CN109144948A (en) * | 2017-06-15 | 2019-01-04 | 海马云（天津）信息技术有限公司 | Method, apparatus, electronic equipment and the memory of application file positioning |
| WO2019149002A1 (en) * | 2018-02-05 | 2019-08-08 | 清华大学 | Capacity expansion method and system based on dual-layer list structure |
| CN110430480A (en) * | 2019-08-09 | 2019-11-08 | 深圳市迈岭信息技术有限公司 | A kind of system and method that Streaming Media cloud storage is synchronous |
| CN110569219A (en) * | 2019-09-02 | 2019-12-13 | 深信服科技股份有限公司 | Fragment garbage processing method, user equipment, storage medium and device |
| CN110888837A (en) * | 2019-11-15 | 2020-03-17 | 星辰天合（北京）数据科技有限公司 | Object storage small file merging method and device |
| CN111367855A (en) * | 2020-02-08 | 2020-07-03 | 汤忠武 | Method and device for moving out downloaded file, computer equipment and storage medium |
| CN111597149A (en) * | 2020-04-27 | 2020-08-28 | 五八有限公司 | Data cleaning method and device for database |
| CN112579726A (en) * | 2019-09-29 | 2021-03-30 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for managing index table |
| CN112637616A (en) * | 2020-12-08 | 2021-04-09 | 网宿科技股份有限公司 | Object storage method, system and server |
| CN112860660A (en) * | 2021-01-19 | 2021-05-28 | 奥特酷智能科技(南京)有限公司 | Unified management method for Rosbag file information |
| CN113051219A (en) * | 2019-12-26 | 2021-06-29 | 贵州白山云科技股份有限公司 | Database management method, device, equipment and storage medium |
| CN116860509A (en) * | 2023-09-04 | 2023-10-10 | 深圳麦风科技有限公司 | PST file repairing method, device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708107A (en) * | 2011-12-13 | 2012-10-03 | 北京安天电子设备有限公司 | Method and system for storing appended files |
| CN103207889A (en) * | 2013-01-31 | 2013-07-17 | 重庆大学 | Method for retrieving massive face images based on Hadoop |
| CN103544318A (en) * | 2013-11-05 | 2014-01-29 | 浪潮(北京)电子信息产业有限公司 | Document management method of cloud data center |
| CN104156376A (en) * | 2013-05-15 | 2014-11-19 | 腾讯科技（深圳）有限公司 | Storage method, device and server for file |
-
2014
- 2014-12-31 CN CN201410849959.2A patent/CN105808589A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708107A (en) * | 2011-12-13 | 2012-10-03 | 北京安天电子设备有限公司 | Method and system for storing appended files |
| CN103207889A (en) * | 2013-01-31 | 2013-07-17 | 重庆大学 | Method for retrieving massive face images based on Hadoop |
| CN104156376A (en) * | 2013-05-15 | 2014-11-19 | 腾讯科技（深圳）有限公司 | Storage method, device and server for file |
| CN103544318A (en) * | 2013-11-05 | 2014-01-29 | 浪潮(北京)电子信息产业有限公司 | Document management method of cloud data center |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106775459B (en) * | 2016-11-28 | 2020-06-02 | 北京同有飞骥科技股份有限公司 | Safe distributed file system based on FT-1500A processor |
| CN106775459A (en) * | 2016-11-28 | 2017-05-31 | 深圳市中博睿存科技有限公司 | A kind of safe distribution file system framework based on FT 1500A processors |
| WO2018201583A1 (en) * | 2017-05-05 | 2018-11-08 | 平安科技（深圳）有限公司 | File management method and system, electronic device, and medium |
| CN109144948B (en) * | 2017-06-15 | 2021-10-08 | 海马云（天津）信息技术有限公司 | Application file positioning method and device, electronic equipment and memory |
| CN109144948A (en) * | 2017-06-15 | 2019-01-04 | 海马云（天津）信息技术有限公司 | Method, apparatus, electronic equipment and the memory of application file positioning |
| CN107577420A (en) * | 2017-07-26 | 2018-01-12 | 阿里巴巴集团控股有限公司 | Document handling method and device, server |
| CN107577420B (en) * | 2017-07-26 | 2021-02-23 | 创新先进技术有限公司 | File processing method and device and server |
| CN107729570A (en) * | 2017-11-20 | 2018-02-23 | 北京百度网讯科技有限公司 | Data migration method and device for server |
| US11467754B2 (en) | 2018-02-05 | 2022-10-11 | Tsinghua University | Capacity expansion method and capacity expansion system based on dual-level list structure |
| WO2019149002A1 (en) * | 2018-02-05 | 2019-08-08 | 清华大学 | Capacity expansion method and system based on dual-layer list structure |
| CN108650298A (en) * | 2018-04-10 | 2018-10-12 | 常州大学 | Cloud storage method towards gene sequencing big data |
| CN110430480A (en) * | 2019-08-09 | 2019-11-08 | 深圳市迈岭信息技术有限公司 | A kind of system and method that Streaming Media cloud storage is synchronous |
| CN110430480B (en) * | 2019-08-09 | 2022-02-18 | 深圳市迈岭信息技术有限公司 | System and method for streaming media cloud storage synchronization |
| CN110569219A (en) * | 2019-09-02 | 2019-12-13 | 深信服科技股份有限公司 | Fragment garbage processing method, user equipment, storage medium and device |
| CN110569219B (en) * | 2019-09-02 | 2023-05-12 | 深信服科技股份有限公司 | Method, user equipment, storage medium and device for processing fragmented garbage |
| CN112579726A (en) * | 2019-09-29 | 2021-03-30 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for managing index table |
| CN110888837A (en) * | 2019-11-15 | 2020-03-17 | 星辰天合（北京）数据科技有限公司 | Object storage small file merging method and device |
| CN113051219A (en) * | 2019-12-26 | 2021-06-29 | 贵州白山云科技股份有限公司 | Database management method, device, equipment and storage medium |
| CN111367855B (en) * | 2020-02-08 | 2021-06-18 | 重庆节节高科技发展有限公司 | Method and device for moving out downloaded file, computer equipment and storage medium |
| CN111367855A (en) * | 2020-02-08 | 2020-07-03 | 汤忠武 | Method and device for moving out downloaded file, computer equipment and storage medium |
| CN111597149A (en) * | 2020-04-27 | 2020-08-28 | 五八有限公司 | Data cleaning method and device for database |
| CN111597149B (en) * | 2020-04-27 | 2023-03-31 | 五八有限公司 | Data cleaning method and device for database |
| CN112637616A (en) * | 2020-12-08 | 2021-04-09 | 网宿科技股份有限公司 | Object storage method, system and server |
| CN112637616B (en) * | 2020-12-08 | 2024-02-23 | 网宿科技股份有限公司 | Object storage method, system and server |
| CN112860660A (en) * | 2021-01-19 | 2021-05-28 | 奥特酷智能科技(南京)有限公司 | Unified management method for Rosbag file information |
| CN116860509A (en) * | 2023-09-04 | 2023-10-10 | 深圳麦风科技有限公司 | PST file repairing method, device and storage medium |
| CN116860509B (en) * | 2023-09-04 | 2023-12-26 | 深圳麦风科技有限公司 | PST file repairing method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105808589A (en) | File processing method and apparatus | |
| US11558485B2 (en) | Systems and methods for caching of managed content in a distributed environment using a multi-tiered architecture including off-line access to cached content | |
| US8997041B2 (en) | Method of managing script, server performing the same and storage media storing the same | |
| US11403262B2 (en) | Local networked storage linked to remote networked storage system | |
| CN103731483B (en) | Virtual file system based on cloud computing | |
| CN106484828B (en) | Distributed internet data rapid acquisition system and acquisition method | |
| CN103038742B (en) | For the method and system of Dynamical data replication in distributed memory system | |
| CN104516974B (en) | A kind of management method and device of file system directories item | |
| JP4491269B2 (en) | Storage system and file management apparatus | |
| CN107958079A (en) | Aggregate file delet method, system, device and readable storage medium storing program for executing | |
| CN104778192B9 (en) | Directory structure representing content addressable storage system | |
| CN102331957B (en) | File backup method and device | |
| KR20100067976A (en) | Method for synchronizing contents files stored separately | |
| CN108255638A (en) | A kind of snapshot rollback method and device | |
| Agrawal et al. | Mobile data sync in a blink | |
| CN105282238B (en) | A kind of method for managing resource | |
| CN105847306A (en) | Cloud storage system and method | |
| CN103701937A (en) | Method for uploading large files | |
| CN103188282A (en) | Method, system and equipment for file synchronization | |
| CN109660611B (en) | Data storage method for cloud backup and data cloud backup method for storage system | |
| KR20120028754A (en) | Apparatus and method for streaming contents | |
| CN109144786A (en) | The restoration methods and recovery system of small documents in packaging file | |
| JP2007323595A (en) | Distributed image preview browsing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160727 |